How Alight Options achieved 55% price financial savings with Amazon OpenSearch Service

This can be a visitor publish by Mark Larson, Andrew Kummerow, and Tim Razik at Alight Options, in partnership with AWS.

Alight Options is a number one cloud-based human capital expertise and providers supplier targeted on built-in advantages administration, healthcare navigation, and worker expertise options. The corporate serves lots of of enterprise prospects globally, with providers that assist thousands and thousands of individuals worldwide.

Alight’s expertise stack generates over 1 billion log data per day throughout their containerized microservices structure, with peaks reaching 100,000 data per second throughout Annual Enrollment durations. Beforehand, Alight relied on a self-managed Elastic Stack (Elasticsearch, Logstash, Kibana) deployment that had been in manufacturing since 2018. As their logging volumes grew and Elasticsearch 7.x approached finish of assist, the operational burden of sustaining this infrastructure consumed their whole operational price range, leaving no capability for innovation.

On this publish, we share how Alight Options migrated from self-managed Elasticsearch to Amazon OpenSearch Service. The migration achieved a 55% price discount, alleviated roughly 2,000 hours per yr of operational overhead, and gave Alight entry to superior observability options they might not prioritize earlier than.

Challenges with self-managed Elasticsearch

Alight’s self-managed Elastic Stack infrastructure introduced compounding technical and operational challenges. Their manufacturing atmosphere consisted of 15 Elasticsearch nodes with 168 TB of EBS storage, dealing with log ingestion from their flagship Alight Worklife system and supporting functions. The infrastructure required an Elastic Platinum subscription, although the staff’s operational bandwidth was totally consumed by upkeep, leaving restricted capability to undertake superior options included within the license.

The operational ache factors included:

• Safety vulnerability patching required working over Christmas holidays to handle vital fixes, with no flexibility on timing.
• Elastic upgrades have been time-consuming and required depth of information to handle at scale.
• Logstash utilizing TCP-socket delivery was unreliable, experiencing log loss at excessive volumes.
• Backpressure from Logstash precipitated two P1 incidents over two years, the place the logging subsystem straight impacted microservice duties.
• Elasticsearch 7.x approaching finish of assist created urgency to behave earlier than the subsequent Annual Enrollment interval (September by January).

Alight was spending greater than $100,000 per thirty days on self-managed infrastructure and Elastic licensing throughout all environments. All operational price range was consumed by cluster upkeep, leaving zero capability for innovation.

Evaluating options

Alight evaluated a number of options earlier than choosing OpenSearch Service:

• New Relic and Dynatrace have been evaluated for log aggregation however proved prohibitively costly at Alight’s quantity.
• Amazon CloudWatch was evaluated however didn’t meet necessities for complicated log analysis at their quantity and visualization complexity.

Amazon OpenSearch Service is a managed service that makes it simple to deploy, function, and scale OpenSearch clusters within the AWS Cloud. You should use it to be used instances equivalent to log analytics and real-time utility monitoring. It provisions cluster assets, mechanically detects and replaces failed nodes, and scales with a single API name or a number of clicks, decreasing the operational overhead related to self-managed infrastructure. It received the analysis primarily based on 5 components:

1. Value: considerably cheaper than self-managed Elastic Stack and competing options.
2. Minimal change administration: as a fork of Elasticsearch 7.10, engineers have been already conversant in the question syntax and dashboards.
3. Compliance: utilizing a local AWS service prevented lots of of hours of vendor compliance, audit, and regulatory work. The staff spent a number of hours getting approval in comparison with probably weeks for an exterior vendor.
4. Cloud-native technique: aligned with Alight’s overarching technique to make use of cloud-native providers.
5. Safety and knowledge privateness: conserving all the pieces inside their AWS touchdown zone alleviated knowledge egress issues.

Answer overview

Alight partnered with AWS to design a cloud-native log aggregation structure that changed self-managed Elasticsearch and Logstash with Amazon OpenSearch Service and Amazon OpenSearch Ingestion (OSIS), assuaging the operational burden, together with the Logstash backpressure that had precipitated two P1 incidents.

The structure makes use of a cross-account mannequin with two main account sorts:

The next diagram illustrates the answer structure.

Ingestion paths

The answer helps a number of ingestion paths relying on the appliance internet hosting mannequin:

• ECS functions: FireLens/Fluent Bit sidecar containers seize stdout/stderr by the awsfirelens log driver, then ship logs over HTTPS on to OSIS within the shared providers account. ECS activity roles assume a cross-account OSIS Ingest Function for authentication.
• EC2 functions: Open-source Fluent Bit (RPM-based, non-containerized) makes use of tail enter to learn log information, then ships to OSIS by an EC2 IAM Function with cross-account belief.
• S3-based ingestion (deliberate): Some functions write to Amazon Easy Storage Service (Amazon S3) with Amazon Easy Queue Service (Amazon SQS) notifications triggering OSIS pipelines.

Spring Boot microservices use a customized logging framework constructed on Logback (not Log4j) that codecs logs as JSON and flushes to console, which FireLens picks up.

Safety mannequin

Site visitors flows over HTTPS. The safety mannequin makes use of position separation with least privilege:

• OSIS Ingest Function: write-only entry to OSIS pipelines, assumed by utility account roles through cross-account belief.
• OSIS Sink Function: utilized by OSIS to write down into the OpenSearch area, with full index entry scoped to the ingestion pipeline.
• Safety teams: limit OSIS site visitors to recognized CIDRs and VPCs.

Every utility has its personal indices, and entry is ruled by application-specific roles.

Persistent buffering

Amazon Elastic File System (Amazon EFS) supplies persistent filesystem buffering for the Fluent Bit sidecar, serving to stop log loss throughout transient failures or backpressure occasions. This straight addresses the P1 incidents Alight skilled with Logstash. For the subsequent Annual Enrollment interval, Alight plans to additionally allow persistent buffering on the OSIS layer to deal with burst ingestion with out log loss.

Consumer entry

Finish-user entry to OpenSearch Dashboards is managed by AWS IAM Id Heart with System for Cross-domain Id Administration (SCIM) synchronization from Alight’s enterprise Id Supplier. Customers navigate to the Purposes tab in Id Heart to entry OpenSearch Dashboards over SAML/HTTPS.

At Alight, IAM Id Heart and SCIM are configured within the payer account. They use the identical synchronization and entitlement request and approval course of that governs Alight’s person and entitlement provisioning into AWS. With this setup, the staff makes use of the identical single sign-on (SSO) and entitlement workflow for OpenSearch Dashboards entry as for the AWS Administration Console, together with fine-grained entry management (FGAC) outlined throughout the OpenSearch domains.

OpenSearch area configuration

For his or her manufacturing workload, Alight deployed:

Part	Configuration
Information nodes	18 im4gn.2xlarge.search
UltraWarm nodes	9
Devoted chief nodes	3
Scorching tier storage	25 TB
UltraWarm storage	180 TB
Major logical knowledge	80 TB
Complete with replicas	100-105 TB

Extra environments embrace a secondary manufacturing cluster (12 scorching nodes, 3 UltraWarm, 3 devoted chief nodes), plus consumer take a look at and engineering clusters with 3 scorching nodes every.

Migration course of

The migration was accomplished over seven months (February by August 2025), with 5 functions migrated together with the flagship Alight Worklife utility.

Infrastructure as code

The staff constructed new Terraform modules to handle deployment of OSIS pipelines, OpenSearch domains, and FireLens sidecar additions to ECS functions. Onboarding new functions is now templatized, leading to important time financial savings in comparison with including new indices in Elasticsearch. Onboarding a brand new utility now takes between 4-8 hours, whereas earlier than we’d spend 80-120 hours per utility.

Migration timeline

Alight first enabled Amazon OpenSearch Service in manufacturing for 2 smaller functions, to verify operational processes have been up and operating earlier than migrating the best quantity log producers. For every utility, logging to OpenSearch was enabled whereas persevering with to write down logs to the prevailing logging infrastructure. This parallel run allowed fine-tuning of OSIS pipeline configuration, OpenSearch cluster measurement and configuration earlier than doing a full cutover. This strategy additionally validated that logs have been being ingested correctly into OpenSearch. It confirmed that the efficiency of OpenSearch Dashboards and queries was nearly as good as or higher than the prevailing self-managed Elasticsearch cluster.

For historic knowledge, Alight migrated the newest 30 days of stay knowledge from Elasticsearch into OpenSearch simply previous to cutover. Additionally they retained a full archive of older log knowledge in an Amazon S3 bucket, in order that knowledge older than 30 days may very well be loaded into OpenSearch on request if a person wants it.

AWS partnership

Alight engaged the AWS staff in the course of the analysis section. By way of AWS Enterprise Help, their Technical Account Supervisor (TAM) served because the devoted level of contact all through the journey. The TAM coordinated classes with OpenSearch Service material consultants to handle particular service capabilities, assist with design, troubleshoot points, and supply efficiency steerage.

Outcomes

The migration to Amazon OpenSearch Service delivered outcomes throughout price, operations, and functionality dimensions.

“Alight’s mission vital functions are constructed on lots of of interdependent microservices, so efficient utility logging is vital for analyzing system behaviors, efficiency tuning, and troubleshooting. Amazon OpenSearch Service supplies us with nice log analytics, very affordably at scale, and integrates seamlessly with our IAM technique for granular entry management and authorization. The power to reconfigure, resize, and improve OpenSearch domains with a number of clicks and 0 downtime is a recreation changer for us.”

— Mark Larson, Enterprise Architect

Value and licensing

Metric	Earlier than	After	Enchancment
Month-to-month infrastructure + licensing price	Self-managed EC2/EBS + Elastic Platinum licensing	Absolutely managed OpenSearch Service, no separate licensing	~55% price discount
Licensing mannequin	Elastic Platinum (mounted)	Zero licensing price	Now not wanted

Not all Elasticsearch clusters are decommissioned but. As soon as decommissioning is full, financial savings will attain roughly 65%. Moreover, extra functions have been added to OpenSearch than have been initially on Elasticsearch, making the per-application price much more favorable. Past compute and licensing, the migration additionally decreased knowledge switch prices beforehand incurred throughout the self-managed cross-account structure, including additional to the general financial savings.

Operational enhancements

Metric	Earlier than	After
Engineering hours on cluster administration	2,000 hours/yr (≈1 FTE)	Close to zero (managed service)
Safety vulnerability patching	Guide, together with vacation work	Dealt with by AWS
Utility onboarding	Guide index creation and configuration	Templatized through Terraform
P1 incidents from logging subsystem	2 in previous 2 years	Zero since migration

Efficiency and scale

Metric	Worth
Each day log quantity	1 billion data
Peak ingestion price	100,000 data/second
Purposes migrated	5 (together with Alight Worklife)
Complete knowledge underneath administration	100–105 TB with replicas

Classes realized and greatest practices

By way of their migration journey, Alight gained the next insights:

• Use your account staff relationship to advocate: When Fluent Bit had a blocking subject, the AWS account staff relationship helped push for the repair and supplied workaround steerage.
• Separate issues for knowledge sturdiness: Don’t put 100% supply ensures on logging infrastructure. Use a separate occasion stream (equivalent to Amazon SQS) for vital knowledge that can’t tolerate loss.
• Templatize all the pieces: Terraform modules for OSIS, OpenSearch domains, and FireLens sidecars cut back the time to onboard new functions.
• Safety structure issues: Separating ingest roles from sync roles (least privilege) and utilizing cross-account belief supplies sturdy safety with out complexity.
• Plan round business-critical durations: Pausing the manufacturing rollout throughout Annual Enrollment was the appropriate name. The chance of introducing adjustments throughout peak was not definitely worth the schedule strain.

What’s subsequent

Alight has a number of initiatives deliberate to increase their OpenSearch Service utilization:

• Anomaly detection: prime precedence, a characteristic they paid for with Elastic Platinum however by no means had capability to implement.
• Amazon OpenSearch Serverless: evaluating for brand new log sources, notably eager about zero-OCU baseline for price optimization.
• OSIS persistent buffer: deliberate for subsequent Annual Enrollment to deal with burst ingestion with out log loss.
• Amazon Bedrock AgentCore logging: new synthetic intelligence (AI) workloads will ship logs to OpenSearch.
• AI-assisted log analytics: adopting the agentic AI capabilities now constructed into Amazon OpenSearch Service. These embrace the Investigation Agent for autonomous, hypothesis-driven root trigger evaluation, which helps website reliability engineering (SRE) and engineering groups acquire deeper insights from utility logs.
• Vector database: already utilizing OpenSearch as a vector retailer for a conversational AI assistant (separate staff).
• Migration progress: All workloads beforehand logging to Elasticsearch have been migrated to OpenSearch, plus an extra eight functions.
• Enterprise Logging Service: All new functions will now log to Amazon OpenSearch Service by default utilizing the templatized strategy.
• Decommission: All current Elasticsearch cases can be decommissioned by July 2026.

Conclusion

Alight’s migration from self-managed Elasticsearch to Amazon OpenSearch Service demonstrates how enterprises can alleviate operational burden whereas attaining important price financial savings. Through the use of Amazon OpenSearch Ingestion and FireLens, Alight constructed a scalable log aggregation system that handles 1 billion data per day with zero P1 incidents since deployment.

The 55% price discount and roughly 2,000 hours per yr of recovered engineering time have freed Alight to pursue superior observability capabilities like anomaly detection and AI-powered log analytics, options they paid for however might by no means use underneath the operational weight of self-managed infrastructure.

To study extra, see the Amazon OpenSearch Service documentation. To get began with ingestion pipelines, see Amazon OpenSearch Ingestion. For migration steerage, see Migrating to Amazon OpenSearch Service.

---

In regards to the authors