Managing petabytes of search knowledge means making robust decisions: hold all the things quick and costly, or make it inexpensive however read-only. UltraWarm is a confirmed, cost-effective answer for read-heavy historic knowledge. Nevertheless, some workloads sometimes have to replace historic data, akin to late-arriving knowledge or compliance corrections. With UltraWarm, you have to migrate these indices again to scorching, carry out the replace, and migrate again. What should you might write on to your cost-effective heat storage as an alternative?
On this put up, I present you the way writable heat storage removes the pricey migration cycle. You possibly can scale back your infrastructure prices by as much as 48 p.c and replace historic knowledge in seconds as an alternative of hours. I stroll by means of a real-world value comparability and efficiency benchmarks, and provide help to resolve when to make use of writable heat versus UltraWarm.
The problem with tiered storage
Amazon OpenSearch Service handles data-intensive search and analytics workloads, from real-time log analytics and utility monitoring to safety occasion detection. As your knowledge volumes develop from terabytes to petabytes, you face a elementary query: how do you retain current knowledge quick whereas making earlier knowledge inexpensive?
OpenSearch Service addresses this with a tiered storage structure:
- Sizzling – Highest efficiency for lively indexing and search utilizing instance-attached storage.
- UltraWarm – Price-effective, read-only tier backed by Amazon Easy Storage Service (Amazon S3) with native caching for much less continuously queried knowledge.
- Chilly – Totally indifferent from the cluster, with the bottom value for not often accessed knowledge. Chilly indices should be migrated again to UltraWarm or scorching earlier than any reads or writes might be carried out.
For immutable log knowledge, this mannequin works properly. Nevertheless, a selected class of workloads hits its limitations after they sometimes want to put in writing to earlier knowledge, and read-only turns into a bottleneck.
Conditions
To make use of writable heat storage, you want the next:
- An Amazon OpenSearch Service area operating model 3.3 or later.
- OpenSearch Optimized (OI2) occasion household help in your AWS Area.
- Workloads with a minimal 5-second refresh interval.
- Information nodes utilizing the OpenSearch Optimized occasion household (OR2 for decent, OI2 for heat).
Word: Writable heat doesn’t presently help the chilly storage tier.
The UltraWarm bottleneck
With UltraWarm, updating even a single doc requires migrating the index again to scorching, performing the write, and migrating it again. This spherical journey entails a drive merge (consolidating index segments), snapshot creation, and shard relocation. These operations eat important CPU, reminiscence, and disk house in your scorching nodes, they usually take roughly 130 minutes per 100 GB index. This time was measured on a site with 3 × r6g.2xlarge scorching nodes, 3 × ultrawarm1.massive heat nodes, and three devoted chief nodes (US East, N. Virginia), utilizing a single-shard index with one reproduction. Precise instances fluctuate based mostly on area configuration, shard rely, phase rely, scorching node utilization, and migration queue depth. The result’s that you simply over-provision scorching nodes, construct complicated pipelines, or hold knowledge in scorching longer than obligatory, which will increase value and complexity.
Introducing writable heat storage
OpenSearch Service now presents writable heat nodes that use OpenSearch Optimized (OI2) situations, the identical occasion household that powers sturdy, Amazon S3-backed storage on scorching nodes. As a result of knowledge is already persevered on Amazon S3, tier transitions turn into a light-weight shard relocation fairly than a resource-intensive migration. The Lucene engine, which is OpenSearch’s underlying search library, operates identically on each tiers. Consequently, writable heat nodes help lively writes, background merges, and periodic refreshes, similar to scorching nodes.
Late-arriving knowledge, compliance backfills, and corrections that beforehand required a warm-to-hot-to-warm spherical journey now resolve with a direct write in seconds. There isn’t any drive merge, no snapshot, no shard relocation, and no scorching node useful resource consumption.
UltraWarm (legacy) knowledge move: Information is ingested into the recent tier (SSD, learn and write). Index State Administration (ISM) insurance policies migrate indices to UltraWarm (Amazon S3-backed, read-only). Any replace requires migrating the index again to scorching (dashed arrow), writing, then migrating again.
Writable heat (new) knowledge move: Similar ingestion path by means of scorching, with ISM transitioning indices to writable heat. The important thing distinction is that writable heat helps each reads and writes. Late-arriving updates go on to heat, with no migration again to scorching. As a result of each tiers use Amazon S3 as sturdy storage by means of OpenSearch Optimized situations, transitions are light-weight shard relocations, not resource-intensive migrations.
The advantages: value, operations, and adaptability
Writable heat delivers benefits in three areas: value, operational simplicity, and adaptability.
Price
In contrast to UltraWarm, which solely presents on-demand pricing, OI2 situations help Reserved Occasion (RI) pricing, a commitment-based low cost mannequin. By committing to a 1-year or 3-year Reserved Occasion, it can save you 31–52 p.c in comparison with UltraWarm nodes. This makes writable heat considerably less expensive for predictable, long-running workloads. The newly launched Database financial savings plan for OpenSearch Service offers financial savings of round 22 p.c over UltraWarm situations. Each tiers use Amazon S3 for sturdy storage, so node failure means solely non permanent unavailability, not knowledge loss. For cost-sensitive workloads that may tolerate transient downtime throughout node restoration, you may configure zero replicas on heat indices to cut back prices additional.
Actual-world value comparability
Think about a workload ingesting 2 TB/day with 210 days complete retention, the place updates can arrive at any level. With UltraWarm’s read-only constraint, you have to hold knowledge in scorching for 30 days earlier than migrating to heat. With writable heat, updates occur instantly on heat, so scorching retention drops to solely 7 days.
At small scale, the recent tier discount profit is modest. Writable heat remains to be cost-effective should you want write functionality on heat knowledge, can decide to RI pricing, or worth the operational simplicity of eliminating migration pipelines. For purely immutable knowledge with brief retention, UltraWarm on-demand would possibly nonetheless be cheaper. Use the AWS Pricing Calculator to mannequin your particular situation.
The next desk exhibits estimated month-to-month prices utilizing on-demand and All Upfront Reserved Occasion (AURI) pricing within the US East (N. Virginia) Area as of March 2026. For the newest pricing, see Amazon OpenSearch Service pricing on the AWS web site.
| Part | Sizzling + UltraWarm (30d scorching / 180d heat) | Sizzling + writable heat (7d scorching / 203d heat) |
| Sizzling knowledge nodes | $12,264 (21 × or2.2xlarge) | $12,264 (21 × or2.2xlarge) |
| Sizzling EBS value | $10,212.84 (21 * 3986 GB) | $2,636 |
| Sizzling distant storage | $2,008.28 | $518 |
| Heat knowledge nodes | $39,128 (20× ultrawarm1.massive) | $50,409 (15× oi2.8xlarge) |
| Amazon S3 storage | $9,504 | $1,070 |
| Chief nodes | $1,307 (3 × m8g.2xlarge) | $1,307 (3 × m8g.2xlarge) |
| On-demand complete | $74,427 | $69,297 |
| 1-year AURI | $69,674 | $43,918 (~36% much less) |
| 3-year AURI | $67,367 | $34,939 (~48% much less) |
| Database financial savings plan | $71,708 | $55,406 (~22%) |
Operations
Reclaim scorching node capability. Writable heat removes two widespread causes of scorching node over-provisioning: reserving 35 p.c of disk house for drive merge operations, and sustaining additional capability to quickly transfer knowledge again to scorching for writes. You possibly can run your scorching tier at greater utilization, which reduces the variety of scorching nodes you want.
Less complicated migrations. UltraWarm migrations are multi-step operations (drive merge, snapshot, and shard relocation) that want cautious scheduling throughout low-traffic home windows, and they’re restricted to 10 queued at a time. Writable heat simplifies this to a light-weight shard relocation, with extra easy ISM insurance policies and no scheduling constraints.
Flexibility
UltraWarm presents solely two occasion sizes: ultrawarm1.medium (1.5 TiB) and ultrawarm1.massive (20 TiB). Writable heat with OI2 situations presents a full vary from oi2.massive to oi2.16xlarge. Every dimension addresses as much as 5× its native cache dimension, so you may right-size heat capability exactly to your workload.
Search efficiency
We benchmarked search latency utilizing the NYC Taxis workload, evaluating writable heat (oi2.massive) towards UltraWarm nodes. All measurements are P90 latencies.
On the NYC_TAXIS benchmark, writable heat matched or beat UltraWarm on 6 of seven question sorts at P90, together with light-weight filters, ranges, types, and time-histogram aggregations. For many real-world search patterns, writable heat delivers comparable or higher efficiency than UltraWarm, plus the flexibility to put in writing on to the tier.
Search efficiency: writable heat in comparison with UltraWarm
| Process | Writable heat node latency in ms | UltraWarm latency in ms | UltraWarm vs. writable heat diff % |
| NYC_TAXIS workload kind | ** ** | ** ** | ** ** |
| default (P90) | 21.287 | 23.857 | 12.07223 |
| vary (P90) | 21.23 | 21.016 | -1.00718 |
| distance_amount_agg (P90) | 5,069 | 3929.23 | -22.48406 |
| autohisto_agg (P90) | 21.076 | 22.002 | 4.39348 |
| date_histogram_agg (P90) | 21.363 | 21.792 | 2.01031 |
| desc_sort_tip_amount (P90) | 23.224 | 23.797 | 2.46636 |
| asc_sort_tip_amount (P90) | 22.483 | 22.482 | -0.00445 |
When to decide on what
Do you have to swap from UltraWarm to writable heat? It relies on your workload.
| Requirement | Writable Heat | UltraWarm |
| Write enabled | ✓ | Learn-only |
| Reserved Occasion pricing | ✓ | ✗ |
| Occasion dimension flexibility | Big selection (massive–8xlarge) | 2 choices solely |
| Chilly tier help | ✗ | ✓ |
| Want for OpenSearch Optimized occasion households | ✗ | ✓ |
| Concurrent tier transitions | ✓ | ✗ (sequential) |
| Sizzling node affect throughout migration | Minimal | Excessive (CPU/reminiscence) |
Clear up assets
In the event you created a take a look at area to guage writable heat storage, delete it to keep away from ongoing fees. Within the OpenSearch Service console, choose your area and select Delete. This removes all nodes and stops Amazon S3 storage fees for that area.
Abstract
On this put up, I confirmed you the way writable heat storage eliminates the pricey migration cycle that UltraWarm’s read-only limitation creates. You rise up to 36 p.c value financial savings with 1-year Reserved Cases, quicker search efficiency, and a less complicated operational mannequin. Writable heat additionally removes knowledge transitions between tiers, and Reserved Occasion pricing turns into obtainable for heat storage for the primary time.
Writable heat requires OpenSearch Service model 3.3 or later with OI2 situations. For domains needing chilly tier help, earlier OpenSearch Service variations, or non-optimized occasion households, UltraWarm stays the proper alternative.
Subsequent steps: Begin by analyzing your present scorching and heat break up. What number of days of information do you retain in scorching solely to accommodate occasional updates? Use the AWS Pricing Calculator to mannequin your potential financial savings, and allow writable heat on a take a look at area in minutes. On the time of this put up, writable heat is supported on OpenSearch Service model 3.3. For step-by-step directions, see Migrating to writable heat storage within the OpenSearch Service documentation.
Have you ever tried writable heat storage? I’d love to listen to about your expertise and any questions you may have within the feedback.
Concerning the writer