Remodeling search at Supply Hero: A migration journey to OpenSearch Service with radial search

Have you ever ever looked for one thing like “low fats yogurt” at any on-line grocery retailer and seen how the outcomes appear to grasp what you imply? As a substitute of solely displaying gadgets with an actual match, the top-ranked merchandise are sometimes semantically associated. You would possibly see gadgets like “Greek yogurt” or “yogurt with 0.5% fats,” even when just one phrase matches lexically. That is the facility of semantic search, and when mixed with conventional lexical search, it creates a hybrid search expertise that delivers each precision and recall.

At Supply Hero, one of many world’s main on-line meals supply platforms, the search crew has been utilizing semantic seek for grocery verticals since 2024. What began as a proof-of-concept has advanced right into a production-grade hybrid search system powered by Amazon OpenSearch Service. This method combines radial vector search with lexical retrieval to ship extremely related product outcomes at scale.

On this submit, we stroll by way of how Supply Hero migrated their semantic search infrastructure to Amazon OpenSearch Service, why they selected radial search over conventional k-nearest neighbor (k-NN) search, and the optimizations that made the system quick, cost-effective, and versatile for experimentation.

Legacy system overview

The unique semantic search system was constructed as a standalone service utilizing SpringBoot and Apache Lucene 9.9, deployed on Kubernetes. The retrieval move labored as follows:

1. A person begins a search on the applying.
2. The semantic search system retrieves the highest 50 nearest-neighbor candidates from a static in-memory Lucene index.
3. These candidates handed by way of a filtering layer to take away out-of-stock gadgets.
4. The filtered semantic outcomes have been merged with a parallel set of lexical search outcomes.
5. A ultimate rating step mixed each candidate units to supply the response.

The crew iterated on this technique over seven variations and carried out a number of A/B exams to refine the strategy. The preliminary system carried out properly, nonetheless because the enterprise scaled, a number of ache factors emerged:

• Scalability limitations: Working vector indices as static, in-memory buildings inside Kubernetes pods meant that scaling required provisioning bigger pods or including replicas. Each choices have been costly and operationally complicated.
• Multi-model experimentation was troublesome: Working A/B/C exams with three totally different product embedding mannequin variants required becoming all fashions inside a Kubernetes stateless workload. This created reminiscence strain and sophisticated deployment pipelines.
• Operational overhead: Managing index builds, deployments, and model rollouts for a customized Lucene-based service required important engineering effort in comparison with a managed service.

Structure modernization with OpenSearch Service

By the tip of 2025, Supply Hero had migrated their complete search infrastructure from self-managed Elasticsearch 7.x on Google Kubernetes Engine (GKE) to the totally managed Amazon OpenSearch Service 3.x. This migration created a pure alternative to consolidate the legacy semantic search service into OpenSearch as properly.

The brand new structure separates issues into two distinct pipelines: an ingestion pipeline for indexing product embeddings, and an inference pipeline for real-time hybrid retrieval.

Ingestion pipeline

For the ingestion pipeline, Supply Hero selected Amazon OpenSearch Ingestion (OSIS) to sync product embedding information from Amazon Easy Storage Service (Amazon S3) to the OpenSearch area.

The move works as follows:

1. ML mannequin
2. Airflow job: An present Apache Airflow job periodically generates product embeddings utilizing an exterior machine studying (ML) mannequin and periodically dumps the outcomes (product father or mother ID + embedding vector) to an S3 bucket.
3. OpenSearch Ingestion pipeline: An OpenSearch Ingestion pipeline is configured with a scheduled S3 scan that performs a nightly scan from S3 and updates the brand new k-NN index in OpenSearch Service.

model: '2'
embedding-pipeline:
  supply:
    s3:
      acknowledgments: true
      scan:
        buckets:
          - bucket:
              identify: my-bucket-name
              filter:
                include_prefix:
                  - vector-search/json-index/newest
        vary: PT24H
        scheduling:
          interval: PT24H
      aws:
        area: eu-central-1
        sts_role_arn: arn:aws:iam:::position/osis-pipeline-role
      codec:
        ndjson: {}
      compression: none
  staff: '1'
  sink:
    - opensearch:
        hosts:
          - "https://..es.amazonaws.com"
        aws:
          serverless: false
          area: eu-central-1
          sts_role_arn: arn:aws:iam:::position/search-xxx
        index_type: customized
        index: emb_products_v1
        template_content: ...
        template_type: index-template
        routing: '${global_entity_id}'
        document_id: '${global_entity_id}:${master_code}'
        max_retries: '3'

As a result of the index shops product father or mother IDs and embeddings are regenerated in batch, there is no such thing as a want for real-time updates. This enables the crew to refresh and force-merge the index as soon as per day, leading to extremely optimized phase buildings and quick retrieval speeds (p99

Establishing the OSIS pipeline required just a few traces of Terraform, making it simple to provision and preserve as infrastructure-as-code.

Inference pipeline

On the retrieval aspect, the system runs a hybrid search technique that mixes radial vector search with lexical search in parallel:

1. Question embedding: A person’s search question first reaches the Question Understanding (QU) service, the place it’s encoded into an embedding utilizing the identical dwell ML mannequin employed for product embeddings. To optimize efficiency, embeddings for prime queries are cached.
2. Parallel lexical and semantic retrieval:
   • A radial k-NN search runs in opposition to the product embeddings index utilizing min_score to retrieve all semantically related merchandise above a similarity threshold.
   • A lexical BM25 search runs in opposition to the product catalog index.
     

     Evaluating p95 OpenSearch time for each lexical and semantic search.

3. ID decision and stock filter: As a result of the k-NN index shops product father or mother IDs, a decision step maps these to particular person product IDs by way of a secondary index that maintains close to real-time stock updates. This strategy satisfies two key enterprise necessities inside a single retrieval name: product-id decision and real-time availability filtering.
4. Merge and re-rank: A customized post-processing step combines outcomes from each lexical and radial search, applies re-ranking logic, and returns the ultimate end result set.

Why radial search?

Conventional k-NN search in OpenSearch makes use of a top-k strategy: you ask for the okay nearest neighbors, and also you get precisely okay outcomes no matter how related they really are. This works properly for a lot of use circumstances, nevertheless it has a basic limitation for product search. It all the time returns a set variety of outcomes, even when a few of these outcomes usually are not semantically related.

Radial search solves this by flipping the paradigm. As a substitute of asking “give me the 50 closest gadgets,” you ask “give me all gadgets which are a minimum of this related.” That is executed utilizing the min_score parameter within the k-NN question:

GET product-embeddings/_search
{
  "question": {
    "knn": {
      "embedding": {
        "vector": [0.12, 0.45, 0.78, ...],
        "min_score": 0.72
      }
    }
  }
}

When utilizing radial search with cosine similarity because the area sort, OpenSearch normalizes scores utilizing the associated components (rating = (1 + cosine_similarity) / 2), as documented within the OpenSearch knn-spaces reference.

This implies a min_score of 0.72 within the question instance, doesn’t immediately correspond to cosine similarity. As a substitute, 0.72 is the normalized OpenSearch rating which interprets to 44% cosine similarity (that’s, cosine_similarity = 2 × 0.72 – 1 = 0.44).

In case you want outcomes with a minimum of 90% cosine similarity, apply the components:

min_score = (1 + 0.90) / 2 = 0.95. So, you’d set “min_score”: 0.95 in your question.

This strategy gives a number of benefits for product search:

• High quality over amount: Low-relevance outcomes are excluded on the retrieval stage relatively than counting on downstream re-ranking to filter them out.
• Variable end result set measurement: The system naturally adapts to question specificity. Area of interest queries return fewer, extra exact outcomes. Broad queries return extra candidates for the re-ranker to work with. For instance, a extremely particular question like “Oatly oat milk barista version” would possibly return 5 outcomes, whereas a broader question like “milk” would possibly return 200.
• Higher recall-precision trade-off: By tuning the min_score threshold, the crew can immediately management the stability between returning too many irrelevant outcomes and lacking related ones.

How Supply Hero chosen the brink for radial search

Choosing the proper min_score threshold is necessary. Set it too excessive and also you miss related merchandise. Set it too low and also you flood the re-ranker with noise.

Supply Hero approaches threshold choice by way of systematic experimentation. To attain optimum precision throughout numerous markets, a tailor-made min_score threshold is assigned to every nation and question sort. These thresholds are meticulously decided by way of rigorous offline evaluations, which use historic person interplay and manually labeled information to determine a tough estimate. This preliminary estimate is then additional refined and validated by way of a collection of dwell A/B experiments.

Analysis of the brand new search system

One of many key benefits of the brand new structure is how naturally it helps experimentation. At Supply Hero, we retailer three variants of product embeddings inside a single doc:

PUT product-embeddings/_doc/1?routing=FP_DE
{
  "master_product_code": "abc123",
  "embedding_variant_1": [0.12, 0.45, 0.78, ...],
  "embedding_variant_2": [0.21, 0.4, 0.98, ...],
  "embedding_variant_3": [0.13, 0.65, 0.58, ...],
  "global_entity_id": "FP_DE"
}

On this instance, embedding_variant_1, embedding_variant_2, and embedding_variant_3 are generated from three totally different fashions for A/B/C testing. After every check, the profitable variant is designated because the management, whereas the opposite two are changed with new fashions for additional experimentation. With this strategy, the crew can iterate repeatedly whereas sustaining fixed area complexity.

Optimizations of enormous scale manufacturing system

Engine improve: OpenSearch 2.17 to three.3

Manufacturing metrics from one of many busiest nations.

OpenSearch 3.x launched important efficiency enhancements for vector search workloads. Put up-upgrade to OpenSearch 3.3, we noticed a ~18% discount in p95 latency for k-NN queries.

For Supply Hero’s use case, the k-NN search latency was already very low on OpenSearch 2.17 (p99 of 20–30 ms), which meant the improve to three.3 was not strictly obligatory for all clusters. The cluster serving the management group in A/B exams nonetheless runs on OpenSearch 2.17.

Shard routing

To attenuate cross-shard overhead throughout k-NN queries, Supply Hero carried out customized shard routing primarily based on geographic market. As a result of every market (for instance, Germany, Sweden, and Finland) has its personal product catalog, routing queries to market-specific shards avoids pointless fan-out throughout your entire index.

That is an instance of how one can configure routing at index time and search time utilizing the _routing discipline:

PUT product-embeddings/_doc/1?routing=FP_DE
{
  "master_product_code": "abc123",
  "embedding_variant_1": [0.12, 0.45, 0.78, ...],
  "embedding_variant_2": [0.21, 0.4, 0.98, ...],
  "embedding_variant_3": [0.13, 0.65, 0.58, ...],
  "global_entity_id": "FP_DE"
}

And at question time:

GET product-embeddings/_search?routing=FP_DE
{
  "question": {
    "knn": {
      "embedding_variant_2": {
        "vector": [0.12, 0.45, 0.78, ...],
        "min_score": 0.72
      }
    }
  }
}

This ensures {that a} question for the German market solely hits shards containing German merchandise, lowering latency and compute overhead.

Refresh interval

As a result of the product embedding index is up to date solely as soon as per day by way of the OSIS batch pipeline, there is no such thing as a want for the default 1-second refresh interval. Supply Hero configured the index with an extended refresh interval throughout ingestion and triggers a handbook refresh + power merge after the nightly batch completes.

Affect on the enterprise

The migration from self-managed Lucene on Kubernetes to Amazon OpenSearch Service achieved a ~50% discount in p95 latency, dropping response instances from a variable 200ms+ to a secure 100ms baseline. This transition considerably improved system consistency by eliminating the excessive variance and rhythmic latency spikes seen within the earlier structure.

Finish service latency after rolling out semantic search with OpenSearch for foodpanda and yemeksepeti.

Past uncooked latency, the operational advantages have been important:

• Decreased infrastructure complexity: Eliminating the standalone Lucene service eliminated a complete deployment pipeline, monitoring stack, and on-call rotation.
• Quicker experimentation: New embedding fashions could be examined by creating a brand new index and adjusting question routing, with out requiring code deployments.
• Value effectivity: Utilizing OpenSearch’s managed infrastructure and the batch ingestion sample (refresh as soon as per day) lowered compute prices in comparison with operating always-on Kubernetes pods with in-memory indices.

Conclusion

By combining radial search with lexical retrieval, Supply Hero’s crew constructed a system that adapts dynamically to question intent. It returns exact outcomes for particular queries and broader candidate units for common ones.

The migration to Amazon OpenSearch Service demonstrates how a managed search platform can simplify the operational complexity of vector search whereas bettering efficiency.

To get began with vector search on Amazon OpenSearch Service, see the AI search documentation and the OpenSearch radial search information.

In regards to the authors