Implement support for weighted rrf

[mridula-s109]

Implement support for weighted RRF

Summary

This PR implements support for weighted RRF (Reciprocal Rank Fusion) retrievers, allowing users to specify custom weights for each sub-retriever within an RRF retriever configuration. This addresses a common customer request to customize the influence of different retrievers in the RRF scoring process.

Core Implementation

• Enhanced RRFRetrieverBuilder: Extended to support both weighted and non-weighted retriever configurations
• New RRFRetrieverComponent: Added weight validation and handling for individual retrievers
• Backward Compatibility: Maintains support for existing RRF configurations without weights (default weight = 1.0)

Key Features

1. Weighted Retriever Support

Users can now specify weights for individual retrievers:

{ "retriever": { "rrf": { "retrievers": [ { "retriever": { "standard": { "query": { "match": { "description": "pizza" } } } }, "weight": 0.7 }, { "retriever": { "knn": { "field": "vector", "query_vector": [1,2,3], "k": 10 } }, "weight": 0.3 } ] } } }

[github-actions]

🔍 Preview links for changed docs

• docs/reference/query-languages/esql/kibana/docs/functions/score.md
• docs/internal/GeneralArchitectureGuide.md
• docs/reference/elasticsearch/configuration-reference/security-settings.md
• docs/reference/elasticsearch/index.md
• docs/reference/elasticsearch/mapping-reference/sparse-vector.md
• docs/reference/elasticsearch/rest-apis/index.md
• docs/reference/elasticsearch/rest-apis/retrievers.md
• docs/reference/query-languages/esql/limitations.md
• docs/reference/query-languages/query-dsl/query-dsl-bool-query.md
• docs/reference/query-languages/query-dsl/query-dsl-knn-query.md

More links …

• docs/reference/query-languages/query-dsl/query-dsl-sparse-vector-query.md
• docs/reference/query-languages/query-dsl/query-dsl-text-expansion-query.md
• docs/reference/query-languages/query-dsl/query-dsl-weighted-tokens-query.md
• docs/reference/search-connectors/elastic-managed-connectors.md
• docs/release-notes/breaking-changes.md
• docs/release-notes/deprecations.md
• docs/release-notes/index.md
• docs/release-notes/known-issues.md

[mridula-s109]

@ioanatia have addressed all the comments. please let me know if there are any more concerns.

[ioanatia]

this is getting super close! thank you for your patience 🙇‍♀️

[ioanatia]

nice work!

[Mikep86]

LGTM, nice work!

[flobernd]

Edit: nvm, the old syntax seems to be still valid.

@mridula-s109 @leemthompo Hey folks. Just double checking here. Judging from the provided JSON example, this seems to be a breaking change since the structure now includes a new nesting:

{ "retriever": { "rrf": { "retrievers": [ { // <- Retriever container "standard": { "query": { "match": { "description": "pizza" } } } },

to

{ "retriever": { "rrf": { "retrievers": [ { // <- New RRFRetrieverComponent class that wraps `retriever` and `weigth` "retriever": { "standard": { "query": { "match": { "description": "pizza" } } } }, "weight": 0.7 },

Is this expected to ship in a minor version?