Closed
Description
Expected:
When I enable hybrid and im using Elasticsearch v8.14+, I should be able to retrieve documents from both a bm25 based search and NN search using dense/sparse.
Actual:
Cannot perform the search as RRF is not supported
Reason:
Hybrid performs two retrieval searches and relies on RRF to balance the different result scores. In 8.14+, top level rank was removed and now the only way to do RRF is through retrievers query.
This is the code that builds the query that retrieves results from different methods and balances with RRF. link
suggested untested code change to equilivent retriever is:
{
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"match": {
self.text_field: {
"query": query,
}
}
}
}
},
{
"standard": {
"query": {
"knn": knn
}
}
}
]
}
}
}
Full example of a query:
{
"retriever": {
"rrf": {
"retrievers": [
{
"standard": {
"query": {
"multi_match": {
"query": "{query}",
"fields": [
"content"
]
}
}
}
},
{
"standard": {
"query": {
"knn": {
"field": "content_embedding",
"num_candidates": 100,
"query_vector_builder": {
"text_embedding": {
"model_id": "openai_embeddings",
"model_text": "{query}"
}
}
}
}
}
}
]
}
}
}