Skip to content

Commit f79cd01

Browse files
richlandermaumar
richlander
and
maumar
authored
full-text / hybrid search docs (#5009) (#5030)
Added note about vector search going out of preview Updated the vector search docs as well Co-authored-by: Maurycy Markowski <[email protected]>
1 parent 5b60a20 commit f79cd01

File tree

3 files changed

+220
-20
lines changed

3 files changed

+220
-20
lines changed

‎entity-framework/core/providers/cosmos/full-text-search.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
---
2+
title: Full Text Search - Azure Cosmos DB Provider - EF Core
3+
description: Full text search with the Azure Cosmos DB EF Core Provider
4+
author: maumar
5+
ms.date: 04/19/2025
6+
uid: core/providers/cosmos/full-text-search
7+
---
8+
# Full text search
9+
10+
Azure Cosmos DB now offers support for [full-text search](/azure/cosmos-db/gen-ai/full-text-search). It enables efficient and effective text searches using advanced techniques like stemming, as well as evaluating the relevance of documents to a given search query. It can be used in combination with vector search (i.e. hybrid search) to improve the accuracy of responses in some AI scenarios.
11+
EF Core allows for modeling the database with full-text search enabled properties and using full-text search functions inside queries targeting Azure Cosmos DB.
12+
13+
## Model configuration
14+
15+
A property can be configured inside `OnModelCreating` to use full-text search by enabling it for the property and defining a full-text index:
16+
17+
```c#
18+
public class Blog
19+
{
20+
...
21+
22+
public string Contents { get; set; }
23+
}
24+
25+
public class BloggingContext
26+
{
27+
...
28+
29+
protected override void OnModelCreating(ModelBuilder modelBuilder)
30+
{
31+
modelBuilder.Entity<Blog>(b =>
32+
{
33+
b.Property(x => x.Contents).EnableFullTextSearch();
34+
b.HasIndex(x => x.Contents).IsFullTextIndex();
35+
});
36+
}
37+
}
38+
```
39+
40+
> [!NOTE]
41+
> Configuring the index is not mandatory, but it is recommended as it greatly improves performance of full-text search queries.
42+
43+
Full-text search operations are language specific, using American English (`en-US`) by default. You can customize the language for individual properties as part of `EnableFullTextSearch` call:
44+
45+
```c#
46+
protected override void OnModelCreating(ModelBuilder modelBuilder)
47+
{
48+
modelBuilder.Entity<Blog>(b =>
49+
{
50+
b.Property(x => x.Contents).EnableFullTextSearch();
51+
b.HasIndex(x => x.Contents).IsFullTextIndex();
52+
b.Property(x => x.ContentsGerman).EnableFullTextSearch("de-DE");
53+
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
54+
});
55+
}
56+
```
57+
58+
You can also set a default language for the container - unless overridden in the `EnableFullTextSearch` method, all full-text properties inside the container will use that language.
59+
60+
```c#
61+
protected override void OnModelCreating(ModelBuilder modelBuilder)
62+
{
63+
modelBuilder.Entity<Blog>(b =>
64+
{
65+
b.HasDefaultFullTextLanguage("de-DE");
66+
b.Property(x => x.ContentsEnglish).EnableFullTextSearch("en-US");
67+
b.HasIndex(x => x.ContentsEnglish).IsFullTextIndex();
68+
b.Property(x => x.ContentsGerman).EnableFullTextSearch();
69+
b.HasIndex(x => x.ContentsGerman).IsFullTextIndex();
70+
b.Property(x => x.TagsGerman).EnableFullTextSearch();
71+
b.HasIndex(x => x.TagsGerman).IsFullTextIndex();
72+
});
73+
}
74+
```
75+
76+
## Querying
77+
78+
As part of the full-text search feature, Azure Cosmos DB introduced several built-in functions which allow for efficient querying of content inside the full-text search enabled properties. These functions are: [`FullTextContains`](/azure/cosmos-db/nosql/query/fulltextcontains), [`FullTextContainsAll`](/azure/cosmos-db/nosql/query/fulltextcontainsall), [`FullTextContainsAny`](/azure/cosmos-db/nosql/query/fulltextcontainsany), which look for specific keyword or keywords and [`FullTextScore`](/azure/cosmos-db/nosql/query/fulltextscore), which returns [BM25 score](https://en.wikipedia.org/wiki/Okapi_BM25) based on provided keywords.
79+
80+
> [!NOTE]
81+
> `FullTextScore` can only be used inside `OrderBy` to rank the documents based on the score.
82+
83+
EF Core exposes these functions as part of `EF.Functions` so they can be used in queries:
84+
85+
```c#
86+
var cosmosBlogs = await context.Blogs.Where(x => EF.Functions.FullTextContainsAll(x.Contents, "database", "cosmos")).ToListAsync();
87+
88+
var keywords = new string[] { "AI", "agent", "breakthrough" };
89+
var mostInteresting = await context.Blogs.OrderBy(x => EF.Functions.FullTextScore(x.Contents, keywords)).Take(5).ToListAsync();
90+
```
91+
92+
## Hybrid search
93+
94+
Full-text search can be used with vector search in the same query (i.e. hybrid search), by combining results of `FullTextScore` and `VectorDistance` functions. It can be done using the [`RRF`](/azure/cosmos-db/nosql/query/rrf) (Reciprocal Rank Fusion) function, which EF Core also provides inside `EF.Functions`:
95+
96+
```c#
97+
public class Blog
98+
{
99+
...
100+
101+
public float[] Vector { get; set; }
102+
public string Contents { get; set; }
103+
}
104+
105+
public class BloggingContext
106+
{
107+
...
108+
109+
protected override void OnModelCreating(ModelBuilder modelBuilder)
110+
{
111+
modelBuilder.Entity<Blog>(b =>
112+
{
113+
b.Property(x => x.Contents).EnableFullTextSearch();
114+
b.HasIndex(x => x.Contents).IsFullTextIndex();
115+
116+
b.Property(x => x.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
117+
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
118+
});
119+
}
120+
}
121+
122+
float[] myVector = /* generate vector data from text, image, etc. */
123+
var hybrid = await context.Blogs.OrderBy(x => EF.Functions.Rrf(
124+
EF.Functions.FullTextScore(x.Contents, "database"),
125+
EF.Functions.VectorDistance(x.Vector, myVector)))
126+
.Take(10)
127+
.ToListAsync();
128+
```
129+
130+
> [!TIP]
131+
> You can combine more than two scoring functions inside `Rrf` call, as well as using only `FullTextScore`, or only `VectorDistance`.

‎entity-framework/core/providers/cosmos/vector-search.md

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,9 @@ uid: core/providers/cosmos/vector-search
77
---
88
# Vector search
99

10-
> [!WARNING]
11-
> Azure Cosmos DB vector search is currently in preview. As a result, using EF's vector search APIs will generate an "experimental API" warning (`EF9103`) which must be suppressed. The APIs and capabilities may change in breaking ways in the future.
10+
Azure Cosmos DB now offers support for vector similarity search. Vector search is a fundamental part of some application types, including AI, semantic search and others. Azure Cosmos DB allows you to store vectors directly in your documents alongside the rest of your data, meaning you can perform all of your queries against a single database. This can considerably simplify your architecture and remove the need for an additional, dedicated vector database solution in your stack. To learn more about Azure Cosmos DB vector search, [see the documentation](/azure/cosmos-db/nosql/vector-search).
1211

13-
Azure Cosmos DB now offers preview support for vector similarity search. Vector search is a fundamental part of some application types, including AI, semantic search and others. Azure Cosmos DB allows you to store vectors directly in your documents alongside the rest of your data, meaning you can perform all of your queries against a single database. This can considerably simplify your architecture and remove the need for an additional, dedicated vector database solution in your stack. To learn more about Azure Cosmos DB vector search, [see the documentation](/azure/cosmos-db/nosql/vector-search).
14-
15-
To use vector search, you must first [enroll in the preview feature](/azure/cosmos-db/nosql/vector-search#enroll-in-the-vector-search-preview-feature). Then, [define vector policies on your container](/azure/cosmos-db/nosql/vector-search#container-vector-policies) to identify which JSON properties in your documents contain vectors and vector-related information for those properties (dimensions, data type, distance function).
16-
17-
Once your container is properly set up, add a vector property to your model in the path you defined in the container policy, and configure it with EF as a vector:
12+
Vector property can be configured inside `OnModelCreating`:
1813

1914
```c#
2015
public class Blog
@@ -30,9 +25,11 @@ public class BloggingContext
3025

3126
protected override void OnModelCreating(ModelBuilder modelBuilder)
3227
{
33-
modelBuilder.Entity<Blog>()
34-
.Property(b => b.Embeddings)
35-
.IsVector(DistanceFunction.Cosine, dimensions: 1536);
28+
modelBuilder.Entity<Blog>(b =>
29+
{
30+
b.Property(b => b.Vector).IsVectorProperty(DistanceFunction.Cosine, dimensions: 1536);
31+
b.HasIndex(x => x.Vector).IsVectorIndex(VectorIndexType.Flat);
32+
});
3633
}
3734
}
3835
```
@@ -56,3 +53,9 @@ var blogs = await context.Blogs
5653
```
5754

5855
This will returns the top five Blogs, based on the similarity of their `Vector` property and the externally-provided `anotherVector` data.
56+
57+
## Hybrid search
58+
59+
Vector similarity search can be used with full-text search in the same query (i.e. hybrid search), by combining results of `VectorDistance` and `FullTextScore` functions using the [`RRF`](/azure/cosmos-db/nosql/query/rrf) (Reciprocal Rank Fusion) function.
60+
61+
See [documentation](xref:core/providers/cosmos/full-text-search?#hybrid-search) to learn how to enable full-text search support in EF model and how to use hybrid search in queries.

‎entity-framework/core/what-is-new/ef-core-10.0/whatsnew.md

Lines changed: 76 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,72 @@ EF10 requires the .NET 10 SDK to build and requires the .NET 10 runtime to run.
2121

2222
## Azure Cosmos DB for NoSQL
2323

24+
<a name="full-text-search-support"></a>
25+
26+
### Full-text search support
27+
28+
Azure Cosmos DB now offers support for [full-text search](/azure/cosmos-db/gen-ai/full-text-search). It enables efficient and effective text searches, as well as evaluating the relevance of documents to a given search query. It can be used in combination with vector search to improve the accuracy of responses in some AI scenarios.
29+
EF Core 10 is adding support for this feature allowing for modeling the database with full-text search enabled properties and using full-text search functions inside queries targeting Azure Cosmos DB.
30+
31+
Here is a basic EF model configuration enabling full-text search on one of the properties:
32+
33+
```c#
34+
public class Blog
35+
{
36+
...
37+
38+
public string Contents { get; set; }
39+
}
40+
41+
public class BloggingContext
42+
{
43+
...
44+
45+
protected override void OnModelCreating(ModelBuilder modelBuilder)
46+
{
47+
modelBuilder.Entity<Blog>(b =>
48+
{
49+
b.Property(x => x.Contents).EnableFullTextSearch();
50+
b.HasIndex(x => x.Contents).IsFullTextIndex();
51+
});
52+
}
53+
}
54+
```
55+
56+
Once the model is configured, we can use full-text search operations in queries using methods provided in `EF.Functions`:
57+
58+
```c#
59+
var cosmosBlogs = await context.Blogs.Where(x => EF.Functions.FullTextContains(x.Contents, "cosmos")).ToListAsync();
60+
```
61+
62+
The following full-text operations are currently supported: [`FullTextContains`](/azure/cosmos-db/nosql/query/fulltextcontains), [`FullTextContainsAll`](/azure/cosmos-db/nosql/query/fulltextcontainsall), [`FullTextContainsAny`](/azure/cosmos-db/nosql/query/fulltextcontainsany), [`FullTextScore`](/azure/cosmos-db/nosql/query/fulltextscore).
63+
64+
For more information on Cosmos full-text search, see the [docs](xref:core/providers/cosmos/full-text-search).
65+
66+
### Hybrid search
67+
68+
EF Core now supports [`RRF`](/azure/cosmos-db/nosql/query/rrf) (Reciprocal Rank Fusion) function, which combines vector similarity search and full-text search (i.e. hybrid search). Here is an example query using hybrid search:
69+
70+
```c#
71+
float[] myVector = /* generate vector data from text, image, etc. */
72+
var hybrid = await context.Blogs.OrderBy(x => EF.Functions.Rrf(
73+
EF.Functions.FullTextScore(x.Contents, "database"),
74+
EF.Functions.VectorDistance(x.Vector, myVector)))
75+
.Take(10)
76+
.ToListAsync();
77+
```
78+
79+
For more information on Cosmos hybrid search, see the [docs](xref:core/providers/cosmos/full-text-search?#hybrid-search).
80+
81+
### Vector similarity search exits preview
82+
83+
In EF9 we added experimental support for [vector similarity search](xref:core/what-is-new/ef-core-9.0/whatsnew#vector-similarity-search-preview). In EF Core 10, vector similarity search support is no longer experimental. We have also made some improvements to the feature:
84+
85+
- EF Core can now generate containers with vector properties defined on owned reference entities. Containers with vector properties defined on owned collections still have to be created by other means. However, they can be used in queries.
86+
- Model building APIs have been renamed. A vector property can now be configured using the `IsVectorProperty` method, and vector index can be configured using the `IsVectorIndex` method.
87+
88+
For more information on Cosmos vector search, see the [docs](xref:core/providers/cosmos/vector-search).
89+
2490
<a name="improved-model-evolution"></a>
2591

2692
### Improved experience when evolving the model
@@ -64,12 +130,12 @@ See [#12793](https://github.com/dotnet/efcore/issues/12793) and [#35367](https:/
64130

65131
### Other query improvements
66132

67-
* Translation for DateOnly.ToDateTime(timeOnly) ([#35194](https://github.com/dotnet/efcore/pull/35194), contributed by [@mseada94](https://github.com/mseada94)).
68-
* Optimization for multiple consecutive `LIMIT`s ([#35384](https://github.com/dotnet/efcore/pull/35384), contributed by [@ranma42](https://github.com/ranma42)).
69-
* Optimization for use of `Count` operation on `ICollection<T>` ([#35381](https://github.com/dotnet/efcore/pull/35381), contributed by [@ChrisJollyAU](https://github.com/ChrisJollyAU)).
70-
* Optimization for `MIN`/`MAX` over `DISTINCT` ([#34699](https://github.com/dotnet/efcore/pull/34699), contributed by [@ranma42](https://github.com/ranma42)).
71-
* Translation for date/time functions using `DatePart.Microsecond` and `DatePart.Nanosecond` arguments ([#34861](https://github.com/dotnet/efcore/pull/34861)).
72-
* Simplifying parameter names (e.g. from `@__city_0` to `city`) ([#35200](https://github.com/dotnet/efcore/pull/35200)).
133+
- Translation for DateOnly.ToDateTime(timeOnly) ([#35194](https://github.com/dotnet/efcore/pull/35194), contributed by [@mseada94](https://github.com/mseada94)).
134+
- Optimization for multiple consecutive `LIMIT`s ([#35384](https://github.com/dotnet/efcore/pull/35384), contributed by [@ranma42](https://github.com/ranma42)).
135+
- Optimization for use of `Count` operation on `ICollection<T>` ([#35381](https://github.com/dotnet/efcore/pull/35381), contributed by [@ChrisJollyAU](https://github.com/ChrisJollyAU)).
136+
- Optimization for `MIN`/`MAX` over `DISTINCT` ([#34699](https://github.com/dotnet/efcore/pull/34699), contributed by [@ranma42](https://github.com/ranma42)).
137+
- Translation for date/time functions using `DatePart.Microsecond` and `DatePart.Nanosecond` arguments ([#34861](https://github.com/dotnet/efcore/pull/34861)).
138+
- Simplifying parameter names (e.g. from `@__city_0` to `city`) ([#35200](https://github.com/dotnet/efcore/pull/35200)).
73139

74140
## ExecuteUpdateAsync now accepts a regular, non-expression lambda
75141

@@ -120,7 +186,7 @@ Thanks to [@aradalvand](https://github.com/aradalvand) for proposing and pushing
120186

121187
## Other improvements
122188

123-
* Make SQL Server scaffolding compatible with Azure Data Explorer ([#34832](https://github.com/dotnet/efcore/pull/34832), contributed by [@barnuri](https://github.com/barnuri)).
124-
* Associate the DatabaseRoot with the scoped options instance and not the singleton options ([#34477](https://github.com/dotnet/efcore/pull/34477), contributed by [@koenigst](https://github.com/koenigst)).
125-
* Redact inlined constants from log when sensitive logging is off ([#35724](https://github.com/dotnet/efcore/pull/35724)).
126-
* Improve LoadExtension to work correctly with dotnet run and lib* named libs ([#35617](https://github.com/dotnet/efcore/pull/35617), contributed by [@krwq](https://github.com/krwq)).
189+
- Make SQL Server scaffolding compatible with Azure Data Explorer ([#34832](https://github.com/dotnet/efcore/pull/34832), contributed by [@barnuri](https://github.com/barnuri)).
190+
- Associate the DatabaseRoot with the scoped options instance and not the singleton options ([#34477](https://github.com/dotnet/efcore/pull/34477), contributed by [@koenigst](https://github.com/koenigst)).
191+
- Redact inlined constants from log when sensitive logging is off ([#35724](https://github.com/dotnet/efcore/pull/35724)).
192+
- Improve LoadExtension to work correctly with dotnet run and lib* named libs ([#35617](https://github.com/dotnet/efcore/pull/35617), contributed by [@krwq](https://github.com/krwq)).

0 commit comments

Comments
 (0)