Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Add docs for new semantic text query functionality #119520

Merged
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference/mapping/types/semantic-text.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Long passages are <<auto-text-chunking, automatically chunked>> to smaller secti
The `semantic_text` field type specifies an inference endpoint identifier that will be used to generate embeddings.
You can create the inference endpoint by using the <<put-inference-api>>.
This field type and the <<query-dsl-semantic-query,`semantic` query>> type make it simpler to perform semantic search on your data.
The `semantic_text` field type may also be queried with <<query-dsl-match-query, match>>, <<query-dsl-sparse-vector-query, sparse_vector>> or <<query-dsl-knn-query, knn>> queries.

If you don’t specify an inference endpoint, the `inference_id` field defaults to `.elser-2-elasticsearch`, a preconfigured endpoint for the elasticsearch service.

Expand Down
15 changes: 12 additions & 3 deletions docs/reference/query-dsl/knn-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ Finds the _k_ nearest vectors to a query vector, as measured by a similarity
metric. _knn_ query finds nearest vectors through approximate search on indexed
dense_vectors. The preferred way to do approximate kNN search is through the
<<knn-search,top level knn section>> of a search request. _knn_ query is reserved for
expert cases, where there is a need to combine this query with other queries.
expert cases, where there is a need to combine this query with other queries, or
perform a kNN search against a <<semantic-text, semantic_text>> field.

[[knn-query-ex-request]]
==== Example request
Expand Down Expand Up @@ -77,15 +78,17 @@ POST my-image-index/_search
+
--
(Required, string) The name of the vector field to search against. Must be a
<<index-vectors-knn-search, `dense_vector` field with indexing enabled>>.
<<index-vectors-knn-search, `dense_vector` field with indexing enabled>>, or a
<<semantic-text, `semantic_text` field>> with a compatible dense vector inference model.
--

`query_vector`::
+
--
(Optional, array of floats or string) Query vector. Must have the same number of dimensions
as the vector field you are searching against. Must be either an array of floats or a hex-encoded byte vector.
Either this or `query_vector_builder` must be provided.
Either this or `query_vector_builder` must be provided, unless all queried fields are of type <<semantic-text, semantic_text>>,
in which case the inference ID associated with the `semantic_text` field will be inferred.
kderusso marked this conversation as resolved.
Show resolved Hide resolved
--

`query_vector_builder`::
Expand Down Expand Up @@ -275,3 +278,9 @@ Thus, the final results from aggregations contain
`k * number_of_shards` documents. This is different from
the <<knn-search,top level knn section>> where aggregations are
calculated on the global top `k` nearest documents.

[[knn-query-multiple-indices]]
==== Knn query over multiple indices
`knn` query calculates the top `k` documents from each index searched.
Therefore, if `k` is smaller than the query `size`, `k` matching documents
from each index will be returned.
kderusso marked this conversation as resolved.
Show resolved Hide resolved
3 changes: 2 additions & 1 deletion docs/reference/query-dsl/match-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ provided text is analyzed before matching.
The `match` query is the standard query for performing a full-text search,
including options for fuzzy matching.

`Match` will also work against <<semantic-text, semantic_text>> fields.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth adding another note on what match query options can be used with semantic_text fields?
e.g. boost can be used, but specifying fuzziness has no impact when querying semantic_text fields?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good callout, I will update that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, LMKWYT!



[[match-query-ex-request]]
==== Example request
Expand Down Expand Up @@ -296,4 +298,3 @@ The example above creates a boolean query:

that matches documents with the term `ny` or the conjunction `new AND york`.
By default the parameter `auto_generate_synonyms_phrase_query` is set to `true`.

6 changes: 3 additions & 3 deletions docs/reference/query-dsl/sparse-vector-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ This can be achieved with one of two strategies:
- Using an {nlp} model to convert query text into a list of token-weight pairs
- Sending in precalculated token-weight pairs as query vectors

These token-weight pairs are then used in a query against a <<sparse-vector,sparse vector>>.
These token-weight pairs are then used in a query against a <<sparse-vector,sparse vector>>
or a <<semantic-text, semantic_text>> field with a compatible sparse inference model.
At query time, query vectors are calculated using the same inference model that was used to create the tokens.
When querying, these query vectors are ORed together with their respective weights, which means scoring is effectively a <<vector-functions-dot-product,dot product>> calculation between stored dimensions and query dimensions.

Expand Down Expand Up @@ -65,6 +66,7 @@ GET _search
It must be the same inference ID that was used to create the tokens from the input text.
Only one of `inference_id` and `query_vector` is allowed.
If `inference_id` is specified, `query` must also be specified.
If all queried fields are of type <<semantic-text, semantic_text>>, the inference ID associated with the `semantic_text` field will be inferred.

`query`::
(Optional, string) The query text you want to use for search.
Expand Down Expand Up @@ -291,5 +293,3 @@ GET my-index/_search
//TEST[skip: Requires inference]

NOTE: When performing <<modules-cross-cluster-search, cross-cluster search>>, inference is performed on the local cluster.