Skip to content

Commit

Permalink
[DOCS] Adds links to token section in ESLER conceptual. (elastic#101033)
Browse files Browse the repository at this point in the history
  • Loading branch information
szabosteve committed Oct 18, 2023
1 parent 0ea5e02 commit 60ccb37
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,10 @@ GET my-index/_search
The result is the top 10 documents that are closest in meaning to your query
text from the `my-index` index sorted by their relevancy. The result also
contains the extracted tokens for each of the relevant search results with their
weights.
weights. Tokens are learned associations capturing relevance, they are not
synonyms. To learn more about what tokens are, refer to
{ml-docs}/ml-nlp-elser.html#elser-tokens[this page]. It is possible to exclude
tokens from source, refer to <<save-space,this section>> to learn more.

[source,consol-result]
----
Expand Down Expand Up @@ -325,12 +328,14 @@ by using the <<include-exclude,source exclude>> mapping to remove the ELSER
terms from the document source.

WARNING: Reindex uses the document source to populate the destination index.
Once the ELSER terms have been excluded from the source, they cannot be
recovered through reindexing. Excluding the tokens from the source is a
**Once the ELSER terms have been excluded from the source, they cannot be**
**recovered through reindexing.** Excluding the tokens from the source is a
space-saving optimsation that should only be applied if you are certain that
reindexing will not be required in the future! It's important to carefully
consider this trade-off and make sure that excluding the ELSER terms from the
source aligns with your specific requirements and use case.
source aligns with your specific requirements and use case. Review the
<<disable-source-field>> and <<include-exclude>> sections carefully to learn
more about the possible consequences of excluding the tokens from the `_source`.

The mapping that excludes `content_embedding` from the `_source` field can be
created by the following API call:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,9 @@ and the `output_field` that will contain the {infer} results.
To ingest data through the pipeline to generate tokens with ELSER, refer to the
<<reindexing-data-elser>> section of the tutorial. After you successfully
ingested documents by using the pipeline, your index will contain the tokens
generated by ELSER.
generated by ELSER. Tokens are learned associations capturing relevance, they
are not synonyms. To learn more about what tokens are, refer to
{ml-docs}/ml-nlp-elser.html#elser-tokens[this page].

// end::elser[]

Expand Down

0 comments on commit 60ccb37

Please sign in to comment.