documentNeighbors
documentNeighbors:* components compute documents that are similar to one input document. You can use
the document neighborhood components to score documents based on complex criteria, such as results of per-document
queries combined with embedding vector similarity filtering.
You can use the following documentNeighbors:* in your analysis requests:
-
documentNeighbors:byQuery -
Executes per-document search queries to compute the document's neighbors.
-
documentNeighbors:filteredByEmbeddingVectorSimilarity -
Rejects neighbors whose similarity to the seed document is lower than the threshold.
documentNeighbors:reference-
References a
documentNeighbors:*component defined in the request or in the project's default components.
documentNeighbors:byQuery
Executes per-document search queries to compute the document's neighbors.
{
"type": "documentNeighbors:byQuery",
"limit": 1000,
"queryBuilder": {
"type": "queryBuilder:reference",
"auto": true
}
}
The queries to execute come from the queryBuilder
you provide. The query builder can build queries specific to each seed document, based on the values of the seed
document's fields.
limit
The maximum number of neighbors to produce for each seed document.
queryBuilder
The query builder to provide the seed document-specific query to execute.
documentNeighbors:filteredByEmbeddingVectorSimilarity
Rejects neighbors whose similarity to the seed document is lower than the threshold.
{
"type": "documentNeighbors:filteredByEmbeddingVectorSimilarity",
"documentNeighbors": {
"type": "documentNeighbors:reference",
"auto": true
},
"failIfEmbeddingsNotAvailable": true,
"ifVectorUndefined": "REJECT",
"minSimilarity": 0.7
}
You can use this component to filter the results produced by the provided documentNeighbors:*
based on the document's embedding similarity to the seed document.
documentNeighbors
The document neighbors to filter.
failIfEmbeddingsNotAvailable
Determines the behavior of this stage if the index does not contain document embeddings.
If the index does not contain document embeddings and failIfEmbeddingsNotAvailable is:
true- this stage fails and logs an error.
false-
performs filtering based on the value of the
ifVectorUndefinedproperty.
If your request combines keyword- and embedding-based processing, you can set
failIfEmbeddingsNotAvailable to false to have Lingo4G degrade gently to keyword-based
processing if the index does not contain document embeddings.
ifVectorUndefined
Determines what happens when an embedding vector is not available for the documents being filtered.
It may happen that document vectors are not available for specific documents being filtered or for the seed document.
The ifVectorUndefined determines the result of filtering in these cases:
ACCEPT-
Documents with undefined embedding vectors pass the filtering and are included in the resulting neighbor list.
REJECT-
Documents with undefined vectors don't pass the filtering and are excluded from the resulting neighbor list.
minSimilarity
The minimum embedding vector similarity to the seed document each filtered document must have to be included in the resulting list of neighbors.
Consumers of documentNeighbors:*
The following stages and components take documentNeighbors:* as
input: