Analysis Response JSON
You probably saw a glimpse of an what an analysis "result" looks like in the Analysis Stages chapter. Here, we will describe all elements of the analysis response JSON in more detail.
Here is an outline of the top-level properties that can appear in an analysis response JSON.
{
"result": { ... },
"status": { ... },
"log": { ... },
"request": { ... }
}
The above result JSON structure is identical for both the
HTTP REST API response and the
run-request
command line application output.
We will describe each of these blocks in sections below.
result
Response
The result
element is an object with properties corresponding to each stage identifier defined in the
stages
element of the request. The value of each property contains the result of the corresponding stage.
Consider the following example request that has two
documents:byQuery
stages, each one configured slightly differently:
The result
block for the above request can look like this (note source stage identifiers):
Each stage type will have a different JSON result structure, depending on what the stage actually does. The
description of the expected result is part of each stage's documentation. For example, here is the corresponding
documentation for the
documents:byQuery
stage used above.
Finally, note that the result
block may be missing for
asynchronous analyses. Use the
status block to determine the lifecycle state of your analysis. We describe the
status field in the following section.
status
Response
The status
element carries information about the outcome and progress of the request's execution. It
has the following outline structure:
"status": {
"status" : "...",
"elapsedMs" : 0,
"tasks": []
},
The properties of the status block are:
-
status
-
The lifecycle state the analysis is in. The status can take one of the following values:
PROCESSING
-
when the analysis is still executing. The result is not yet available.
AVAILABLE
-
when the result for an analysis has been computed and is available.
NOT_FOUND
-
when the analysis (or its result) is not available. There are two possible causes of this status:
-
The analysis with the hash ID you provided never existed. This may be due to a misspelling or other malformation of the hash ID you provided to the API.
-
Lingo4G already evicted the analysis or its result from the results cache. In this case, you need to re-submit the analysis for computation. The new analysis will receive a new hash ID, which you'll need to use to retrieve the result.
-
FAILED
-
when the analysis failed for some reason. The
log
block contains more information about the error. VALID
-
when the analysis JSON validates properly (used by the validation endpoint).
-
elapsedMs
-
The number of milliseconds since the analysis has been created (queued or executing on the server).
-
tasks
-
An array of top-level tasks describing units of work that Lingo4G is about to perform. You can use the tasks property to display partial progress information while Lingo4G is computing the results. Here is an outline of a typical single task:
{ "name" : "Task name", "status" : "...", "startedAt": ..., "progress" : 1.0, "tasks" : [], "attributes" : [] }
A task consists of a mandatory
name
andstatus
(NEW
,STARTED
,SKIPPED
orDONE
). A task may contain sub-tasks (tasks
array) and arbitrary key-valueattributes
providing additional insight into the task's details.An optional
progress
property contains the task's completion ratio as a floating point value (0-1). ThestartedAt
property contains the wall time of when the task was started.
Shown below is an actual status block for the example request we used above. Note the nested task structure.
request
Response
When the analysis request has an
output.request
option set to
true
, the request
block will contain an echo of the input request,
with all references resolved. This may be helpful for
archival or diagnostic purposes (to keep the fully resolved request together with the result).
For example, consider this request, which uses variables, components, components defined at project descriptor level, and references:
The resolved echoed request looks like this:
log
Response
The log
array contains validation or runtime errors for the analysis and is typically empty. If an
error occurred, the status
property will indicate a failed analysis (FAILED
) and the log
will contain details of what exactly went wrong. It is also possible for the analysis to be successful with a
non-empty log (when it contains warnings only).
The log array contains objects with the following outline structure:
{
"level": "...",
"code": "...",
"message": "...",
"details": {
...
}
}
The properties of each error are described below.
-
level
-
Severity level:
ERROR
,WARNING
,INFO
orDEBUG
. -
code
-
Log entry code for diagnostics.
-
message
-
A human-friendly log entry message.
-
details
-
Log entry details. The content of this object depends on the specific log entry code.
For example, an invalid input (JSON parsing exception) could return the following log and status:
{
"status" : {
"status" : "FAILED",
"error" : "Invalid analysis request configuration"
},
"log" : [
{
"level" : "ERROR",
"code" : "E001",
"message" : "JSON parse error.",
"details" : {
"description" : "Unexpected character (',' (code 44)): was expecting a colon to separate field name and value",
"json" : "{\n Ooops, this is not valid json.\n}\n",
"line" : 2,
"column" : 9,
"offset" : 10
}
}
]
}
The log can contain more than one entry. Here is an example response for a request with two violated parameter constraints:
{
"stages": {
"stage1": {
"type" : "documents:byQuery",
"limit": -1,
"query": {
"type": "query:string",
"query": "photon"
}
},
"stage2": {
"type" : "documents:byQuery",
"limit": -2,
"query": {
"type": "query:string",
"query": "ray"
}
}
}
}
Selected log entry codes
This section contains more information on the most frequent log entry codes and their associated details objects.
-
W001
-
A warning that the request exceeded the license limit on document count and the limit has been applied.
-
W111
-
A warning that certain cluster members have been filtered out due to missing document index mappings.
-
E001
-
The input analysis request is not a valid JSON. The
details
object contains more details from the internal JSON parser, here is an example:{ "level": "ERROR", "code": "E001", "message": "JSON parse error.", "details": { "description": "Unexpected character (',' (code 44)): was expecting a colon to separate field name and value", "json": "{\n Ooops, this is not valid json.\n}\n", "line": 2, "column": 9, "offset": 10 } }
-
E002
-
The input analysis request does not validate against the analysis schema. The
details
object contains more details from the schema validator:{ "level": "ERROR", "code": "E002", "message": "JSON schema error.", "details": { "description": "Unrecognized field \"max\" (class com.carrotsearch.lingo4g.impl.ri), not marked as ignorable", "json": "{\n \"stages\": {\n \"stage1\": {\n \"type\" : \"documents:byQuery\",\n \"max\": 10,\n \"query\": {\n \"type\": \"query:string\",\n \"query\": \"photon\"\n }\n }\n }\n}\n", "line": -1, "column": -1, "offset": -1 } }
-
E003
-
The input analysis request has non-existing component or stage reference.
Invalid reference name is part of the
message
field.{ "level": "ERROR", "code": "E003", "message": "Reference target 'missing-reference-key' not found." }
Double-check the stage names for typos. The analysis JSON editor in the JSON Sandbox app should highlight such errors for you.
-
E004
-
The input analysis request has a component and a stage with the same identifier. A reference to this identifier cannot be resolved due to this ambiguity. Here is an example invalid request showing this problem (
docs
is declared as a stage and as a component):{ "components": { "docs": { "type": "query:string", "query": "test" } }, "stages": { "docs": { "type": "documents:byQuery", "query": { "type": "query:reference", "use": "docs" } } } }
This would result in the following error log message:
{ "level": "ERROR", "code": "E004", "message": "Both components and stages contain a config with id 'docs'. Change one of the ids to resolve the ambiguity." }
To remove the error, rename either the stage or the component.
-
E005
-
The input analysis has an auto-reference cannot be resolved. Here is a request with an auto-reference in the documents field of the
documentContent
stage:{ "stages": { "content": { "type": "documentContent" } } }
Note the field is not explicitly declared or mentioned. This request would result in the following error log message:
{ "level": "ERROR", "code": "E005", "message": "Unsatisfied auto reference for type 'documents:*'." }
To correct the error, define the missing stage.
-
E006
-
The input analysis has an auto-reference that can be resolved to more than one target, making it ambiguous. Here is a request with an auto-reference in the documents field of the
documentContent
stage:{ "stages": { "docs1": { "type": "documents:byId", "documents": [ { "id": 0 } ] }, "docs2": { "type": "documents:byId", "documents": [ { "id": 0 } ] }, "content": { "type": "documentContent" } } }
Note the field is not explicitly declared or mentioned and two other stages match the required reference type. This request would result in the following error log message:
{ "level": "ERROR", "code": "E006", "message": "More than one component found for auto reference of type 'documents:*'. Use an explicit reference to resolve the ambiguity.", "details": { "matchingStages": [ "docs1", "docs2" ], "matchingComponents": [] } }
To correct the error, make an explicit reference to the desired stage result.
-
E007
-
The request contained a component or stage reference at the top-level. Here is an example of such an invalid request:
{ "stages": { "docs": { "type": "documents:reference" } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E007", "message": "Top-level stages and components must not be of reference type: $.stages.docs" }
Lingo4G does not support top-level references.
-
E008
-
The request referenced an unknown stage in the output element. Here is an example of such an invalid request:
{ "stages": { "documents": { "type": "documents:byQuery", "limit": 3, "query": { "type": "query:string", "query": "photon" } } }, "output": { "stages": [ "documentosTypo" ] } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E008", "message": "Output stage not found: 'documentosTypo'." }
Double-check the stage names for typos. The analysis JSON editor in the JSON Sandbox app should highlight such errors for you.
-
E009
-
The target of a reference has an incompatible type and the reference is not resolvable. Here is an example of such an invalid request:
{ "stages": { "documentStats": { "type": "stats:documents", "documents": { "type": "documents:reference", "use": "labels" } }, "labels": { "type": "labels:direct", "labels": [] } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E009", "message": "Incompatible reference target type: documents:* is required but reference target is labels:direct." }
-
E010
-
A reference cycle has been detected in the input graph of stages and components. Here is an example of such an invalid request:
{ "components": { "labelFilter": { "type": "labelFilter:composite", "labelFilters": { "auto": { "type": "labelFilter:reference", "use": "labelFilter" } } } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E010", "message": "Analysis request contains a reference cycle.", "details": { "nodeChain": [ "$.components.labelFilter", "$.components.labelFilter.labelFilters" ], "refChain": [ "$.components.labelFilter.labelFilters", "$.components.labelFilter.labelFilters.auto" ] } }
The returned error details object contains JSON paths pointing at the reference cycle. The
nodeChain
array contains stage and component node paths, therefChain
includes stage and component properties (edges in the graph). -
E011
-
A component or a stage has a required property but the provided value is empty (
null
).{ "stages": { "docs": { "type": "documents:byQuery", "query": { "type": "query:forLabels", "labels": null, "fields": null } } } }
The returned error logs for this request would look like this:
{ "level": "ERROR", "code": "E011", "message": "Property $.stages.docs.query.labels must not be null.", "details": { "pointer": "$.stages.docs.query.labels", "constraint": "NotNull" } }
-
E012
-
An error returned when a negative value was provided to a component or stage property requiring a non-negative value. Here is an example of such an invalid request:
{ "stages": { "stage1": { "type" : "documents:byQuery", "limit": -1, "query": { "type": "query:string", "query": "photon" } }, "stage2": { "type" : "documents:byQuery", "limit": -2, "query": { "type": "query:string", "query": "ray" } } } }
The returned error log message would look like this:
[ { "level": "ERROR", "code": "E012", "message": "Property $.stages.stage1.limit must be greater than or equal to 0.", "details": { "pointer": "$.stages.stage1.limit", "constraint": "PositiveOrZero" } }, { "level": "ERROR", "code": "E012", "message": "Property $.stages.stage2.limit must be greater than or equal to 0.", "details": { "pointer": "$.stages.stage2.limit", "constraint": "PositiveOrZero" } } ]
The
pointer
is a JSON path that points at the problematic parameter in the source JSON. -
E013
-
An error returned when a negative or zero value was provided to a component or stage property requiring a positive value. Here is an example of such an invalid request:
{ "stages": { "documents": { "type": "documents:byQuery", "query": { "type": "query:forLabels", "labels": { "type": "labels:direct", "labels": [ { "label": "w01" } ] }, "minOrMatches": 0 } } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E013", "message": "Property $.stages.documents.query.minOrMatches must be greater than 0.", "details": { "pointer": "$.stages.documents.query.minOrMatches", "constraint": "Positive" } }
The
pointer
is a JSON path that points at the problematic parameter in the source JSON. -
E014
-
An error returned when a value outside the [0, 1] range was provided to a component or stage property requiring a normalized value.
The returned error log message would look like this:
{ "level": "ERROR", "code": "E014", "message": "Property $.stages.clusters.softening must be in the 0..1 range, inclusive.", "details": { "pointer": "$.stages.clusters.softening", "constraint": "Normalized" } }
The
pointer
is a JSON path that points at the problematic parameter in the source JSON. -
E015
-
An error returned when a value outside the required range was provided to a component or stage property.
The returned error log message would look like this:
{ "level": "ERROR", "code": "E015", "message": "Property '$.stages.pairs.hashGrouping.pairing.maxHashBitsDifferent' must be in the 0.00..5.00 range.", "details": { "pointer": "$.stages.pairs.hashGrouping.pairing.maxHashBitsDifferent", "constraint": "InRange" } }
The
pointer
is a JSON path that points at the problematic parameter in the source JSON. -
E030
-
The top-level element of the received analysis request was not a JSON object.
The returned error log message would look like this:
{ "level": "ERROR", "code": "E030", "message": "Request config must be a JSON object." }
E031
,E032
-
Errors returned when the request uses a variable and the variables block is missing, empty or does not declare the declared variable. Here is an example of such an invalid request:
{ "stages": { "docs": { "type": "documents:byQuery", "query": { "@var": "query" } } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E031", "message": "Undefined variable 'query'. The variables section does not contain variables or is not present.", "details": { "variable": "query", "path": "$stages.docs.query" } }
Double-check the variable names for typos. The analysis JSON editor in the JSON Sandbox app should highlight such errors for you.
E033
,E034
-
The declaration of a variable is incorrect. Here is an example of such an invalid request:
{ "variables": { "query": { "noValueProperty": 1 } }, "stages": { "docs": { "type": "documents:byQuery", "query": { "@var": "query" } } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E033", "message": "Variable 'query' must be an object and must have the \"value\" or \"variables\" property." }
-
E035
-
Variables are not unique (multiple variables use the same identifier). Here is an example of such an invalid request:
{ "variables": { "query": "z", "group": { "variables": { "query": "x" } } }, "stages": { "docs": { "type": "documents:byQuery", "query": { "@var": "query" } } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E035", "message": "Variables must be unique. Offending variable: 'query'." }
-
E040
-
An error thrown when the request used a stage or component that called for the default query parser but the project descriptor's query parser block is empty.
-
E041
-
A stage or component referenced a query parser that is not declared in the project descriptor's query parser block. Here is an example of such an invalid request:
{ "stages": { "docs": { "type": "documents:byQuery", "query":{ "type": "query:string", "queryParser": { "type": "queryParser:project", "queryParserKey": "fancyQueryParser" }, "query": "photon" } } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E041", "message": "Unknown query parser: 'fancyQueryParser'. Available query parsers: 'enhanced'." }
-
E042
-
An error thrown when a Lucene query has a syntax error and cannot be parsed properly.
This error can be thrown if the query in
documents:byQuery
has an invalid syntax, for example.{ "level": "ERROR", "code": "E042", "message": "Failed to parse query.", "details": { "query": "invalid query [something TO", "queryParser": "", "error": "Syntax Error, cannot parse invalid query [something TO: Encountered \"<EOF>\" at line 1, column 27.\r\nWas expecting one of:\r\n \"TO\" ...\r\n <RANGE_QUOTED> ...\r\n <RANGE_GOOP> ...\r\n " } }
-
E043
-
A stage or a component requires a field that supports tokenization (dividing text into words) but the referenced fields do not support it.
The returned error log message would look like this:
{ "level": "ERROR", "code": "E043", "message": "Only fields that support a token stream are supported, these do not: category." }
-
E044
-
A logical analysis request configuration error. This can occur when an arrangement from components or stages has a valid syntax but cannot be executed because it is nonsensical.
-
E060
-
This error signals an invalid or missing matrix somewhere during the request execution.
-
E070
,E071
,E072
-
Input argument errors returned by the contrast scores document stage.
-
E080
-
One of the components or stages required label embeddings but they are not available in the index.
You have to learn label embeddings, then reload the server to use the new index, once the embeddings are available.
-
E090
-
One of the components or stages required document embeddings but they are not available in the index.
You have to learn document embeddings, then reload the server to use the new index, once the embeddings are available.
-
E100
-
An unknown field reference was found. Here is an example of such an invalid request:
{ "stages": { "values": { "type": "values:fromDocumentField", "documents": { "type": "documents:byQuery", "limit": "unlimited", "query": { "type": "query:all" } }, "fieldName": "foobar" } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E100", "message": "Unknown document field: foobar." }
Double-check the field names for typos. The analysis JSON editor in the JSON Sandbox app should highlight such errors for you.
E101
,E102
-
Errors thrown by the
values:fromDocumentField
stage when the number of field values does not match the constraints specified inmultipleValues
property. Here is an example of such an invalid request:{ "stages": { "values": { "type": "values:fromDocumentField", "documents": { "type": "documents:byQuery", "limit": "unlimited", "query": { "type": "query:all" } }, "fieldName": "i", "multipleValues": "REQUIRE_EXACTLY_ONE", "threads": 1 } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E100", "message": "Unknown document field: i." }
-
E111
-
Duplicate document identifiers detected in the input that required unique document identifiers. Here is an example of such an invalid request:
{ "stages": { "documents": { "type": "documents:byId", "documents": [ { "id": 1 }, { "id": 2 }, { "id": 3 }, { "id": 4 }, { "id": 4 } ] } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E111", "message": "Duplicate document id: 4." }
-
E112
-
An invalid document identifier detected in the input. The document identifier may point at a deleted or non-existing document. Here is an example of such an invalid request:
{ "stages": { "documents": { "type": "documents:byId", "documents": [ { "id": 1 }, { "id": 2 }, { "id": 10000000 } ] } } }
The returned error log message would look like this:
{ "level": "ERROR", "code": "E112", "message": "Document id out of range: 10000000." }
-
E113
-
An non-existent project dictionary identifier was used by a component or stage.
The returned error log message would look like this:
{ "level": "ERROR", "code": "E113", "message": "Unknown project dictionary reference: dictionary_foobar" }
-
E999
,E110
-
An unhandled error occurred during execution.
This type of error signals an unexpected exception in Lingo4G that is most likely a software defect. Please report to info@carrotsearch.com, ideally together with the request that caused the problem.