JSON Sandbox app

JSON Sandbox app helps you to edit, execute, tune and debug Lingo4G analysis request JSONs.

To run Lingo4G JSON Sandbox app, perform the following steps:

  1. Start Lingo4G REST API server.

  2. Open http://localhost:8080/explorer/v2#code in a modern browser.

User interface overview

JSON Sandbox user interface has two main parts. On the left, there is the Lingo4G analysis request JSON editor along with a list of example requests you can execute. The right-hand-side part displays the results of the request. The JSON, status, diagram and other tabs present various facets of the same result.

Lingo4G JSON sandbox app user interface overview (light theme).
Lingo4G JSON sandbox app user interface overview (dark theme).

Lingo4G JSON Sandbox app user interface overview.

Request editor

The left-hand-side part of JSON Sandbox app contains the analysis request JSON editor.

Example requests

The requests tab of the side panel shows a list of example requests you can execute. The list has two categories:

Project

Project-specific requests, if any are defined in your project.

General

Built-in example requests. Some of the requests are tuned for the arXiv example data set, so you may need to tune the search query or field names to match your project.

Use the input box at the top of the sidebar to search through request titles and descriptions.

Request outline

The outline tab displays the overview of the current request, including name, comments and the list of variables.

Request JSON editor

The large text area on the left-hand-side of the screen is the analysis JSON request editor. The editing tips section below offers some hints on how to edit your analysis requests efficiently.

Note for Visual Studio Code users

Lingo4G request JSON editor uses the code editor component from Visual Studio Code (VSC). If you are familiar with VSC, some of the keyboard shortcuts will also work in the JSON Sandbox request editor.

Request variables editor

If your request contains variables, you can change their values using familiar GUI elements, such as text inputs for string-typed variables or sliders for numeric variables. To do so, activate the variables view.

Lingo4G JSON sandbox app: variable editor (light theme).
Lingo4G JSON sandbox app: variable editor (dark theme).
JSON Sandbox app with request variables editor active.

JSON sandbox synchronizes the JSON and variable editor views. Any variable value changes you make in one view will be reflected in the other view.

Results area

The right-hand-side part of the JSON Sandbox app presents the results of the current request. Depending on the request and processing status, you may see results tabs described below.

JSON

Shows the analysis response JSON received from the server. For successful requests, you can inspect the results produced by your analysis. For failed requests, you can inspect the details of the error.

Lingo4G JSON sandbox app: JSON results tab (light theme).
Lingo4G JSON sandbox app: JSON results tab (dark theme).
Heads up, long array truncation.

To keep the application fast, the JSON view by default presents only the first 25 elements of each array. The status line at the top shows how many results arrays have been truncated. Use JSON view settings to show more results array elements.

Status

Shows the status of the request.

For successful requests, you will see a breakdown of the tasks required to compute the result. Use view settings to enable additional information about each task.

Lingo4G JSON sandbox app: successful analysis status (light theme).
Lingo4G JSON sandbox app: successful analysis status (dark theme).

For failed requests, the status tab shows detailed error messages. The following example shows search query syntax error, where the field name should be title instead of tilte.

Lingo4G JSON sandbox app: failed analysis status (light theme).
Lingo4G JSON sandbox app: failed analysis status (dark theme).

Diagram

Shows a graph of dependencies between stages and components of the request. Use the graph to get a high-level overview of the structure of your request for learning or debugging purposes.

Lingo4G JSON sandbox app: request diagram (light theme).
Lingo4G JSON sandbox app: request diagram (dark theme).

Summary

Shows summary statistics, such as numbers of documents or labels, for all results produced by the request. If entities are related, such as documents and clusters of those documents, the relevant statistics appear in the same line. In the example below shows the number of selected documents and the number and coverage of clusters generated for those documents.

Lingo4G JSON sandbox app: summary statistics (light theme).
Lingo4G JSON sandbox app: summary statistics (dark theme).

Stage-specific-tabs

Depending on the contents of your request, you may see a number of tabs showing results of specific request stages. The following example shows the contents for a list of documents.

Lingo4G JSON sandbox app: stage-specific views (light theme).
Lingo4G JSON sandbox app: stage-specific views (dark theme).

Currently, JSON Sandbox has type-specific views for:

If the request contains output from multiple stages of the same type, like in the example above, a select box with all available results will appear.

JSON Sandbox tries to integrate results from related stages, if possible. For example, if the request retrieves per-document labels, the labels also appear as an extra column in the document list view. If the same request computes both the 2d embedding and clusters for the same set of documents, the 2d map view colors the 2d document points based on the cluster to which they belong.

Stage-specific view missing?

To display a stage-specific view, such as document​Content results, JSON Sandbox requires that the output also contains the base entity, such as the document identifiers produced by documents:​* stages.

For example, the document contents view will not show for the following request:

{
  "stages": {
    "documents": {
      "type": "documents:byQuery",
      "query": {
        "type": "query:string",
        "query": ""
      },
      "limit": 10
    },
    "documentContent": {
      "type": "documentContent"
    }
  },
  "output": {
    "stages": [
      "documentContent"
    ]
  }
}

The above request does not output the result of the documents stage, and JSON Sandbox is unable to make a connection between document identifiers and document content lists.

To display the document contents view, include the output of the documents stage:

{
  "stages": {
    "documents": {
      "type": "documents:byQuery",
      "query": {
        "type": "query:string",
        "query": ""
      },
      "limit": 10
    },
    "documentContent": {
      "type": "documentContent"
    }
  },
  "output": {
    "stages": [
      "documents",

      "documentContent"
    ]
  }
}

Excel export

You can export lists of documents and labels as Excel workbooks.

Document list

To export a document list into Excel, perform the following steps:

  1. In the JSON Sandbox app, run a request that returns documents with contents, such as the Documents by query with contents, More-Like-This document, keyword-based or Contrast scores (MLT) example request or a similar request of your own.

  2. Activate the documents list results tab. Choose the document list in the select box, if needed.

  3. Click the Excel export icon to export the documents as an Excel workbook.

    Excel export of a document list (light theme).
    Excel export of a document list (dark theme).

Label list

To export a label list into Excel, perform the following steps:

  1. In the JSON Sandbox app, run a request that returns a list of labels, such the Labels by label embedding vector or Label clusters by embedding similarity example request or a similar request of your own.

  2. Activate the labels list results tab. Choose the label list in the select box, if needed.

  3. Click the Excel export icon to export the labels as an Excel workbook.

    Excel export of a label list (light theme).
    Excel export of a label list (dark theme).

Analysis request editing tips

Try the following tips for faster editing of your Lingo4G analysis request JSONs.

Use autocomplete

Press Ctrl + Space to see a list of context-specific suggestions.

When you invoke autocomplete at the top-level of the request JSON, you will get suggestions of the relevant top-level request properties, such as stages or variables:

JSON request editor top-level autocomplete (light theme).
JSON request editor top-level autocomplete (light theme).

Use autocomplete for stages and components

When a property requires a specific stage or component type, autocomplete will suggest only the options valid at the specific context:

JSON request editor stage-level autocomplete (light theme).
JSON request editor stage-level autocomplete (light theme).

To narrow down the selection, start typing the name of the specific stage or component:

JSON request editor filtered stage-level autocomplete (light theme).
JSON request editor filtered stage-level autocomplete (light theme).

Use autocomplete for references

To quickly insert a reference stage or component, do this:

  1. Press Ctrl + Space to show suggestions.

  2. Type ref to narrow down the list to reference types.

    JSON request editor reference suggestions (light theme).
    JSON request editor reference suggestions (light theme).

    Notice that the suggestion list contain only references matching the type required by the property you are editing.

    Press Enter to insert the selected reference stage or component.

  3. Again press Ctrl + Space to bring up stage name suggestions.

    JSON request editor stage name suggestions (light theme).
    JSON request editor stage name suggestions (light theme).

    Notice again that autocomplete only offer names of stages that are compatible with the enclosing reference type.

Use autocomplete for variable names

When editing a variable reference, press Ctrl + Space to get variable name suggestions. Notice that the suggestions only include variables of types compatible with the property you are editing.

JSON request editor variable name suggestions (light theme).
JSON request editor variable name suggestions (light theme).

Pay attention to validation error highlights

The editor puts yellow squiggly lines under invalid elements of your request JSON. Hover over the highlighted text or press Alt + F8 for error details.

JSON request editor validation error highlights (light theme).
JSON request editor validation error highlights (dark theme).

For the example above, the editor highlighted the following problems:

  • Incorrect value of the limit variable. The request uses the limit variable in the limit property of the documents:​by​Query stage. The property requires an integer or the "unlimited" string.

  • Incorrect definition of the documents stage. The definition misses the required query

    property.
  • Unknown query​String property. The documents:​by​Query stage does not allow a query​String property.

  • Unknown stage name. The labels stage references a non-existing stage: documnets should probably be documents.