{"componentChunkName":"component---node-modules-carrotsearch-gatsby-theme-apidocs-templates-documentation-page-js","path":"/basic-concepts/","result":{"data":{"html":{"frontmatter":{"id":"basic-concepts","title":"Basic concepts"},"html":"
\n

Basic concepts

\n\n

\n This section explains the basic concepts behind Lingo3G clustering, such as\n input content requirements, characteristics of clusters, quality tuning\n options and the available tools and APIs.\n

\n\n

\n This section is optional, feel free to skip to\n Installation and come back in case of any\n doubts.\n

\n\n
\n

\n Clusters\n

\n\n

\n Lingo3G organizes collections of documents into groups called\n clusters.\n

\n\n
\n \n
\n \n \"Document\n
\n
\n \n
\n \n
\n

\n Document clusters generated by Lingo3G\n for the clustering web search results.\n

\n
\n
\n\n

Clusters created by Lingo3G share the following characteristics:

\n\n \n
\n\n
\n

\n Documents for clustering\n

\n\n

\n The quality and performance of clustering depends on the characteristics\n of documents you provide to Lingo3G for clustering. Here are some\n recommendations about the perfect input for Lingo3G:\n

\n\n \n
\n\n
\n

\n Tools and APIs\n

\n\n

\n Lingo3G comes with a suite of tools and APIs that you can use to quickly\n try clustering on your own data, tune clustering parameters and integrate\n Lingo3G with your software.\n

\n\n
\n

\n Document Clustering Server (DCS)\n

\n\n

\n The Document Clustering Server (DCS) is an application you need to run\n to access the following elements of Lingo3G:\n

\n\n \n\n

\n You will find the DCS in the /dcs directory of Lingo3G\n distribution package.\n

\n
\n\n
\n

\n End-user apps\n

\n\n

\n Lingo3G comes with two browser-based apps for end users: Search Results\n Clustering and Clustering Workbench. We created the apps to help you\n quickly try Lingo3G clustering on your data. However, if you find the\n apps useful in your daily text mining work, feel free to use them on a\n regular basis.\n

\n\n

\n While we plan to add new features to the end-user apps, our priority is\n the development of the Lingo3G clustering algorithm core.\n

\n\n

\n To access the apps, you need to run the\n Document Clustering Server.\n

\n\n
\n

\n Search Results Clustering\n

\n\n

\n Lingo3G Search Results Clustering app helps novice users to cluster\n search results from public search engines. To keep the experience\n simple, the app only lets the user type the query and browse the\n clusters in textual and graphical form.\n

\n\n

\n The quickest way to use the Search Results Clustering app is to\n try the on-line version. For other options, see\n Trying Lingo3G. See the\n Search Results Clustering app\n section for a brief description of the user interface.\n

\n\n
\n \n
\n \n \"Lingo3G\n
\n
\n \n \"Lingo3G\n
\n \n
\n

\n Lingo3G search result clustering app\n showing results for the data mining web\n search query.\n

\n
\n
\n
\n\n
\n

\n Clustering Workbench\n

\n\n

\n If you have some basic IT experience, you can use Clustering Workbench\n to quickly try Lingo3G clustering on your data. Clustering Workbench\n may also be suitable as a text mining tool you use on a regular basis.\n

\n\n
\n \n
\n \n \"Lingo3G\n
\n
\n \n \"Lingo3G\n
\n \n
\n

\n Lingo3G Clustering Workbench\n app clustering results from a local file.\n

\n
\n
\n\n

With Lingo3G Workbench you can accomplish the following:

\n\n
    \n
  • \n

    Cluster results from the following sources:

    \n\n
      \n
    • web search engine
      • \n
    • PubMed search engine
      • \n
    • Excel, CSV, JSON and XML files
      • \n
    • Apache Solr
      • \n
    • Elasticsearch
      • \n
    \n
    • \n\n
  • \n

    \n Change the parameters of\n Lingo3G, including the structure of clusters, cluster label\n characteristics, label filtering dictionaries and more.\n

    \n
    • \n\n
  • \n

    \n Export the clusters and search results as Excel, OpenOffice, CSV\n or JSON.\n

    \n
    • \n\n
  • \n

    \n Visualize clusters using a pie chart and treemap, including export\n to JPEG or PNG.\n

    \n
    • \n\n
  • \n

    \n Export the modified Lingo3G parameters as JSON ready for pasting\n into Lingo3G REST API requests.\n

    \n
    • \n
\n\n

\n The quickest way to use Clustering Workbench is to\n try the on-line version. For other options, see\n Trying Lingo3G. See the\n Clustering Workbench app\n section for a brief description of the user interface.\n

\n
\n
\n\n
\n

\n APIs\n

\n\n

\n Lingo3G comes with two APIs: the REST API for integration with any\n programming language and the Java API for direct integration with Java\n software.\n

\n\n
\n

\n REST API\n

\n\n

\n Lingo3G REST API exposes clustering as a HTTP service: you make a POST\n request with documents to cluster in JSON format and get back a JSON\n response with a list of clusters.\n

\n\n

\n To access the REST API, you need to run the\n Document Clustering Server\n application.\n

\n
\n\n
\n

\n Java API\n

\n\n

\n If you develop a Java application, you can use Lingo3G as a library.\n See Java API basics for more\n details.\n

\n
\n\n
\n

\n REST or Java API?\n

\n\n

Use Lingo3G REST API in the following cases:

\n\n
    \n
  • \n

    You write your software in a language other than Java.

    \n\n

    \n In particular, you can call Lingo3G REST API directly from a\n browser-based app, just like\n Lingo3G end-user apps\n do.\n

    \n
    • \n\n
  • \n

    \n You write your software in Java, but you'd like to use the\n decoupled architecture, which has the following advantages:\n

    \n\n
      \n
    • \n

      \n You can increase the number of Lingo3G REST API servers\n dynamically without taking down your application.\n

      \n
      • \n\n
    • \n

      \n You can upgrade Lingo3G to a newer version without taking down\n your application.\n

      \n
      • \n\n
    • \n

      \n If the JVM running Lingo3G DCS crashes, it will not take down\n your application.\n

      \n
      • \n
    \n
    • \n
\n\n

Use Lingo3G Java API in the following cases:

\n\n
    \n
  • \n

    \n You would like to avoid the JSON serialization and deserialization\n overhead. The overhead should be negligible for small collections\n of documents, but may become significant when clustering larger\n numbers of documents.\n

    \n
    • \n
\n
\n
\n\n
\n

\n Integrations\n

\n\n

\n Apart from apps and APIs, Lingo3G comes with plugins that perform search\n results clustering inside two popular document indexing engines.\n

\n\n

\n An alternative to running Lingo3G inside your indexing engine is having\n your application fetch search results from the engine and push them to\n Lingo3G REST API for clustering. In addition to the universal\n advantages of a decoupled architecture, you will save some time configuring and upgrading Lingo3G indexing\n engine plugins.\n

\n\n
\n

\n Elasticsearch\n

\n\n

\n The\n elasticsearch-carrot2\n plugin accompanied by an extension plugin\n elasticsearch-lingo3g\n provide search results clustering for\n Elasticsearch.\n

\n
\n\n
\n

\n Apache Solr\n

\n\n

\n Apache Solr has\n built-in support\n for clustering search results. You can configure Solr to use Lingo3G\n for search results clustering.\n

\n
\n
\n
\n\n
\n

\n Carrot2\n

\n\n

\n Carrot2 is an\n open source project developed by Carrot Search. Lingo3G and Carrot2\n live in symbiosis: Lingo3G is a commercially licensed clustering algorithm\n built on top of the Carrot2-defined infrastructure. The source\n code of all Lingo3G tools and APIs is\n available in the\n Carrot2 GitHub repository\n under a permissive license. The only closed-source part is the Lingo3G\n clustering algorithm.\n

\n\n

\n If you'd like to work only with open source clustering algorithms, we\n recommend Carrot2's Lingo, the predecessor of Lingo3G. It is\n fast, fairly accurate and works very well for small document sets.\n

\n
\n
","tableOfContents":[{"heading":"Clusters","anchor":"clusters"},{"heading":"Documents for clustering","anchor":"documents-for-clustering"},{"heading":"Tools and APIs","anchor":"tools-and-apis","sections":[{"heading":"Document Clustering Server (DCS)","anchor":"apis-and-tools:document-clustering-server"},{"heading":"End-user apps","anchor":"tools-and-apis:end-user-apps","sections":[{"heading":"Search Results Clustering","anchor":"tools-and-apis:end-user-applications:search-results-clustering"},{"heading":"Clustering Workbench","anchor":"tools-and-apis:end-user-applications:clustering-workbench"}]},{"heading":"APIs","anchor":"#tools-and-apis:apis","sections":[{"heading":"REST API","anchor":"tools-and-apis:apis:rest-api"},{"heading":"Java API","anchor":"tools-and-apis:apis:java-api"},{"heading":"REST or Java API?","anchor":"tools-and-apis:apis:rest-or-java-api"}]},{"heading":"Integrations","anchor":"tools-and-apis:integrations","sections":[{"heading":"Elasticsearch","anchor":"tools-and-apis:integrations:elasticsearch"},{"heading":"Apache Solr","anchor":"tools-and-apis:integrations:apache-solr"}]}]},{"heading":"Carrot2","anchor":"carrot2"}]}},"pageContext":{"slug":"/basic-concepts/"}},"staticQueryHashes":["1164750013"]}