{"componentChunkName":"component---node-modules-carrotsearch-gatsby-theme-apidocs-templates-documentation-page-js","path":"/java-api-basics/","result":{"data":{"html":{"frontmatter":{"id":"java-api-basics","title":"Java API Basics"},"html":"
\n

Java API Basics

\n\n

\n You can use Lingo3G Java API to call clustering from your Java software.\n This article explains the basics of the API.\n

\n\n
\n

\n Dependency setup\n

\n\n

\n To set up Lingo3G as a dependency in your project, perform the following\n steps:\n

\n\n
    \n
  1. \n

    \n Install Lingo3G POM file in your local Maven repository by running:\n

    \n\n
    \n 
    mvn install:install-file -Dfile=lib/lingo3g-*.jar -DpomFile=lib/lingo3g-*.pom
    \n
    \n\n

    \n Alternatively, install the dependency on your company's\n shared-artifact servers, such as\n Sonatype Nexus.\n

    \n
    2. \n\n
  2. \n

    Add a Lingo3G dependency to your project.

    \n\n
      \n
    • \n

      For Maven projects, add the following dependency:

      \n\n
      \n
      \n 
      <dependency>\n  <groupid>com.carrotsearch.lingo3g</groupid>\n  <artifactid>lingo3g</artifactid>\n  <version>2.0.0</version>\n</dependency>
      \n
      \n\n
      \n

      Lingo3G Maven dependency.

      \n
      \n
      \n
      • \n\n
    • \n

      For Gradle projects, add the following dependency:

      \n\n
      \n
      \n 
      dependencies {\n  implementation(\"com.carrotsearch.lingo3g:lingo3g:2.0.0\")\n}
      \n
      \n\n
      \n

      Lingo3G Gradle dependency

      \n
      \n
      \n\n

      \n (Here we useimplementation as a particular\n configuration to attach the dependency to.)\n

      \n\n

      \n Alternatively, a gradle dependency can just include the set of JAR\n files from under the distribution's lib/ folder:\n

      \n\n
      \n 
      dependencies {\n  implementation fileTree(\"lingo3g-distribution/lib\").include(\"**/*.jar\")\n}
      \n
      \n
      • \n
    \n
    3. \n
\n
\n\n
\n

\n API overview\n

\n\n

\n The Java code invoking Lingo3G clustering needs to perform the following\n steps:\n

\n\n
    \n
  1. \n

    \n Load and cache the reusable component – the\n LanguageComponents instance.\n

    \n\n
    \n Load LanguageComponents only once.\n\n

    \n LanguageComponents contains various linguistic\n resources, such as dictionaries and stemmers, and may take\n significant time to load. Load LanguageComponents once\n and reuse for all clustering calls. The class is thread-safe, so\n share one instance between all processing threads.\n

    \n
    \n
    2. \n\n
  2. \n

    For each clustering call:

    \n\n
      \n
    1. \n

      \n Create the disposable component – the\n Lingo3GClusteringAlgorithm\n instance.\n

      \n
      2. \n\n
    2. \n

      \n Prepare input for clustering – a stream of\n Document instances.\n

      \n
      3. \n\n
    3. \n

      \n Perform clustering, reusing theLanguageComponents\n instance loaded previously.\n

      \n
      4. \n
    \n
    3. \n
\n\n

\n The Basic clustering section shows code\n examples that implement the above steps.\n

\n
\n\n
\n

\n Basic clustering\n

\n\n

\n This section shows how to invoke Lingo3G clustering, illustrating each\n step with code taken from the\n E01_ClusteringBasics.java example class. It shows just the\n key elements required to process a stream of documents in English, without\n any parameter or language resource tuning.\n

\n\n

\n Make sure you install the license file\n before running any Java API code examples. For quick tests, you can\n copy the license file to your home directory.\n

\n\n
\n

\n Load reusable components\n

\n\n

\n First, load the reusable components – the default language\n resources for the English language. The\n LanguageComponents instance is thread-safe, so you should\n reuse it for all subsequent clustering calls.\n

\n\n
\n 
// Our documents are in English so we load appropriate language resources.\n// This call can be heavy and an instance of LanguageComponents should be\n// created once and reused across different clustering calls.\nLanguageComponents languageComponents = LanguageComponents.loader().load().language(\"English\");
\n
\n
\n\n
\n

\n Create disposable components\n

\n\n

\n Next, create the disposable component – a\n Lingo3GClusteringAlgorithm\n instance:\n

\n\n
\n 
Lingo3GClusteringAlgorithm algorithm = new Lingo3GClusteringAlgorithm();
\n
\n
\n\n
\n

\n Prepare documents\n

\n\n

\n Next, assemble the input for clustering – a stream of\n Document instances. Each document must implement a single\n method that presents its clusterable text fields to the algorithm:\n

\n\n
\n 
void visitFields(BiConsumer<String, String> fieldConsumer);
\n
\n\n

\n This example uses hardcoded values from an array defined in the\n ExamplesData class:\n

\n\n
\n 
static final String[][] DOCUMENTS_DATA_MINING = {\n  {\n    \"http://en.wikipedia.org/wiki/Data_mining\",\n    \"Data mining -\" + \" Wikipedia, the free \" + \"encyclopedia\",\n    \"Article about knowledge-discovery in databases (KDD), the practice \"\n        + \"of automatically \"\n        + \"searching large stores of data for patterns.\"\n  },
\n
\n\n

\n Use Java streams for an easy way to covert data into\n Document instances. Note that the code only submits the\n title and snippet fields for clustering, it\n omits the URL because it is not clusterable text.\n

\n\n
\n 
// Create a stream of \"documents\" for clustering.\n// Each such document provides text content fields to a visitor.\nStream<Document> documentStream =\n    Arrays.stream(ExamplesData.DOCUMENTS_DATA_MINING)\n        .map(\n            fields ->\n                (fieldVisitor) -> {\n                  fieldVisitor.accept(\"title\", fields[1]);\n                  fieldVisitor.accept(\"content\", fields[2]);\n                });
\n
\n\n

\n Because the Document interface has a single method, you can\n implement it with a lambda expression.\n

\n
\n\n
\n

\n Perform clustering\n

\n\n

\n Finally, invoke clustering, passing the document stream and the language\n components instance. This example just prints the top-level cluster\n labels and document counts to the console:\n

\n\n
\n 
List<Cluster<Document>> clusters;\nclusters = algorithm.cluster(documentStream, languageComponents);\n\nExamplesCommon.printClusters(clusters);
\n
\n\n

When executed, this example should result in this output:

\n\n
\n 
Knowledge Discovery [docs: 0, score: 1.00]\n  Software [docs: 4, score: 1.00]\n  Databases [docs: 3, score: 0.94]\n  Application Areas [docs: 2, score: 0.89]\n  Process [docs: 2, score: 0.87]\n  Market [docs: 2, score: 0.86]\nData-mining Software [docs: 0, score: 0.98]\n  Model [docs: 3, score: 1.00]\n  Data Mining and Knowledge Discovery [docs: 4, score: 0.99]\n  Data Mining Applications [docs: 2, score: 0.95]\n  Development [docs: 2, score: 0.90]\n  Field [docs: 2, score: 0.90]\nApplications [docs: 0, score: 0.94]\n  Application Areas [docs: 3, score: 1.00]\n  Algorithms [docs: 2, score: 0.92]\n  Data Mining Software [docs: 2, score: 0.91]\n  SIAM International Conference on Data Mining [docs: 2, score: 0.90]\n  Customers [docs: 2, score: 0.90]\n  Standard [docs: 2, score: 0.90]\n  Trends [docs: 2, score: 0.90]\nText Mining [docs: 8, score: 0.92]\nData Mining Tools [docs: 9, score: 0.92]\nTechniques [docs: 10, score: 0.92]\nData Mining Technology [docs: 8, score: 0.89]\nData Mining Solutions [docs: 7, score: 0.88]\nConference [docs: 8, score: 0.88]\nData Mining Group [docs: 7, score: 0.86]\nBook [docs: 6, score: 0.84]\nData Analysis [docs: 6, score: 0.83]\nPractice [docs: 6, score: 0.83]\nData Management [docs: 5, score: 0.82]\nOracle Data Mining [docs: 4, score: 0.80]\nData Warehousing [docs: 3, score: 0.75]\nAssociation [docs: 3, score: 0.73]\nMicrosoft SQL Server [docs: 2, score: 0.69]\nData Mining Project [docs: 2, score: 0.69]\nCentral Connecticut State University [docs: 2, score: 0.69]\nProcess of Extracting [docs: 2, score: 0.68]\nCCSU [docs: 2, score: 0.65]\nSPSS [docs: 2, score: 0.65]\nDan [docs: 2, score: 0.65]\nScience [docs: 2, score: 0.65]\nSuccess [docs: 2, score: 0.65]\nBusiness Intelligence [docs: 3, score: 0.65]\nPredictive Analytics [docs: 3, score: 0.65]
\n
\n
\n
\n\n
\n

\n Iterating through results\n

\n\n

\n The Cluster class comes with utility methods to retrieve the\n label, score, list of contained documents and sub-clusters of a cluster.\n For example:\n

\n\n
\n 
/** Returns all documents that belong directly to this cluster. */\npublic List<T> getDocuments() {\n  return documents;\n}
\n
\n\n

\n A method printing a cluster summary, along with subclusters, can look like\n this:\n

\n\n
\n 
static <T> void printClusters(List<Cluster<T>> clusters, String indent) {\n  for (Cluster<T> c : clusters) {\n    System.out.printf(\n        Locale.ROOT,\n        indent + \"%s [docs: %,d, score: %.2f]\\n\",\n        String.join(\", \", c.getLabels()),\n        c.getDocuments().size(),\n        c.getScore());\n    printClusters(c.getClusters(), indent + \"  \");\n  }\n}
\n
\n\n

\n Note that the document within the cluster has a generic type identical to\n the type of the documents you pass for clustering. Therefore, you can add\n domain-specific fields and methods to your\n Document implementations and use them when consuming the\n clusters.\n

\n
\n\n
\n

\n Changing parameters\n

\n\n

\n Lingo3G comes with a number of parameters to adjust its behavior. The\n Lingo3GClusteringAlgorithm class exposes the parameters as\n public fields. To modify a parameter, use the set() method of\n the corresponding field.\n

\n\n

\n The following example restricts the cluster hierarchy depth to one level\n and filters out one-word labels:\n

\n\n
\n 
Lingo3GClusteringAlgorithm algorithm = new Lingo3GClusteringAlgorithm();\nalgorithm.labels.queryHint.set(\"data mining\");\nalgorithm.labels.minLabelWords.set(2);\nalgorithm.hierarchy.maxHierarchyDepth.set(1);
\n
\n\n

and produces this output:

\n\n
\n 
Knowledge Discovery [docs: 13, score: 1.00]\nData-mining Software [docs: 13, score: 0.98]\nData Mining Applications [docs: 11, score: 0.93]\nText Mining [docs: 8, score: 0.92]\nData Mining Tools [docs: 9, score: 0.92]\nData Mining Techniques [docs: 10, score: 0.91]\nData Mining Technology [docs: 8, score: 0.88]\nData Mining Solutions [docs: 7, score: 0.88]\nData Mining Conference [docs: 8, score: 0.86]\nData Mining Group [docs: 7, score: 0.85]\nData Analysis [docs: 6, score: 0.83]\nBook on Data Mining [docs: 6, score: 0.82]\nData Management [docs: 5, score: 0.81]\nOracle Data Mining [docs: 4, score: 0.80]\nIntroduction to Data Mining [docs: 4, score: 0.77]\nData Warehousing [docs: 3, score: 0.75]\nPredictive Analytics [docs: 3, score: 0.74]\nMicrosoft SQL Server [docs: 2, score: 0.69]\nCentral Connecticut State University [docs: 2, score: 0.69]\nData Mining Project [docs: 2, score: 0.69]\nProcess of Extracting [docs: 2, score: 0.68]\nBusiness Intelligence [docs: 3, score: 0.65]
\n
\n\n

\n Note that Lingo3G validates parameter values when you call the\n set() method. The method throws exceptions when you submit an\n incorrect, such as out-of-range, value.\n

\n\n

\n See\n JavaDoc documentation\n and the\n parameters reference\n page for a list and descriptions of the parameters.\n

\n
\n
","tableOfContents":[{"heading":"Dependency setup","anchor":"dependency-setup"},{"heading":"API overview","anchor":"api-overview"},{"heading":"Basic clustering","anchor":"basic-clustering","sections":[{"heading":"Load reusable components","anchor":"basic-clustering:load-reusable-components"},{"heading":"Create disposable components","anchor":"basic-clustering:create-disposable-components"},{"heading":"Prepare documents","anchor":"basic-clustering:prepare-documents"},{"heading":"Perform clustering","anchor":"basic-clustering:perform-clustering"}]},{"heading":"Iterating through results","anchor":"iterating-results"},{"heading":"Changing parameters","anchor":"changing-parameters"}]}},"pageContext":{"slug":"/java-api-basics/"}},"staticQueryHashes":["1164750013"]}