{"componentChunkName":"component---node-modules-carrotsearch-gatsby-theme-apidocs-templates-documentation-page-js","path":"/parameter-tuning/","result":{"data":{"html":{"frontmatter":{"id":"parameter-tuning","title":"Parameter tuning"},"html":"
\n

Parameter tuning

\n\n

\n This chapter answers the most commonly asked questions about Lingo3G\n clustering quality or performance tuning.\n

\n\n
\n

\n How can I remove or promote certain phrases?\n

\n\n

\n Lingo3G has an advanced model of boosting or penalizing potential cluster\n phrases. See the Label dictionary\n chapter for more information.\n

\n
\n\n
\n

\n Why are not all documents assigned to clusters?\n

\n\n

\n When Lingo3G discovers a set of clusters, it assigns documents relevant to\n these clusters and may leave a certain portion of the input unassigned\n anywhere. Documents not referenced from any cluster are essentially\n outliers: they are not sufficiently related to any other document\n in the input.\n

\n\n

\n The fraction of unassigned documents increases with the number of input\n documents as Lingo3G by default tries to output a set of clusters that is\n concise and readable to human users. In certain applications, for example\n when you pass cluster labels to another automated step, you can increase\n the number of clusters and therefore decrease the number of unassigned\n documents.\n

\n\n

\n To reduce the number of unassigned documents, apply the following\n parameter changes:\n

\n\n
    \n
  1. \n

    \n Increase\n maxClusteringPassesTop\n above the default value or set it to zero to force Lingo3G to create\n as many clusters as possible.\n

    \n
    2. \n
  2. \n

    \n Increase\n documentCoverageTarget\n above the default value.\n

    \n
    3. \n
  3. \n

    \n Increase\n singleWordLabelWeight\n above the default value. This will increase the number of one-word\n labels, which may not always be desirable.\n

    \n
    4. \n
  4. \n

    \n When you cluster more than a hundred documents, you can further reduce\n the number of unassigned documents by lowering\n wordDfThresholdScalingFactor\n and\n phraseDfThresholdScalingFactor. This will force Lingo3G to consider lower-frequency words and\n phrases hence create more clusters. Note that lowering these\n parameters may significantly increase processing time.\n

    \n
    5. \n
\n
\n\n
\n

\n How to make more general clusters?\n

\n\n

\n To make the clusters more general (containing more documents, covering\n broader topics), apply the following parameter changes:\n

\n\n
    \n
  1. \n

    \n Increase\n singleWordLabelWeight\n above the default value, possibly up to 1.00. Note that this will\n increase the number of one-word labels, which may not always be\n desirable.\n

    \n
    2. \n\n
  2. \n

    \n Increase\n maxClusterSize\n above the default value, possibly up to 1.00.\n

    \n
    3. \n\n
  3. \n

    \n Increase\n minClusterSize\n in steps of 0.01 to eliminate the clusters with smallest numbers of\n documents.\n

    \n
    4. \n\n
  4. \n

    \n To further increase the size of clusters, try lowering\n mergeThreshold. This will cause Lingo3G to merge similar clusters into larger ones.\n

    \n
    5. \n
\n
\n\n
\n

\n How to make more focused clusters?\n

\n\n

\n To make the clusters more specific (containing fewer documents, covering\n more narrow topics), apply the following parameter changes:\n

\n\n
    \n
  1. \n

    \n Decrease\n maxClusterSize\n below the default value to eliminate large clusters.\n

    \n
    2. \n\n
  2. \n

    \n Decrease\n maxClusteringPassesTop\n to 0 to force Lingo3G to create as many clusters as possible.\n

    \n
    3. \n\n
  3. \n

    \n If there are too many one-word meaningless cluster labels, try\n lowering\n singleWordLabelWeight. Setting this attribute to 0.00 will eliminate one-word labels\n altogether.\n

    \n
    4. \n
\n
\n
","tableOfContents":[{"heading":"How can I remove or promote certain phrases?","anchor":"faq-labels"},{"heading":"Why are not all documents assigned to clusters?","anchor":"faq-unclustered-documents"},{"heading":"How to make more general clusters?","anchor":"faq-general-clusters"},{"heading":"How to make more focused clusters?","anchor":"faq-specific-clusters"}]}},"pageContext":{"slug":"/parameter-tuning/"}},"staticQueryHashes":["1164750013"]}