net.nutch.analysis
Class CommonGrams

java.lang.Object
  net.nutch.analysis.CommonGrams

public class CommonGrams

extends Object

Construct n-grams for frequently occuring terms and phrases while indexing. Optimize phrase queries to use the n-grams. Single terms are still indexed too, with n-grams overlaid. This is achieved through the use of Token.setPositionIncrement(int).

Method Summary

static TokenFilter	getFilter(TokenStream ts, String field) Construct a token filter that inserts n-grams for common terms.
static void	main(String[] args) For debugging.
static String[]	optimizePhrase(Query.Phrase phrase, String field) Optimizes phrase queries to use n-grams when possible.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getFilter

public static TokenFilter getFilter(TokenStream ts,
                                    String field)

Construct a token filter that inserts n-grams for common terms. For use while indexing documents.

optimizePhrase

public static String[] optimizePhrase(Query.Phrase phrase,
                                      String field)

Optimizes phrase queries to use n-grams when possible.

main

public static void main(String[] args)
                 throws Exception

For debugging.

Throws:

Exception