net.nutch.analysis
Class NutchAnalysisTokenManager
java.lang.Object
net.nutch.analysis.NutchAnalysisTokenManager
- All Implemented Interfaces:
- NutchAnalysisConstants
- public class NutchAnalysisTokenManager
- extends Object
- implements NutchAnalysisConstants
Fields inherited from interface net.nutch.analysis.NutchAnalysisConstants |
ACRONYM, APOSTROPHE, ATSIGN, CJK, COLON, DEFAULT, DIGIT, DOT, EOF, LETTER, MINUS, PLUS, QUOTE, SIGRAM, SLASH, tokenImage, WHITE, WORD, WORD_PUNCT |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
debugStream
public PrintStream debugStream
jjstrLiteralImages
public static final String[] jjstrLiteralImages
lexStateNames
public static final String[] lexStateNames
input_stream
protected net.nutch.analysis.CharStream input_stream
curChar
protected char curChar
NutchAnalysisTokenManager
public NutchAnalysisTokenManager(Reader reader)
- Constructs a token manager for the provided Reader.
NutchAnalysisTokenManager
public NutchAnalysisTokenManager(net.nutch.analysis.CharStream stream)
NutchAnalysisTokenManager
public NutchAnalysisTokenManager(net.nutch.analysis.CharStream stream,
int lexState)
setDebugStream
public void setDebugStream(PrintStream ds)
ReInit
public void ReInit(net.nutch.analysis.CharStream stream)
ReInit
public void ReInit(net.nutch.analysis.CharStream stream,
int lexState)
SwitchTo
public void SwitchTo(int lexState)
jjFillToken
protected net.nutch.analysis.Token jjFillToken()
getNextToken
public net.nutch.analysis.Token getNextToken()
Copyright © 2004 The Nutch Organization.