opennlp.tools.tokenize
Class TokenizerModel
java.lang.Object
opennlp.tools.util.model.BaseModel
opennlp.tools.tokenize.TokenizerModel
public final class TokenizerModel
- extends BaseModel
The TokenizerModel
is the model used
by a learnable Tokenizer
.
- See Also:
TokenizerME
Constructor Summary |
TokenizerModel(InputStream in)
Initializes the current instance. |
TokenizerModel(String language,
opennlp.model.AbstractModel tokenizerMaxentModel,
boolean useAlphaNumericOptimization)
Initializes the current instance. |
TokenizerModel(String language,
opennlp.model.AbstractModel tokenizerMaxentModel,
boolean useAlphaNumericOptimization,
Map<String,String> manifestInfoEntries)
Initializes the current instance. |
TokenizerModel(String language,
opennlp.model.AbstractModel tokenizerMaxentModel,
Dictionary abbreviations,
boolean useAlphaNumericOptimization,
Map<String,String> manifestInfoEntries)
Initializes the current instance. |
TokenizerModel
public TokenizerModel(String language,
opennlp.model.AbstractModel tokenizerMaxentModel,
Dictionary abbreviations,
boolean useAlphaNumericOptimization,
Map<String,String> manifestInfoEntries)
- Initializes the current instance.
- Parameters:
tokenizerMaxentModel
- useAlphaNumericOptimization
-
TokenizerModel
public TokenizerModel(String language,
opennlp.model.AbstractModel tokenizerMaxentModel,
boolean useAlphaNumericOptimization,
Map<String,String> manifestInfoEntries)
- Initializes the current instance.
- Parameters:
language
- tokenizerMaxentModel
- useAlphaNumericOptimization
- manifestInfoEntries
-
TokenizerModel
public TokenizerModel(String language,
opennlp.model.AbstractModel tokenizerMaxentModel,
boolean useAlphaNumericOptimization)
- Initializes the current instance.
- Parameters:
language
- tokenizerMaxentModel
- useAlphaNumericOptimization
-
TokenizerModel
public TokenizerModel(InputStream in)
throws IOException,
InvalidFormatException
- Initializes the current instance.
- Parameters:
in
-
- Throws:
IOException
InvalidFormatException
getMaxentModel
public opennlp.model.AbstractModel getMaxentModel()
getAbbreviations
public Dictionary getAbbreviations()
useAlphaNumericOptimization
public boolean useAlphaNumericOptimization()
main
public static void main(String[] args)
throws IOException
- Throws:
IOException
Copyright © 2011 The Apache Software Foundation. All Rights Reserved.