opennlp.tools.tokenize
Class TokenizerModel

java.lang.Object
  extended by opennlp.tools.util.model.BaseModel
      extended by opennlp.tools.tokenize.TokenizerModel

public final class TokenizerModel
extends BaseModel

The TokenizerModel is the model used by a learnable Tokenizer.

See Also:
TokenizerME

Field Summary
 
Fields inherited from class opennlp.tools.util.model.BaseModel
TRAINING_CUTOFF_PROPERTY, TRAINING_EVENTHASH_PROPERTY, TRAINING_ITERATIONS_PROPERTY
 
Constructor Summary
TokenizerModel(java.io.InputStream in)
          Initializes the current instance.
TokenizerModel(java.lang.String language, AbstractModel tokenizerMaxentModel, boolean useAlphaNumericOptimization)
          Initializes the current instance.
TokenizerModel(java.lang.String language, AbstractModel tokenizerMaxentModel, boolean useAlphaNumericOptimization, java.util.Map<java.lang.String,java.lang.String> manifestInfoEntries)
          Initializes the current instance.
 
Method Summary
 AbstractModel getMaxentModel()
           
static void main(java.lang.String[] args)
           
 boolean useAlphaNumericOptimization()
           
 
Methods inherited from class opennlp.tools.util.model.BaseModel
getLanguage, getManifestProperty, getVersion, serialize
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TokenizerModel

public TokenizerModel(java.lang.String language,
                      AbstractModel tokenizerMaxentModel,
                      boolean useAlphaNumericOptimization,
                      java.util.Map<java.lang.String,java.lang.String> manifestInfoEntries)
Initializes the current instance.

Parameters:
tokenizerMaxentModel -
useAlphaNumericOptimization -

TokenizerModel

public TokenizerModel(java.lang.String language,
                      AbstractModel tokenizerMaxentModel,
                      boolean useAlphaNumericOptimization)
Initializes the current instance.

Parameters:
tokenizerMaxentModel -
useAlphaNumericOptimization -

TokenizerModel

public TokenizerModel(java.io.InputStream in)
               throws java.io.IOException,
                      InvalidFormatException
Initializes the current instance.

Parameters:
in -
Throws:
java.io.IOException
InvalidFormatException
Method Detail

getMaxentModel

public AbstractModel getMaxentModel()

useAlphaNumericOptimization

public boolean useAlphaNumericOptimization()

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException
Throws:
java.io.IOException


Copyright © 2011 The Apache Software Foundation. All Rights Reserved.