Uses of Class
opennlp.tools.tokenize.TokenSample

Packages that use TokenSample
opennlp.tools.cmdline.tokenizer   
opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format. 
opennlp.tools.tokenize Contains classes related to finding token or words in a string. 
opennlp.tools.tokenize.lang.en   
 

Uses of TokenSample in opennlp.tools.cmdline.tokenizer
 

Methods in opennlp.tools.cmdline.tokenizer that return types with arguments of type TokenSample
protected  ObjectStreamFactory<TokenSample> TokenizerConverterTool.createStreamFactory(String format)
           
 

Methods in opennlp.tools.cmdline.tokenizer with parameters of type TokenSample
 void TokenEvaluationErrorListener.missclassified(TokenSample reference, TokenSample prediction)
           
 

Uses of TokenSample in opennlp.tools.formats
 

Methods in opennlp.tools.formats that return TokenSample
 TokenSample NameToTokenSampleStream.read()
           
 TokenSample POSToTokenSampleStream.read()
           
 

Methods in opennlp.tools.formats that return types with arguments of type TokenSample
 ObjectStream<TokenSample> POSToTokenSampleStreamFactory.create(String[] args)
           
 ObjectStream<TokenSample> NameToTokenSampleStreamFactory.create(String[] args)
           
 ObjectStream<TokenSample> ConllXTokenSampleStreamFactory.create(String[] args)
           
 

Uses of TokenSample in opennlp.tools.tokenize
 

Methods in opennlp.tools.tokenize that return TokenSample
static TokenSample TokenSample.parse(String sampleString, String separatorChars)
           
protected  TokenSample TokenizerEvaluator.processSample(TokenSample reference)
           
 TokenSample TokenSampleStream.read()
           
 TokenSample TokenizerStream.read()
           
 

Methods in opennlp.tools.tokenize with parameters of type TokenSample
protected  Iterator<opennlp.model.Event> TokSpanEventStream.createEvents(TokenSample tokenSample)
          Adds training events to the event stream for each of the specified tokens.
protected  TokenSample TokenizerEvaluator.processSample(TokenSample reference)
           
 

Method parameters in opennlp.tools.tokenize with type arguments of type TokenSample
 void TokenizerCrossValidator.evaluate(ObjectStream<TokenSample> samples, int nFolds)
          Starts the evaluation.
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, boolean useAlphaNumericOptimization)
          Trains a model for the TokenizerME with a default cutoff of 5 and 100 iterations.
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, boolean useAlphaNumericOptimization, int cutoff, int iterations)
          Deprecated. use TokenizerME.train(String, ObjectStream, boolean, TrainingParameters) instead and pass in a TrainingParameters object.
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, boolean useAlphaNumericOptimization, TrainingParameters mlParams)
          Trains a model for the TokenizerME.
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, Dictionary abbreviations, boolean useAlphaNumericOptimization, TrainingParameters mlParams)
          Trains a model for the TokenizerME.
 

Constructor parameters in opennlp.tools.tokenize with type arguments of type TokenSample
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics)
          Initializes the current instance.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg)
          Initializes the current instance.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)
          Initializes the current instance.
WhitespaceTokenStream(ObjectStream<TokenSample> tokens)
           
 

Uses of TokenSample in opennlp.tools.tokenize.lang.en
 

Methods in opennlp.tools.tokenize.lang.en that return TokenSample
 TokenSample TokenSampleStream.next()
           
 



Copyright © 2011 The Apache Software Foundation. All Rights Reserved.