Uses of Class
opennlp.tools.tokenize.TokenSample

Packages that use TokenSample
opennlp.tools.cmdline.tokenizer   
opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format. 
opennlp.tools.formats.ad   
opennlp.tools.formats.convert   
opennlp.tools.tokenize Contains classes related to finding token or words in a string. 
opennlp.tools.tokenize.lang.en   
 

Uses of TokenSample in opennlp.tools.cmdline.tokenizer
 

Methods in opennlp.tools.cmdline.tokenizer with parameters of type TokenSample
 void TokenEvaluationErrorListener.missclassified(TokenSample reference, TokenSample prediction)
           
 

Uses of TokenSample in opennlp.tools.formats
 

Methods in opennlp.tools.formats that return types with arguments of type TokenSample
 ObjectStream<TokenSample> TokenSampleStreamFactory.create(String[] args)
           
 ObjectStream<TokenSample> ConllXTokenSampleStreamFactory.create(String[] args)
           
 

Uses of TokenSample in opennlp.tools.formats.ad
 

Methods in opennlp.tools.formats.ad that return types with arguments of type TokenSample
 ObjectStream<TokenSample> ADTokenSampleStreamFactory.create(String[] args)
           
 

Uses of TokenSample in opennlp.tools.formats.convert
 

Methods in opennlp.tools.formats.convert that return TokenSample
 TokenSample POSToTokenSampleStream.read()
           
 TokenSample NameToTokenSampleStream.read()
           
 

Methods in opennlp.tools.formats.convert that return types with arguments of type TokenSample
 ObjectStream<TokenSample> POSToTokenSampleStreamFactory.create(String[] args)
           
 ObjectStream<TokenSample> NameToTokenSampleStreamFactory.create(String[] args)
           
 ObjectStream<TokenSample> ParseToTokenSampleStreamFactory.create(String[] args)
           
 

Uses of TokenSample in opennlp.tools.tokenize
 

Methods in opennlp.tools.tokenize that return TokenSample
static TokenSample TokenSample.parse(String sampleString, String separatorChars)
           
protected  TokenSample TokenizerEvaluator.processSample(TokenSample reference)
           
 TokenSample TokenSampleStream.read()
           
 TokenSample TokenizerStream.read()
           
 

Methods in opennlp.tools.tokenize with parameters of type TokenSample
protected  Iterator<opennlp.model.Event> TokSpanEventStream.createEvents(TokenSample tokenSample)
          Adds training events to the event stream for each of the specified tokens.
protected  TokenSample TokenizerEvaluator.processSample(TokenSample reference)
           
 

Method parameters in opennlp.tools.tokenize with type arguments of type TokenSample
 void TokenizerCrossValidator.evaluate(ObjectStream<TokenSample> samples, int nFolds)
          Starts the evaluation.
static TokenizerModel TokenizerME.train(ObjectStream<TokenSample> samples, TokenizerFactory factory, TrainingParameters mlParams)
          Trains a model for the TokenizerME.
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, boolean useAlphaNumericOptimization)
          Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters) and pass in a TokenizerFactory
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, boolean useAlphaNumericOptimization, int cutoff, int iterations)
          Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters) and pass in a TokenizerFactory
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, boolean useAlphaNumericOptimization, TrainingParameters mlParams)
          Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters) and pass in a TokenizerFactory
static TokenizerModel TokenizerME.train(String languageCode, ObjectStream<TokenSample> samples, Dictionary abbreviations, boolean useAlphaNumericOptimization, TrainingParameters mlParams)
          Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters) and pass in a TokenizerFactory
 

Constructor parameters in opennlp.tools.tokenize with type arguments of type TokenSample
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics)
          Initializes the current instance.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg)
          Initializes the current instance.
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)
          Initializes the current instance.
WhitespaceTokenStream(ObjectStream<TokenSample> tokens)
           
 

Uses of TokenSample in opennlp.tools.tokenize.lang.en
 

Methods in opennlp.tools.tokenize.lang.en that return TokenSample
 TokenSample TokenSampleStream.next()
           
 



Copyright © 2013 The Apache Software Foundation. All Rights Reserved.