## Uses of Interface
opennlp.tools.tokenize.Detokenizer

- Packages that use Detokenizer
  Package | Description
  --- | ---
  opennlp.tools.formats | Experimental package related to converting various corpora to OpenNLP Format.
  opennlp.tools.formats.convert |
  opennlp.tools.formats.letsmt |
  opennlp.tools.sentdetect | Package related to identifying sentece boundries.
  opennlp.tools.tokenize | Contains classes related to finding token or words in a string.

- - ### Uses of Detokenizer in opennlp.tools.formats

    Methods in opennlp.tools.formats that return Detokenizer
    Modifier and Type | Method and Description
    --- | ---
    `protected Detokenizer` | DetokenizerSampleStreamFactory.`createDetokenizer(DetokenizerParameter p)`

  - ### Uses of Detokenizer in opennlp.tools.formats.convert

    Constructors in opennlp.tools.formats.convert with parameters of type Detokenizer
    Constructor and Description
    `NameToSentenceSampleStream(Detokenizer detokenizer, ObjectStream<NameSample> samples, int chunkSize)`
    `NameToTokenSampleStream(Detokenizer detokenizer, ObjectStream<NameSample> samples)`
    `POSToSentenceSampleStream(Detokenizer detokenizer, ObjectStream<POSSample> samples, int chunkSize)`
    `POSToTokenSampleStream(Detokenizer detokenizer, ObjectStream<POSSample> samples)`

  - ### Uses of Detokenizer in opennlp.tools.formats.letsmt

    Constructors in opennlp.tools.formats.letsmt with parameters of type Detokenizer
    Constructor and Description
    `DetokenizeSentenceSampleStream(Detokenizer detokenizer, ObjectStream<SentenceSample> samples)`

  - ### Uses of Detokenizer in opennlp.tools.sentdetect

    Constructors in opennlp.tools.sentdetect with parameters of type Detokenizer
    Constructor and Description
    `SentenceSample(Detokenizer detokenizer, String[][] sentences)`

  - ### Uses of Detokenizer in opennlp.tools.tokenize

    Classes in opennlp.tools.tokenize that implement Detokenizer
    Modifier and Type | Class and Description
    --- | ---
    `class` | `DictionaryDetokenizer` A rule based detokenizer.

    Constructors in opennlp.tools.tokenize with parameters of type Detokenizer
    Constructor and Description
    `DetokenizerEvaluator(Detokenizer detokenizer, DetokenEvaluationErrorListener... listeners)` Initializes the current instance with the given `Detokenizer`.
    `TokenSample(Detokenizer detokenizer, String[] tokens)`

Copyright © 2026 The Apache Software Foundation. All rights reserved.