weka.core.tokenizers

## Class AlphabeticTokenizer

- java.lang.Object
- - weka.core.tokenizers.Tokenizer
  - - weka.core.tokenizers.AlphabeticTokenizer

- All Implemented Interfaces:
    java.io.Serializable, java.util.Enumeration<java.lang.String>, OptionHandler, RevisionHandler

      public class AlphabeticTokenizer
      extends Tokenizer

  Alphabetic string tokenizer, tokens are to be formed only from contiguous alphabetic sequences.

  Version:
    $Revision: 10203 $
  Author:
    Asrhaf M. Kibriya (amk14@cs.waikato.ac.nz), FracPete (fracpete at waikato dot ac dot nz)
  See Also:
    Serialized Form

- - ### Constructor Summary

    Constructors
    Constructor and Description
    `AlphabeticTokenizer()`

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.lang.String` | `getRevision()` Returns the revision string.
    `java.lang.String` | `globalInfo()` Returns a string describing the stemmer
    `boolean` | `hasMoreElements()` returns whether there are more elements still
    `static void` | `main(java.lang.String[] args)` Runs the tokenizer with the given options and strings to tokenize.
    `java.lang.String` | `nextElement()` returns the next element
    `void` | `tokenize(java.lang.String s)` Sets the string to tokenize.

    - ### Methods inherited from class weka.core.tokenizers.Tokenizer

      `getOptions, listOptions, runTokenizer, setOptions, tokenize`

    - ### Methods inherited from class java.lang.Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Constructor Detail

    - #### AlphabeticTokenizer

          public AlphabeticTokenizer()

  - ### Method Detail

    - #### globalInfo

          public java.lang.String globalInfo()

      Returns a string describing the stemmer
      Specified by:
        `globalInfo` in class `Tokenizer`
      Returns:
        a description suitable for displaying in the explorer/experimenter gui

    - #### hasMoreElements

          public boolean hasMoreElements()

      returns whether there are more elements still
      Specified by:
        `hasMoreElements` in interface `java.util.Enumeration<java.lang.String>`
      Specified by:
        `hasMoreElements` in class `Tokenizer`
      Returns:
        true if there are still more elements

    - #### nextElement

          public java.lang.String nextElement()

      returns the next element
      Specified by:
        `nextElement` in interface `java.util.Enumeration<java.lang.String>`
      Specified by:
        `nextElement` in class `Tokenizer`
      Returns:
        the next element

    - #### tokenize

          public void tokenize(java.lang.String s)

      Sets the string to tokenize. Tokenization happens immediately.
      Specified by:
        `tokenize` in class `Tokenizer`
      Parameters:
        `s` - the string to tokenize

    - #### getRevision

          public java.lang.String getRevision()

      Returns the revision string.
      Returns:
        the revision

    - #### main

          public static void main(java.lang.String[] args)

      Runs the tokenizer with the given options and strings to tokenize. The tokens are printed to stdout.
      Parameters:
        `args` - the commandline options and strings to tokenize