com.topologi.diffx.load.text
Class TokenizerByWord

java.lang.Object
  com.topologi.diffx.load.text.TokenizerByWord

All Implemented Interfaces:

TextTokenizer

public final class TokenizerByWord
extends Object
implements TextTokenizer

The tokeniser for characters events.

This class is not synchronized.

Version:

11 May 2010

Author:

Christophe Lauret

Constructor Summary

TokenizerByWord(WhiteSpaceProcessing whitespace)
Creates a new tokenizer.

Method Summary

TextGranularity	granurality() Always TextGranularity.WORD.
List<TextEvent>	tokenize(CharSequence seq) Returns the list of TextEvent corresponding to the specified character sequence.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenizerByWord

public TokenizerByWord(WhiteSpaceProcessing whitespace)

Creates a new tokenizer.

Parameters:

whitespace - the whitespace processing for this tokenizer.

Throws:

NullPointerException - if the white space processing is not specified.

Method Detail

tokenize

public List<TextEvent> tokenize(CharSequence seq)

Returns the list of TextEvent corresponding to the specified character sequence.

Specified by:

tokenize in interface TextTokenizer

Parameters:

seq - the character sequence to tokenize.

Returns:

the corresponding list.

granurality

public TextGranularity granurality()

Always TextGranularity.WORD. Returns the text granularity of this tokenizer.

Specified by:

granurality in interface TextTokenizer

Returns:

the text granularity of this tokenizer.