com.topologi.diffx.load.text
Class TextTokeniserByWord

java.lang.Object
  com.topologi.diffx.load.text.TextTokeniserByWord

All Implemented Interfaces:

TextTokeniser

public final class TextTokeniserByWord

extends Object

implements TextTokeniser

A tokeniser for characters events.

This tokeniser is a bit smarter than the other tokeniser, and more configurable; eventually, it will replace the CharactersTokeniser.

Version:

23 December 2004

Author:

Christophe Lauret

Constructor Summary

TextTokeniserByWord(CharSequence cs)
Creates a new tokeniser.

Method Summary

int	countTokens() Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.
TextEvent	nextToken() Returns the following token.
void	useRepertory(Repertory rep) Specifies a repertory to use for this tokeniser.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextTokeniserByWord

public TextTokeniserByWord(CharSequence cs)
                    throws NullPointerException

Creates a new tokeniser.

Parameters:

cs - The character sequence to tokenise.

Throws:

NullPointerException - If the specified character sequence is null.

Method Detail

countTokens

public int countTokens()

Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.

Specified by:

countTokens in interface TextTokeniser

Returns:

The number of tokens.

nextToken

public TextEvent nextToken()
                    throws NoSuchElementException

Returns the following token.

Specified by:

nextToken in interface TextTokeniser

Returns:

The character event.

Throws:

NoSuchElementException - If the last token has already been returned.

useRepertory

public void useRepertory(Repertory rep)

Specifies a repertory to use for this tokeniser.

Specified by:

useRepertory in interface TextTokeniser

Parameters:

rep - The repertory to use.