com.topologi.diffx.load.text
Interface TextTokeniser

All Known Implementing Classes:

CharactersTokeniser, TextTokeniserByWord, TextTokeniserIgnoreSpace, TextTokeniserNoSpace

public interface TextTokeniser

An interface for text tokenisers.

Text tokenisers are used to return TextEvent from a piece of text.

Version:

3 February 2005

Author:

Christophe Lauret

Method Summary

int	countTokens() Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.
TextEvent	nextToken() Returns the following token.
void	useRepertory(Repertory repertory) Specifies a repertory to use for this tokeniser.

Method Detail

countTokens

public int countTokens()

Calculates the number of times that this tokenizer's nextToken method can be called before it generates an exception.

Returns:

The number of tokens.

nextToken

public TextEvent nextToken()
                    throws NoSuchElementException

Returns the following token.

Returns:

The character event.

Throws:

NoSuchElementException - If the last token has already been returned.

useRepertory

public void useRepertory(Repertory repertory)

Specifies a repertory to use for this tokeniser.

Tokenisers don't have to use a repertory, but they should specify whether they will use the repertory or not.

Parameters:

repertory - The repertory to use.