scone.util.tokenstream
Class TokenInputStreamTokenizerImpl

java.lang.Object
  scone.util.tokenstream.TokenInputStream
      scone.util.tokenstream.TokenInputStreamTokenizerImpl

public class TokenInputStreamTokenizerImpl
extends TokenInputStream

converts a Reader into a TokenInputStream. If no Tokenizer is specified, the standard HTML tokenizer is used. Any Tokenizer can be specified to process HTML, XML or any other doument type.

Version:

1.0

Author:

Volkert Buchmann, IBM Almaden Research Center, (c) Copyright IBM Corp 2000

Constructor Summary

TokenInputStreamTokenizerImpl(java.io.Reader reader)
creates a new TokenInputStreamTokenizer that parses a Reader into HTML tokens.

TokenInputStreamTokenizerImpl(java.io.Reader reader, java.lang.String tokenizer)
creates a new TokenInputStreamTokenizerImpl that parses a Reader into Tokens using the specified Tokenizer.

Method Summary

void	close() Close the input stream.
java.util.Hashtable	getMetaInfo() returns the meta info associated with this TokenInputStream
boolean	isClosed() Determines if the input stream is closed.
void	mark(int readAheadLimit) Mark the stream for later resetting
boolean	markSupported() Test whether mark() and reset() are supported.
Token	read() Read a Token from the input stream.
int	read(Token[] buf) Read Tokens from the input stream filling the specified Token array.
int	read(Token[] buf, int offset, int length) Read multiple Tokens from the input stream.
void	reset() Reset the stream to its previous mark

Methods inherited from class scone.util.tokenstream.TokenInputStream

create, create

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenInputStreamTokenizerImpl

public TokenInputStreamTokenizerImpl(java.io.Reader reader)

creates a new TokenInputStreamTokenizer that parses a Reader into HTML tokens.

Parameters:

reader - the Reader

TokenInputStreamTokenizerImpl

public TokenInputStreamTokenizerImpl(java.io.Reader reader,
                                     java.lang.String tokenizer)

creates a new TokenInputStreamTokenizerImpl that parses a Reader into Tokens using the specified Tokenizer.

Parameters:

reader - the Reader

tokenizer - the complete class name of the Tokenizer

Method Detail

read

public Token read()
           throws java.io.IOException

Read a Token from the input stream. Blocks if no data available and end of stream not yet reached. If no Token is available because the end of the stream has been reached, null is returned.

Specified by:

read in class TokenInputStream

Returns:

The next Token from the stream, or null if the end of the stream has been reached

Throws:

java.io.IOException - If stream closed.

MarkPreservedException - If read would invalidate preserved mark

read

public int read(Token[] buf,
                int offset,
                int length)
         throws java.io.IOException

Read multiple Tokens from the input stream. Blocks if no data available and end of stream not yet reached.

Specified by:

read in class TokenInputStream

Parameters:

buf - The Token buffer to populate.

offset - The offset in buf to starting writing to.

length - The amount of Tokens to write.

Returns:

The amount of Tokens read, or -1 if no more Tokens available to read.

Throws:

java.io.IOException - If stream closed.

MarkPreservedException - If read would invalidate preserved mark

read

public int read(Token[] buf)
         throws java.io.IOException

Read Tokens from the input stream filling the specified Token array. Blocks if no data available and end of stream not yet reached.

Specified by:

read in class TokenInputStream

Parameters:

buf - The Token buffer to populate.

Returns:

The amount of Tokens read, or -1 if no more Tokens available to read.

Throws:

java.io.IOException - If stream closed.

MarkPreservedException - If read would invalidate preserved mark

close

public void close()
           throws java.io.IOException

Close the input stream. If the stream is already closed, this method has not effect.

Specified by:

close in class TokenInputStream

Throws:

java.io.IOException

isClosed

public boolean isClosed()

Determines if the input stream is closed.

Specified by:

isClosed in class TokenInputStream

Returns:

true if stream closed; else false.

markSupported

public boolean markSupported()

Test whether mark() and reset() are supported.

Specified by:

markSupported in class TokenInputStream

Returns:

true if mark() and reset() are supported, false otherwise.

mark

public void mark(int readAheadLimit)
          throws java.io.IOException

Mark the stream for later resetting

Specified by:

mark in class TokenInputStream

Parameters:

readlimit - The number of Tokens read beyond which the mark will become invalid.

Throws:

java.io.IOException

reset

public void reset()
           throws java.io.IOException

Reset the stream to its previous mark

Specified by:

reset in class TokenInputStream

Throws:

java.io.IOException - if the stream was not marked or the mark has become invalid.

getMetaInfo

public java.util.Hashtable getMetaInfo()

returns the meta info associated with this TokenInputStream

Specified by:

getMetaInfo in class TokenInputStream

Returns:

the metaInfo