jsimple.util

Class StringTokenizer

java.lang.Object

jsimple.util.StringTokenizer

public class StringTokenizer
extends java.lang.Object

This class was based on, and modified from, the Apache Harmony java.util.StringTokenizer class. Unlike the standard Java class, this class returns BasicException instead of NoSuchElementException when there are no more tokens & the caller didn't properly check for that case first (via calling hasMoreElements). Other that that pretty minor difference, it's the same as the standard Java class.

The StringTokenizer class allows an application to break a string into tokens by performing code point comparison. The StringTokenizer methods do not distinguish among identifiers, numbers, and quoted strings, nor do they recognize and skip comments.

The set of delimiters (the codepoints that separate tokens) may be specified either at creation time or on a per-token basis.

An instance of StringTokenizer behaves in one of three ways, depending on whether it was created with the returnDelimiters flag having the value true or false:

• If returnDelims is false, delimiter code points serve to separate tokens. A token is a maximal sequence of consecutive code points that are not delimiters.
• If returnDelims is true, delimiter code points are themselves considered to be tokens. In this case a token will be received for each delimiter code point.

A token is thus either one delimiter code point, or a maximal sequence of consecutive code points that are not delimiters.

A StringTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the code point processed.

A token is returned by taking a substring of the string that was used to create the StringTokenizer object.

Here's an example of the use of the default delimiter StringTokenizer :

 StringTokenizer st = new StringTokenizer("this is a test");
 while (st.hasMoreTokens()) {
     println(st.nextToken());
 }
 

This prints the following output:

     this
     is
     a
     test
 

Here's an example of how to use a StringTokenizer with a user specified delimiter:

 StringTokenizer st = new StringTokenizer(
         "this is a test with supplementary characters ???",
         " ?");
 while (st.hasMoreTokens()) {
     println(st.nextToken());
 }
 

This prints the following output:

     this
     is
     a
     test
     with
     supplementary
     characters
     ?
     ?
 

Constructor Summary

Constructors

Constructor and Description
StringTokenizer(java.lang.String string) Constructs a new StringTokenizer for the parameter string using whitespace as the delimiter.
StringTokenizer(java.lang.String string, java.lang.String delimiters) Constructs a new StringTokenizer for the parameter string using the specified delimiters.
StringTokenizer(java.lang.String string, java.lang.String delimiters, boolean returnDelimiters) Constructs a new StringTokenizer for the parameter string using the specified delimiters, returning the delimiters as tokens if the parameter returnDelimiters is true.

Method Summary

Modifier and Type	Method and Description
int	countTokens() Returns the number of unprocessed tokens remaining in the string.
boolean	hasMoreElements() Returns true if unprocessed tokens remain.
boolean	hasMoreTokens() Returns true if unprocessed tokens remain.
java.lang.Object	nextElement() Returns the next token in the string as an Object.
java.lang.String	nextToken() Returns the next token in the string as a String.
java.lang.String	nextToken(java.lang.String delims) Returns the next token in the string as a String.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StringTokenizer

public StringTokenizer(java.lang.String string)

Constructs a new StringTokenizer for the parameter string using whitespace as the delimiter. The returnDelimiters flag is set to false.

Parameters:

string - the string to be tokenized.

StringTokenizer

public StringTokenizer(java.lang.String string,
                       java.lang.String delimiters)

Constructs a new StringTokenizer for the parameter string using the specified delimiters. The returnDelimiters flag is set to false.

Parameters:

string - the string to be tokenized.

delimiters - the delimiters to use.

StringTokenizer

public StringTokenizer(java.lang.String string,
                       java.lang.String delimiters,
                       boolean returnDelimiters)

Constructs a new StringTokenizer for the parameter string using the specified delimiters, returning the delimiters as tokens if the parameter returnDelimiters is true.

Parameters:

string - the string to be tokenized.

delimiters - the delimiters to use.

returnDelimiters - true to return each delimiter as a token.

Method Detail

countTokens

public int countTokens()

Returns the number of unprocessed tokens remaining in the string.

Returns:

number of tokens that can be retrieved before an Exception will result from a call to nextToken().

hasMoreElements

public boolean hasMoreElements()

Returns true if unprocessed tokens remain. This method is implemented in order to satisfy the Enumeration interface.

Returns:

true if unprocessed tokens remain.

hasMoreTokens

public boolean hasMoreTokens()

Returns true if unprocessed tokens remain.

nextElement

public java.lang.Object nextElement()

Returns the next token in the string as an Object. This method is implemented in order to satisfy the Enumeration interface.

Returns:

next token in the string as an Object

Throws:

BasicException - if no tokens remain.

nextToken

public java.lang.String nextToken()

Returns the next token in the string as a String.

Returns:

next token in the string as a String.

Throws:

BasicException - if no tokens remain.

nextToken

public java.lang.String nextToken(java.lang.String delims)

Returns the next token in the string as a String. The delimiters used are changed to the specified delimiters.

Parameters:

delims - the new delimiters to use.

Returns:

next token in the string as a String.

Throws:

BasicException - if no tokens remain.