Domino Code Fragment

Code Name*
STRING TOKENIZING		Date*
04/28/2024		Source (or email address if you prefer)*
Rlatulippe@romac.com
IP address:.3.12.71.237
Description*
STRING TOKENIZING. A very common programming problem is one where you have a string of characters, that you'd like to split into individual chunks (or tokens). Typically, white space (created by inserting spaces and tabs) is used to separate one token from the next. The Java language provides an easy way to solve this problem: the java.util.StringTokenizer class. You create a new instance of StringTokenizer, specifying a string to be tokenized. Then hasMoreTokens and nextToken are called repeatedly to obtain the next token from the string.		Type*
Java		Categories*
(Misc)
Implementation:
Required Client:
Server:
Limitations:
Comments:
Files/Graphics attachments (if applicable): Code:

To see how tokenizing works, consider the following example:

import java.util.StringTokenizer;

public class token {

public static void main(String args[])
{
String testdata = "123,456, 789,,,1011 \n 1213";

StringTokenizer st = new StringTokenizer(testdata, " ,\t\r\n");
while (st.hasMoreTokens()) {
String token = st.nextToken();
System.out.println(token);
}
}

}

In the example, a string of numeric test data is set up, and a
StringTokenizer object is established to parse the string. By default, the
standard delimiters for tokenizing are " \t\r\n", and the example adds
comma to the standard list. hasMoreTokens is called to determine whether
the string of data has been exhausted, and if not, nextToken is called to
retrieve the next token. In this example tokens are numeric values that
represent fields of some type.

The program output is the following:

123
456
789
1011
1213

StringTokenizer has additional useful facilities. If you are interested in
learning more about them, one method to try is countTokens, a method that
returns a count of the number of tokens in the specified string.

THE ENUMERATION INTERFACE. In the string tokenizing example above, a
string is tokenized, and the tokens are retrieved and printed in turn. The
example above illustrates a general concept: enumeration. That is, all
the values in a list of some type are returned one at a time until the list
is exhausted. In the above example the list is a list of tokens extracted
from a string.

The Java utility library provides an interface to capture this type of
behavior. java.util.Enumeration declares two methods: hasMoreElements and
nextElement. The first of these returns a true/false value, according to
whether a given enumeration is exhausted. The second method returns the
next element as an Object reference.

Enumeration is an interface, meaning that it simply describes methods that
must be present in any class that implements the interface. The
description doesn't say anything about how those methods are implemented.
To make the idea of enumerations more concrete, here's a specific example
that defines a Stepper class to step through the values in an array:

import java.util.Enumeration;

class Stepper implements Enumeration {

private Object vec[];
private int currpos;

public Stepper(Object x[])
{
vec = x;
currpos = 0;
}

public boolean hasMoreElements()
{
return currpos < vec.length;
}

public Object nextElement()
{
return vec[currpos++];
}

}

public class enum {

public static void main(String args[])
{
String data[] = {"alpha", "beta", "gamma", "delta"};

Stepper st = new Stepper(data);

while (st.hasMoreElements())
System.out.println(st.nextElement());
}

}

Stepper implements the Enumeration interface, so it must define the methods
hasMoreElements and nextElement. The implementation simply defines an
internal variable "currpos" that defines the current position in the array.
If this value is less than the array's length, there are more elements, and
the next element is retrieved based on the current array position, which is
then incremented.

Stepper will work with any array of objects. String is a subclass of
Object, and as such, a String[] array reference can be assigned to an
Object[] array reference (via a "widening" conversion).

Finally, the Enumeration interface is particularly useful in cases where
you want to hide the details of the enumerated data structure. For
example, a common error might be to assume that the indices of an array
range in value from 1 to the array length, when, in fact, the range is from
0 to the length - 1. Using an enumeration hides this detail, and is less
error-prone than explicitly indexing through the array.