All About Class StringTokenizer in Java

0
165
Spread the love
  • 29
  • 34
  • 28
  • 35
  •  
  •  
  •  
  • 12
  •  
  •  
    138
    Shares

StringTokenizer in Java
Class StringTokenizer in Java
StringTokenizer in Java

StringTokenizer in Java:

The StringTokenizer class allows us to break a string into tokens with the specified characters as a delimiter. The tokenization method is much simpler than the one used by the StringTokenizer class.

The StringTokenizer class is also used to recover the words in multiword String. It is used for reading the input. To use StringTokenizer we need to import java.util.StringTokenizer class to our code.

The StringTokenizer does not distinguish among identifiers, numbers and quoted Strings. It also does not recognize and skip comments. The set of delimiters (the characters that separate tokens) may be specified either at creation time or on a per-token basis.

For an example of StringTokenizer behaves in one of the two ways, depending on whether it was created with the return tokens flag. This flag can have two values true or false.
StringTokenizer class implements the Enumeration interface hence we can access elements with hasMoreElements() and nextElement() methods.

In addition, It is often useful to break a set of text apart into tokens, where the characters separating the tokens can be specified.

The StringTokenizer class provides that. We can create an instance of the StringTokenizer class using a String and an optional second String as a separator.

Once it is done, we can use hasMoreTokens() method and nextToken() methods to fetch the tokens from the String.

When we create a StringTokenizer, we can specify a string of delimiter characters to use for the entire string or we can use the default delimiter as whitespace.

We can also specify if the delimiters themselves should be returned as tokens or not.

Token

The individual words after StringTokenizer generates are called Tokens.

Case-1:: If the flag is false:

If the flag is false, delimiters’ characters merely serve to separate tokens.

A token is a maximal sequence of consecutive characters that are delimiters.

Case-2:: If the flag is true:

If the flag is true, delimiters’ characters are considered to be tokens.

A token is either one delimiter character or a maximal sequence of consecutive characters that are not delimiters.

How to work:

StringTokenizer  st=new StringTokenizer("I love Java");
while(st.hasMoreToken())
{
System.out.println(st.nextToken());
} 

The output of the code is:
I
love
Java

When the default separator is blank or white space StringTokenizer class allows us to provide our own separator. For this, we need to call another version of the StringTokenizer constructor with the second argument as a separator.

StringTokenizer  st=new StringTokenizer("I(tab)love(tab)Java","\t");
while(st.hasMoreToken())
{
System.out.println(st.nextToken());
} 

The output of the code is:
I
love
Java

The structure of class StringTokenizer is given as:

public class java.util.StringTokenizer extends java.lang.Object implements 
java.util.Enumeration{
//constructors:
public StringTokenizer(String str);//constructs a StringTokenizer for the 
//specified String str.The tokenizer uses the default delimiter set,which 
//is "tnr",the space character,the tab character,the newline character 
//and the carriage return character.
public StringTokenizer(String str,String delimiter);//constructs a StringTokenizer
//for the specified String str.The character specified in the argument as String 
//delimiter is the delimiter for separating tokens.
public StringTokenizer(String str,String delimiter,boolean returnTokens);//constructs
//A StringTokenizer for the specified String str.The character specified in the
//argument as String delimiter is the delimiter for separating tokens.If the return
// Tokens flag is true,then the delimiter characters are also returned as Tokens.
//Each delimiter characters are also returned as a String of length one.
//If the flag is false,the delimiter characters are skipped and only serve as
//separators between tokens.
//Methods:
public int countTokens();
public boolean hasMoreElements();
public boolean hasMoreTokens();
public Object nextElement();
public String nextToken();
public String nextToken(String delimit)
} 
hasMoreToken()

This method tests if there are more tokens available from this tokenizer’s String. When used in conjunction with nextToken(), it returns true as long as nextToken has not yet returned all the tokens in the String. Otherwise, it returns false.

nextToken()

Returns the next token from this tokenizer’s String. It may throw NoSuchElementException if there are no more tokens to return.

nextToken(String delimiters)

This method first changes the delimiter character as the argument. Then it returns the next token from this Tokenizer’s string. After the invocation is completed, the delimiter character is the same as passed to the argument. It may throw NoSuchElementException if there are no more tokens to return. It may throw NullPointerException if the delimiter character is null.

countTokens()

countTokens() method returns the number of tokens remaining to be returned by nextToken() method.

Don't miss out!
Subscribe To Newsletter

Receive top technical news, lesson ideas, travel tips and more!

Invalid email address
Give it a try. You can unsubscribe at any time.

Spread the love
  • 29
  • 34
  • 28
  • 35
  •  
  •  
  •  
  • 12
  •  
  •  
    138
    Shares

LEAVE A REPLY

Please enter your comment!
Please enter your name here