StringTokenizer in Java

Introduction to Stringtokenizer in Java

The java.util package has a class called StringTokenizer that is used to tokenize strings.

To put it another way, we can break a sentence up into its words and conduct several operations, such as counting the amount of tokens or splitting a sentence up into tokens.

This StringTokenizer includes constructors and methods that allow us to separate a sentence into tokens. StringTokenizer, tokenize the string on the basis of the delimiters provided to the String tokenizer class object.

Whitespaces, tab, newlines, carriage returns, and form feeds are considered common delimiters.

These delimiters are used by default, however users can define their own delimiters by including them as arguments in the parameter.

Constructors of the StringTokenizer Class

Three different function types are commonly described by StringTokenizer:

1. StringTokenizer(String str) Tokenization of a certain string that is supplied in the parameter is accomplished by this constructor.

All of the standard delimiters specified in the StringTokenizer class description are taken by constructor.

WhiteSpaces, newlines, tabs, carriage returns ("r"), line feeds ("n"), and form feeds ("f") are the delimiters.

2.StringTokenizer(String str, String delimiter) Based on the delimiter the user specifies in the input, this constructor implements string tokenization.

3.StringTokenizer(String str, String delimiter, boolean flag) This constructor has the ability to display the delimiter in addition to doing string tokenization based on the delimiter.

Methods of the StringTokenizer Class

1. int countTokens()

The countToken() method returns the number of tokens in the input string that are separated by any white space.

The number of tokens in the String can be determined using this approach, and we can easily process the entire string by using this number of tokens as a loop parameter.

2. nextToken()

The StringTokenizer class's nextToken() method returns the next token as a String. The first time we use it, it returns the following token as the string's first token.

3. String nextToken(String delimiter)

The only difference between the nextToken(String delimiter) and nextToken() methods is that the former returns the next token based on the delimiter we pass as an argument to the latter.

Any delimiter, including a symbol, a number, or a character, is acceptable.

4. boolean hasMoreTokens()

This method merely determines if the String contains any additional tokens. If a token is available, it returns true; otherwise, it returns false.

It can be used in the while loop to process the entire string up until there are no more tokens left.

5. Object nextElement()

The nextElement() method is comparable to the StringTokenizer class's nextToken() method;

The distinction is that, as opposed to the nextToken() method, which returns a String, it returns the value in the form of an Object.

Java Program to Illustrate StringTokenizer Class

Example1: In Example Below we are printing each tokens of StringTokenizer st.

Output:

Code Explanation

In Above code String tokenizer Object is created by passing string to Constructor.Each Token/Word is printed is using while loop.

Example2: In this code we are going to use String Tokenizer class to create Tokens with delimeter "," and print each tokens.

Output:

Code Explanation

In Above code as Tokens are created on based of delimeter passed in constructor(,) so 2 tokens are created.Each token is printed using while loop.

Example3: In this example we are going to create StringTokenizer Object and print each of Tokens with delimeter.

Output:

Code Explanation

In Above Code there is string,delimeter and Boolean parameter are passed.This boolean parameter(True) ensure that delimeter is printed.If False is passed then delimeter will not be printed.

StringTokenizer in a Real Use Case

Let's try utilising StringTokenizer in a practical scenario now.

In some cases, we attempt to read data from CSV files and parse it using the user-specified delimiter.

StringTokenizer makes it simple to reach the following:

Example: In this example we are going to read a csv file and convert it into StrinTokenizer Object and print each token of each line.

Output:

Code Explanation

In Above code We have creted a Object of StringBuffer class by passing example.csv file then read each line of csv using br.readline() and printed each token of each line using while loop.