String Tokenizer

Part of the Logic & Flow section of Coddy's Java journey — lesson 48 of 59.

StringTokenizer is used to break a string into tokens (smaller parts) based on a delimiter. By default, it uses whitespace as a delimiter.

Create a StringTokenizer with default delimiter (space):

StringTokenizer st = new StringTokenizer("Hello World Java");

Count and get tokens:

int count = st.countTokens();
// Gets number of tokens
String first = st.nextToken();
// Gets next token

After executing the above code, the variables contain:

count: 3
first: "Hello"

Important: Tokens are consumed!

When you call nextToken(), it removes that token from the tokenizer. The countTokens() method returns the number of remaining tokens.

StringTokenizer st = new StringTokenizer("Hello World Java");
System.out.println(st.countTokens());  // Prints: 3

st.nextToken();  // Consumes "Hello"
System.out.println(st.countTokens());  // Prints: 2 (not 3!)

st.nextToken();  // Consumes "World"
System.out.println(st.countTokens());  // Prints: 1

Tip: If you need to know the total token count, call countTokens() before consuming any tokens. Use hasMoreTokens() to check if there are more tokens available while iterating.

Challenge

Easy

Create a method named tokenizeText that takes two arguments:

A String (text) to tokenize
A String (delimiter) to split by

The method should:

Create a StringTokenizer with the given text and delimiter
Return a string containing the token count followed by each token on a new line

Return messages should be:

If text is null: return "Invalid text"
If delimiter is null: use space as delimiter
Format: "Token count: X\nToken: token1\nToken: token2\n..."

Try it yourself

import java.util.Scanner;
import java.util.StringTokenizer;

public class Main {
    public static String tokenizeText(String text, String delimiter) {
        // Write your code here
    }
    
    public static void main(String[] args) {
        Scanner scanner = new Scanner(System.in);
        String text = scanner.nextLine();
        String delimiter = scanner.nextLine();
        
        if (text.equals("null")) text = null;
        if (delimiter.equals("null")) delimiter = null;
        
        System.out.println(tokenizeText(text, delimiter));
    }
}