Words ≠ Tokens

A common mistake is assuming that one word equals one token. It doesn't work that way.

The word cat is one token. But cats? That might be two tokens: cat and s. The word hello is one token, but Hello with a capital H could be tokenized differently.

Spaces, punctuation, and capitalization all affect how text gets split.

Here are some surprising examples:

Text	Approximate Tokens
dog	1
dogs	1-2
extraordinary	2-3
123456	2-3
!!!	1-3

Numbers are especially tricky. You might think 1000 is one token, but it could be split into 100 and 0, or even digit by digit. This is why AI sometimes struggles with precise math—it's not seeing numbers the way you do.

The key insight: token counts are unpredictable if you're just counting words. A 100-word prompt might use 130 tokens, or 150, depending on the specific words you chose.

Watch the token counter in the chat panel as you type—you'll see it doesn't increase by exactly one per word.