What a token actually is
Language models do not read characters or words. They read tokens: chunks of text the tokenizer has learned to treat as single units. A common English word is usually one token, a rare one splits into two or three, and punctuation, whitespace and emoji all carry their own cost. That is why a 1,000-character prompt is never reliably 250 tokens, and why counting characters is a poor substitute for counting tokens.