Fundamentals

Token

A token is the fundamental unit of processing in large language models. Rather than processing raw characters, LLMs break down text into tokens using a tokenizer. A single token typically represents 3-4 characters or roughly 75% of an English word. The number of tokens matters because it directly affects latency, memory usage, and API costs. Understanding token counts is critical when working with APIs—what looks like a short prompt might consume hundreds of tokens. Different models use different tokenizers, so token counts can vary between providers.