Before text reaches a model it is split into tokens by a tokenizer. Modern systems use sub-word schemes such as byte-pair encoding (Sennrich et al., 2016) or SentencePiece (Kudo & Richardson, 2018), which break rare words into smaller pieces while keeping common words intact — balancing vocabulary size against sequence length.
A token is roughly three-quarters of an English word on average, but this varies by language and script. That is why context limits and API prices are quoted in tokens rather than words or characters.