How Claude actually thinks

Tokens, context, and the architecture behind every response

Why does this matter?

Most people use Claude without understanding how it processes text. That's fine — until you hit a wall. Your prompts get inconsistent results, Claude ignores something you said, or you burn through tokens for no reason. Understanding the architecture turns those mysteries into predictable behavior.

Step 1: Claude doesn't read text — it reads tokens

Claude doesn't process words. It processes tokens: fragments that are sometimes words, sometimes syllables, sometimes punctuation. The model converts text to numeric IDs before doing anything.

Tokenization exampletext

"The artificial intelligence is fascinating"
→ ["The", " artif", "icial", " intel", "ligence", " is", " fasci", "nating"]
→ [464, 34345, 1967, 39941, 44369, 318, 50421, 1174]

Spanish costs 2-3x more tokens than English — the tokenizer was trained mostly on English. Same information, higher price.
Numbers behave oddly: "127" might be 1 token or 3 tokens ("1","2","7"). This is why Claude is bad at arithmetic — it sees fragments, not numbers.
The 200k context window is tokens, not words. Roughly 150k English words, fewer in Spanish.

INFO

Practical implication: if cost matters, write prompts in English. If you need Spanish output, prompt in English and ask for Spanish response.

Step 2: Attention — how Claude connects distant ideas

Before Transformers (the architecture behind Claude), AI systems processed text left-to-right, word by word. By the time they reached "large" in "the trophy didn't fit in the suitcase because it was too large", the context of "trophy" had faded.

Attention solves this radically: every token can look directly at every other token in the context, regardless of distance. For each token, the model calculates how much attention it should pay to every other token.

Attention in actiontext

"The cat drank the milk because it was hungry"

When Claude processes "it":
  The    → 0.02
  cat    → 0.71  ← high weight: "it" refers to the cat
  drank  → 0.08
  milk   → 0.06
  ...

Claude learned that "it" most likely refers to "cat".
This updates the representation of "it" to carry info about who is hungry.

The "lost in the middle" phenomenon

With very long contexts, Claude remembers the beginning and end well — but information buried in the middle degrades. This is empirically documented and matters a lot for how you structure prompts.

TIP

Put critical instructions at the start of your prompt, and repeat the most important constraint at the end. The middle is no man's land. This is not a workaround — it's how the architecture works.

Temperature: not "creativity", something more precise

After processing, Claude produces a probability distribution over its full vocabulary (~100k tokens). Temperature modifies those probabilities before picking the next token.

Temperature 0: always picks the highest-probability token. Deterministic, predictable, sometimes repetitive.
Temperature 1: default. Proportional sampling from the distribution.
Temperature 2: flattens the distribution. Less-probable tokens get more chances. More varied, more risk of incoherence.

WARNING

"Creativity" is a metaphor. What temperature actually controls is how deterministic vs. varied the output is. For internal tools, company policies, or structured extraction: use low temperature. For brainstorming or creative writing: higher temperature.

Prompt caching: why keeping the same system prompt saves money

Anthropic caches tokens it has already processed. If you send the same system prompt in every request, those tokens cost ~10% of normal price. Claude Code exploits this: it keeps your CLAUDE.md at the start of every session so it's cached on repeated calls.