WTF is really going down with a LLM, in lay speak
Dedicated to my good friend Terry. If he "gets it", mission accomplished.
LLM stands for large language model, the technology behind chatbots such as Claude. "Transformer" is the name of its design, not a gadget. This explainer grew out of a riddle Andrew set Claude, and the three tests below are where that conversation led.
A transformer takes your text, cuts it into small pieces called tokens, and turns each token into a long list of numbers. It then transforms those lists, again and again, through a stack of layers. Each pass refines what the model "knows" about each token in the light of the others. After the last layer it picks the next token, adds it to the text, and starts again. That is the whole trick: iterative transformations.
There are two loops, and people often blur them. One runs down through the layers for each token (the depth loop). The other runs round and round, one new token at a time, until the reply is finished (the time loop).
Every layer does two things in turn. Attention lets each token look at the others and decide which ones matter to it, worked out fresh from this particular text. The feed-forward step then reshapes what was gathered, using numbers that were learned once in training and are now frozen.
The learned numbers are static, and the attention is dynamic. That one contrast explains why a model can know a great deal and still miss something sitting right in front of it. Attention also works to a fixed budget: it spreads 100% across everything in view, so more text means a thinner slice for each part. It leans toward what looks relevant to the task of the moment, and toward the start and the end of a long text rather than the middle. None of this is laziness. It is arithmetic.
| Term | Plain words |
|---|---|
| Token | A small piece of text: a word, part of a word, or a symbol. The model never sees letters or sounds, only tokens. |
| Embedding | The list of numbers a token is turned into, so the model can work with it. |
| Layer | One round of transformation (attention, then feed-forward). Large models stack dozens. |
| Parameters (weights) | The learned numbers, often billions of them. Set in training, frozen when you chat. |
| Context window | How much text the model can look at in one go. |
| Decoder | In chatbot-style models, the whole stack that produces text. It is not a separate part. |
| Autoregressive | A long word for "one token at a time, each fed back in": the time loop above. |
| Sampling | The model scores every possible next token and one is picked, with some chance in it. That is why replies vary. |
| Logits | The raw score for every possible next token, before one is picked. Think of a scoreboard. |
| Temperature | A dial for how adventurous the pick is. Low means safe and predictable. High means more variety. |
| Attention head | One of many small spotters inside a layer. Each watches for a different kind of link between words, such as who "it" refers to. |
| Query, key, value | How attention matches by meaning, like a library. The query is your question, each key is the label on a shelf, and the value is the book you are handed when they match. |
| Softmax | The step that turns match scores into shares adding up to 100%. It is why attention is a fixed budget: more for one thing means less for the rest. |
| Residual connection | Each layer adds its changes to what came in instead of replacing it, so earlier information is not lost on the way up the stack. |
| Training vs inference | Training is the learning. Inference is the chatting, with the learned numbers frozen. |
| Hallucination | A fluent answer that is not grounded in anything real. The model picks likely words. It does not look facts up. |
You need no tools to learn where a model's limits are. You need small puzzles with a known answer, and a willingness to watch what the model does with them.
1. The riddle with a hidden answer. The one that started it all, posed by Andrew: "A cook who cannot rely on smell adds depth to savoury food with one secret ingredient, kept handy and used sparingly. What is it?"
The model reasoned from the clues (without smell only the tongue's tastes count, and "depth" means umami) and named MSG, then fish sauce. The real answer was nước mắm, its second guess. It shows: a model infers from context and offers the most probable answer first. The catch: a hit on an obvious puzzle proves little. Check by hand where the clues point two ways.
2. A made-up word that only works by sound. "... a nod to RESPECT, Elelemmm ... hmmm?"
Said aloud, "Elelemmm" is el-el-em, the letter names for LLM. The model read it as a typo until Andrew added a tag: phonetic. It shows: a model sees tokens, not sound. Quote marks and tags do not change how text is cut up, but they tell the model a strange word is deliberate, and that changes how it reads it.
3. The question with a trap in its framing. "Is it attention laziness, an assignment, or a default?"
None of the three options was the full truth, and "lazy" lends a machine a human trait. It shows: whether a model answers the question or just accepts the framing. The honest reply described the mechanism: a fixed budget of attention shared across everything in view.
Write a puzzle whose answer you know. Ask it a few times in fresh chats and compare. When it misses, add one small hint (a tag, a quote mark, a clue) and see whether the miss goes away. If it does, you have found a real, working limit of that model and a real workaround.
A simplified, general account. Real models differ in detail, and some of it is still active research. Written by Claude, an AI made by Anthropic, in conversation with Andrew ("Chip") on 20 October 2026.