Transformers: Meaning in Context

The most common dismissal of a language model is that it only predicts the next word. The sentence is accurate and the dismissal is not. Next-token prediction is the objective the model was trained on and the way it produces text when you use it. It says nothing about what the model had to construct in order to do the job. Consider a human analogy: to predict the last word of a mystery novel you have to have followed the plot, kept the suspects straight, and noticed what the detective noticed. The task is prediction. The competence required to perform it is something else.

This essay is about the architecture in which that competence is built. The previous essay described training as turning billions of dials until predictions improve. The dials belong to a structure, and for almost every language model in use today the structure is a transformer. Nothing about a transformer is separate from the trained model. Its layers are where the parameters live. When people say a model “has” a transformer, they mean the shape of the arithmetic, and training fills that shape with values.

From text to numbers

A model never sees letters. Text is first cut into tokens, pieces that are often whole words and sometimes fragments of them. In English a token averages about three quarters of a word, and a vocabulary holds somewhere between tens of thousands and a few hundred thousand of them. Each token has a number.

That number is used to look up a vector: a list of a few thousand values that stands for the token. This table of vectors is learned during training like everything else, and at this stage it is static. The word “bank” fetches the same list of numbers every time, whether the sentence is about a river or a loan. The model also encodes each token’s position, so it can tell the first word from the fifth. The original design added a second table of numbers to each token’s vector for this. Most current models, including the Llama family, instead rotate the queries and keys inside attention by an amount that depends on position, so position enters the mixing step directly. From here on, the model works only with these lists of numbers, which the next essay examines more closely as representations.

The sentence that needs its neighbors

Take two sentences.

After the flood, the bank was underwater.

After the audit, the bank was underwater.

The word “bank” means a different thing in each, and “underwater” shifts with it, from a literal river edge to a figurative balance sheet. Any reader resolves this without noticing. A static table of vectors cannot, because it hands over the same “bank” both times. Whatever is going to tell the two apart has to look at “flood” and “audit.”

That looking is what attention does. Within each layer of the transformer, every token’s representation is recomputed as a weighted mixture of the representations at other positions, and the weights are computed from the input itself. Loosely: each token issues a query, each token offers a key and a value, and the degree to which one token’s content flows into another is set by how well the query matches the key. In the first sentence, the representation at “bank” draws heavily on “flood.” In the second, it draws on “audit.” The parameters that compute the queries, keys and values are the same in both cases. They were fixed by training. What differs is the text, and because the mixing weights depend on the text, the same fixed parameters produce different behavior for different sentences. That is the whole trick, and it is why the model can hold meaning in context without any rule that says “bank near flood means river.”

Several of these attention operations run in parallel in each layer, called heads, each free to learn a different kind of relation: one may track which noun a pronoun refers to, another which word is the object of a verb, another which determiner belongs to which noun. These are observed tendencies in trained models, not assignments anyone made.

The rest of the block

Attention is half of a layer. The other half is a feed-forward network applied to each position on its own: a fixed computation that takes the mixed representation and transforms it, with the same parameters at every position. The majority of a model’s parameters sit here. Research on trained models suggests these layers store much of what the model knows about the world, in a distributed form that the essay on models will take up.

Each layer adds its result to a running representation rather than replacing it, a design called the residual pathway. Picture a conveyor carrying one vector per token through the whole stack. Every layer reads from the conveyor, computes something, and adds a correction back onto it. A model of the size I run at home has about eighty such layers. By the top, the vector at “bank” in the flood sentence and the vector at “bank” in the audit sentence have diverged, and the final step converts the vector at the last position into probabilities for what comes next.

That final step is the next-token prediction. Everything before it is the machinery that makes the prediction good.

What attention is not

The word was borrowed from psychology, and the borrowing causes trouble. Attention in a transformer is an arithmetic operation: a set of weights computed from the input that decide how representations are combined. It is not awareness, not focus in the human sense, and not evidence that the model is paying attention to anything the way a person does. The name stuck because it was useful, and it should be read as a technical term.

A second caution. Because attention weights can be read out, it is tempting to treat them as a window into the model’s reasoning: this token looked at that token, therefore the model was thinking about that connection. Researchers have shown that the weights are at best a partial account. The feed-forward layers, the residual stream and the depth of the stack all contribute, and a readable attention pattern can accompany very different internal computations. The transparency is real but limited, and I will not pretend it is more than it is.

One shape, two halves

The transformer was introduced in 2017 for translation, with two halves: an encoder that read the source sentence and a decoder that wrote the target. Most language models since have kept only the decoder, which reads the text so far and produces what follows. “Decoder-only” is the phrase you will see in model descriptions, and it means this. The encoder did not disappear from the field. It became the natural tool for turning an input, a sentence or an image, into a representation that can be compared with other representations, and that is where the next essay goes.

Where the mystery sits

Here is what is established. A transformer learns, through training, to build representations in which a token’s vector reflects its context, and those representations are rich enough to support prediction across the whole range of human writing in the corpus. Here is what is open. Whether and in what sense those representations amount to understanding is not settled, and the people who build these systems disagree about it. I find it accurate to describe this as a mystery, and I mean the word plainly. Something in the stack carries the difference between a river bank and a failed bank, and we can measure that it does, and we do not yet have a full account of what it is.

The painting

The third panel shows one relation among a few tokens, nothing more. A figure holds up the flood sentence and the audit sentence on two cards. Threads of light run from each “bank” back to the word that resolves it, and nowhere else. The diagram on the cards will be drawn as an exact vector graphic and composited in, because a painted diagram that only looks right is worse than none. At the edge of the scene stands Claude Shannon, who in 1951 measured how well people could guess the next letter of English text and in doing so wrote down the question this whole architecture answers.

The next essay turns to the representations themselves: what a vector of a few thousand numbers can hold, how an image and a phrase can be placed in the same space, and what “similar” means once meaning is a position.