Atlas · models
Transformer
The architecture most of today's language models build on: stacked layers that mix tokens together through attention.
In plain words
A transformer turns each token into a list of numbers, then passes those lists through many layers. In each layer, attention lets a token draw on other tokens in the text. In a model that writes text, a token can only use the tokens before it, never the ones after.
How it works
The 2017 paper that named it built the model from attention alone, with no recurrence and no convolution. Attention weighs and combines the tokens it is allowed to see. Full attention compares every pair of tokens, so the work grows with the square of the length. Anthropic also reports that recall can fall as the context grows.
Going deeper
To avoid redoing work on every new token, a running model keeps the keys and values it already computed. This is the KV cache, and the vLLM authors note that it is large and grows and shrinks as each request runs. See serving.
Sources
- Attention Is All You Need arXiv
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
- Effective context engineering for AI agents Anthropic
Context refers to the set of tokens included when sampling from a large-language model (LLM).
the model’s ability to accurately recall information from that context decreases.
Context, therefore, must be treated as a finite resource with diminishing marginal returns.
This results in n² pairwise relationships for n tokens.
Compaction typically serves as the first lever in context engineering to drive better long-term coherence.
These notes get pulled back into the context window at later times.
but returns only a condensed, distilled summary of its work
- Efficient Memory Management for Large Language Model Serving with PagedAttention arXiv
the key-value cache (KV cache) memory for each request is huge and grows and shrinks dynamically.
we propose PagedAttention, an attention algorithm inspired by the classical virtual memory and paging techniques in operating systems.
Quotes were checked against the live pages on 2026-11-10.