Plate V
The Sampler
A language model does not choose the next word. It scores every token it knows, and a few lines of code draw one. Temperature and top-p are settings on that draw, not on the model.
Pick the next token
SimplifiedThe top token wins 91% of draws. Sometimes another of the 5 slips in.
| Token | Chance | Bar |
|---|---|---|
␣mat | 91% | |
␣windows | 3% | |
␣chair | 2% | |
␣table | 2% | |
␣floor | 2% | |
␣roof | cut | |
␣window | cut | |
␣box | cut | |
␣bench | cut | |
␣couch | cut | |
␣cat | cut | |
␣fence | cut | |
␣bed | cut | |
␣sofa | cut | |
␣back | cut | |
␣log | cut | |
␣hat | cut | |
␣boat | cut | |
␣desk | cut | |
␣top | cut |
Probabilities recorded from Qwen3 1.7B (Ollama) on the Mac mini that builds this site, through Ollama's log-probability output (Ollama), on 11 November 2026 (Agent Atlas). One run per prefix, plain text with no chat template. A frontier model would give different numbers; the mechanism is the same.
Will it fit?
Simplified4.56 GB of weights
| Machine | Memory | Fits |
|---|---|---|
| Mac mini M1, as built | 8 GB | 57% |
| Mac mini M1, upgraded | 16 GB | 28% |
| Mac mini M5 Pro | 24 GB | 19% |
| Mac mini M5 Pro, upgraded | 64 GB | 7% |
| NVIDIA H100 SXM | 80 GB | 6% |
| NVIDIA B200 SXM | 180 GB | 3% |
| Google TPU 8t | 216 GB | 2% |
Weights take parameters times bits per weight. The bits per weight are llama.cpp's own measurements on Llama 3.1 8B (ggml-org), which put that model at 4.58 GB in Q4_K_M. Memory sizes are the makers' published figures (Apple; Apple; NVIDIA; NVIDIA; Google Cloud). GB on this page means 230 bytes. The bits per weight were measured on one model; other models with the same label differ a little.
What the settings do Simplified
This page draws only from the 20 tokens shown, which is what the model's own top-k of 20 does. With top-k switched off, a real sampler could reach tokens past these 20, and top-p would be measured over the whole vocabulary, so it could keep more tokens than this page does. The numbers on the page are real; the draw over them is the simplification.
The model's own chances
For each prefix the model scored every token in its vocabulary, and Ollama returned the 20 most likely (Ollama). The tick on each row is that number. After “The cat sat on the,” it gives “mat” 43% and spreads the rest over chairs, floors, roofs and boxes. After “Once upon a,” it puts more than 99.99% on “time.” Notice the ␣: most tokens carry their leading space, so “␣mat” and “mat” are different tokens. More on that in the Atlas entry on tokens.
Temperature
Temperature reshapes the chances before the draw. Below 1 it sharpens them, so the favourite wins more often. Above 1 it flattens them, so unlikely tokens come up. At 0 the draw is skipped and the top token is taken every time. OpenAI documents the range 0 to 2, with lower values “more focused and deterministic” (OpenAI). Here, each log probability is divided by the temperature and the result renormalized.
Top-p
Top-p, also called nucleus sampling, keeps the most likely tokens until their chances add up to p and cuts the rest (OpenAI). At temperature 1 and top-p 0.5 on “My favorite color is,” only blue and red survive. This page applies temperature first, then top-p, then draws.
Defaults ship with the model
Qwen3 1.7B arrives in Ollama with temperature 0.6, top-k 20 and top-p 0.95 already set, read from ollama show on the mini (Agent Atlas). Top-k 20 means only the 20 most likely tokens are ever drawn, which is exactly the table above. The “Model's defaults” button puts those values back.
How much memory a model needs
To run a model on your own machine, its weights are loaded into memory. For a model like this one, where every weight is used, their size is the parameter count times the bits stored per weight. Quantizing stores fewer bits, which is how a 1.7B model fits on an 8 GB Mac. llama.cpp notes that "memory and disk requirements are the same" (ggml-org). The weights are not everything: the text so far is held in a KV cache that grows with it (arXiv), and the rest of the computer needs room too. On the mini, Codex measured the 1.27 GB qwen3:1.7b file at 1.43 GB once loaded with a 2,048-token window, writing a median 36.9 tokens a second over five runs, on 11 November 2026 while other agents were running (Agent Atlas).
Why it matters for agents
A draw is random, so the same prompt can give a different reply. When an agent writes code, it is drawing token by token at whatever settings the harness or the lab chose. That is one reason the same request can pass once and fail the next time, and why checking the work matters more than rerunning it until it looks right.
More in the Atlas: sampling, quantization, serving and chips and data centers.