Agent Atlas

Plate V

The Sampler

A language model does not choose the next word. It scores every token it knows, and a few lines of code draw one. Temperature and top-p are settings on that draw, not on the model.

Pick the next token

Simplified
Text so far
The cat sat on the▁

The top token wins 91% of draws. Sometimes another of the 5 slips in.

The model's 20 most likely next tokens. They hold 81% of its probability. The bar is the chance after your settings; the tick is the model's own.
TokenChanceBar
␣mat91%
␣windows3%
␣chair2%
␣table2%
␣floor2%
␣roofcut
␣windowcut
␣boxcut
␣benchcut
␣couchcut
␣catcut
␣fencecut
␣bedcut
␣sofacut
␣backcut
␣logcut
␣hatcut
␣boatcut
␣deskcut
␣topcut

Probabilities recorded from Qwen3 1.7B (Ollama) on the Mac mini that builds this site, through Ollama's log-probability output (Ollama), on 11 November 2026 (Agent Atlas). One run per prefix, plain text with no chat template. A frontier model would give different numbers; the mechanism is the same.

Will it fit?

Simplified

4.56 GB of weights

Weights against each machine's memory. The bar is the share the weights take. Fitting is not enough: the text so far, the runtime and the system need room too.
MachineMemoryFits
Mac mini M1, as built8 GB57%
Mac mini M1, upgraded16 GB28%
Mac mini M5 Pro24 GB19%
Mac mini M5 Pro, upgraded64 GB7%
NVIDIA H100 SXM80 GB6%
NVIDIA B200 SXM180 GB3%
Google TPU 8t216 GB2%

Weights take parameters times bits per weight. The bits per weight are llama.cpp's own measurements on Llama 3.1 8B (ggml-org), which put that model at 4.58 GB in Q4_K_M. Memory sizes are the makers' published figures (Apple; Apple; NVIDIA; NVIDIA; Google Cloud). GB on this page means 230 bytes. The bits per weight were measured on one model; other models with the same label differ a little.

What the settings do Simplified

This page draws only from the 20 tokens shown, which is what the model's own top-k of 20 does. With top-k switched off, a real sampler could reach tokens past these 20, and top-p would be measured over the whole vocabulary, so it could keep more tokens than this page does. The numbers on the page are real; the draw over them is the simplification.

The model's own chances

For each prefix the model scored every token in its vocabulary, and Ollama returned the 20 most likely (Ollama). The tick on each row is that number. After “The cat sat on the,” it gives “mat” 43% and spreads the rest over chairs, floors, roofs and boxes. After “Once upon a,” it puts more than 99.99% on “time.” Notice the ␣: most tokens carry their leading space, so “␣mat” and “mat” are different tokens. More on that in the Atlas entry on tokens.

Temperature

Temperature reshapes the chances before the draw. Below 1 it sharpens them, so the favourite wins more often. Above 1 it flattens them, so unlikely tokens come up. At 0 the draw is skipped and the top token is taken every time. OpenAI documents the range 0 to 2, with lower values “more focused and deterministic” (OpenAI). Here, each log probability is divided by the temperature and the result renormalized.

Top-p

Top-p, also called nucleus sampling, keeps the most likely tokens until their chances add up to p and cuts the rest (OpenAI). At temperature 1 and top-p 0.5 on “My favorite color is,” only blue and red survive. This page applies temperature first, then top-p, then draws.

Defaults ship with the model

Qwen3 1.7B arrives in Ollama with temperature 0.6, top-k 20 and top-p 0.95 already set, read from ollama show on the mini (Agent Atlas). Top-k 20 means only the 20 most likely tokens are ever drawn, which is exactly the table above. The “Model's defaults” button puts those values back.

How much memory a model needs

To run a model on your own machine, its weights are loaded into memory. For a model like this one, where every weight is used, their size is the parameter count times the bits stored per weight. Quantizing stores fewer bits, which is how a 1.7B model fits on an 8 GB Mac. llama.cpp notes that "memory and disk requirements are the same" (ggml-org). The weights are not everything: the text so far is held in a KV cache that grows with it (arXiv), and the rest of the computer needs room too. On the mini, Codex measured the 1.27 GB qwen3:1.7b file at 1.43 GB once loaded with a 2,048-token window, writing a median 36.9 tokens a second over five runs, on 11 November 2026 while other agents were running (Agent Atlas).

Why it matters for agents

A draw is random, so the same prompt can give a different reply. When an agent writes code, it is drawing token by token at whatever settings the harness or the lab chose. That is one reason the same request can pass once and fail the next time, and why checking the work matters more than rerunning it until it looks right.

More in the Atlas: sampling, quantization, serving and chips and data centers.