Agent Atlas

Atlas · models

Sampling

A model scores every possible next token. A sampler then picks one, using settings such as temperature and top-p.

Established practiceDepth 2 of 4Reviewed 2026-11-12

See it working: Plate V

In plain words

A model writes one token at a time. For each, it gives a chance to every token it knows, and a sampler picks one. When the pick is random, the same prompt can give a different reply.

How it works

Temperature reshapes the chances before the pick: lower makes the favourite win more often, higher flattens the odds. OpenAI documents it from 0 to 2, with lower values more focused and deterministic. Top-p, or nucleus sampling, keeps the smallest set of most likely tokens whose chances add up to at least p. Ollama can return the log probability of each output token and the most likely alternatives. Plate V uses a recorded top 20 from one such run.

Going deeper

The installed Qwen3 tag came with its own defaults: on the mini on 11 November 2026, ollama show reported temperature 0.6, top-k 20 and top-p 0.95. Plate V starts there.

Sources

  1. Completions OpenAI
    • What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
    • An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.
    • Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.
  2. Generate a response Ollama
    • Whether to return log probabilities of the output tokens
    • Number of most likely tokens to return at each token position when logprobs are enabled
  3. Qwen3 next-token probabilities Agent Atlas
    • The page stores the 20 returned log probabilities for each prefix, rounded to five decimals.
    • On this mini on 11 November 2026, ollama show qwen3:1.7b reported temperature 0.6, top-k 20 and top-p 0.95.

Quotes were checked against the live pages on 2026-11-10.