Learn AI /Large Language Models /Top-p
Top-p
- Deeper
- Large Language Models
- Technical readers
Also called: nucleus sampling, top-k.
Back to the milestone you came from
In five seconds
What it is
A decoding setting that limits each next-token choice to the smallest group whose combined probability reaches a chosen threshold, such as 0.9.
In plain words
Top-p tells the system to choose the next piece of text only from the most likely options whose probabilities add up to a chosen amount, such as 90 percent.
An everyday example
Imagine a music app ranks possible next songs by how well each fits the current playlist. With a 90% threshold, it keeps the smallest group whose probabilities add up to at least 90%, adjusts the probabilities within that group and makes a weighted random choice. If one or two songs are strong matches, the group is small. If many songs are similarly plausible, the group is larger.
People in a meeting can explain their reasons and may look back at the full discussion. A language model does not do that in the same way here. It is applying probabilities token by token. Also, in real note-taking, whole phrases are chosen at once, while top-p usually operates on one token at a time, so the analogy hides the very small step size.
Remember thisTop-p retains the smallest set of likely next tokens whose combined probability reaches the threshold, then samples from that set. Its size can change at every step.
Understand it
Understand it
Why it matters
Top-p affects how risky, varied and review-heavy generated text may be. If you set it too low, outputs can become narrow, repetitive or brittle. If you set it too high, outputs may wander into weaker wording or unsupported details. For anyone choosing model settings, evaluating assistants or comparing vendors, top-p is one of the controls that changes behaviour without changing the underlying model weights. It is especially important in tasks where consistency matters, such as drafting summaries, and in tasks where some variation is useful, such as brainstorming alternative phrasing. Knowing top-p also helps you separate two questions that are often mixed together: how the model scores options, and how a system chooses among them.
How top-p keeps a changing set of next-token options
Show why top-p does not keep a fixed number of tokens, but a fixed probability mass that can produce a small or large candidate set.
01
Rank candidates
List possible next tokens from most likely to least likely, each with a probability from the model.
Show a vertical list of 6 to 8 tokens with horizontal probability bars, sorted descending. Use clear labels such as token A 0.50, token B 0.25, token C 0.10, and so on.
02
Accumulate probability
Add probabilities from the top down until the running total reaches the chosen threshold p, such as 0.90.
Place a running-total column beside the bars. Animate or indicate cumulative sums: 0.50, 0.75, 0.85, 0.91. Draw a threshold line at p = 0.90.
03
Form the nucleus
Keep only the smallest set that reaches the threshold. Remove lower-probability tokens from consideration for this step.
Highlight the included rows up to the first row that crosses the threshold. Grey out all remaining rows and label them excluded this step.
04
Sample within the nucleus
Choose the next token only from the kept set, with more likely tokens still favoured inside that set.
Show arrows only from highlighted tokens to an output token slot. Use thicker arrows for higher probabilities. Keep excluded rows disconnected.
05
Repeat with a new distribution
At the next position in the text, probabilities change, so the nucleus may contain fewer or more tokens than before.
Show a second small panel with a flatter distribution where more rows are needed to reach p = 0.90. Add a note: top-p set size changes step by step.
A left-to-right step diagram. Start with ranked next-token candidates and their probabilities. Then accumulate probability from highest to lowest until the chosen threshold is reached. Highlight the kept nucleus set and grey out the rest. End by showing sampling from only the kept set. Include one case where the set is small and one where it is larger.
Hover or tap each step to reveal the running total and see which tokens enter the nucleus at different probability shapes; a toggle switches between a sharp distribution and a flat one.
Describe this without the diagram
This diagram explains top-p in five steps. First, it shows possible next tokens ranked by probability. Second, it adds those probabilities from the top downward until the total reaches a threshold such as 0.90. Third, it keeps only the smallest group of tokens needed to reach that total and excludes the rest. Fourth, it shows that the model samples only from the kept group, still favouring higher-probability tokens. Fifth, it compares two cases: when the distribution is sharp, only a few tokens are kept; when it is flatter, more tokens are needed. The main point is that top-p keeps a changing number of tokens based on total probability, unlike top-k, which keeps a fixed count.
With reduced motion: A static five-panel diagram with highlighted cumulative totals and shaded excluded tokens. Two side examples show that a peaked distribution leads to a small nucleus, while a flatter distribution leads to a larger nucleus.
When a language model generates text, it first assigns a probability to many possible next tokens. A token is a small unit of text, such as a word, part of a word, or punctuation. Top-p controls which of those candidate tokens are allowed to be sampled.
The model sorts candidate tokens from most likely to least likely, then keeps adding them until their probabilities add up to the chosen threshold p. If p is 0.9, it keeps the smallest set whose total probability is at least 0.9. The next token is then sampled only from that set, not from the full vocabulary.
This means the size of the allowed set changes from step to step. If probability is concentrated in a few tokens, only a few may be included. If probability is spread across many tokens, more may be included. That is the main difference from top-k, which always keeps a fixed number k of tokens.
What comes out is not a new score but a filtered token choice process. Lower top-p narrows the candidate set and can reduce variation, but it can also make output repetitive or narrow. Higher top-p includes more lower-probability options and can increase variety. Neither setting by itself guarantees safer or more accurate output. Top-p is often adjusted together with temperature, but they are different controls.
How “Letting a playlist choose the next song” maps onto the real thing
| In the story | In the AI system |
|---|---|
| ranked phrases on the board | candidate next tokens with probabilities |
| smallest group covering 90 percent of likely choices | nucleus set selected by top-p |
| sometimes two phrases, sometimes four | variable number of allowed tokens at each step |
| choosing only within that group | sampling after lower-probability tokens are removed |
A worked example
The model is generating the next token after: "The revised talking points will be circulated by".
Starting from: Candidate tokens and probabilities are: " noon" 0.42, " email" 0.24, " close" 0.11, " 18" 0.08, " Monday" 0.06 and "." 0.03. All remaining tokens together account for 0.06. Use top-p = 0.80.
- Sort the individual tokens from highest to lowest probability.
- Add probabilities in order. The cumulative totals are 0.42, 0.66, 0.77 and then 0.85.
- Stop at the smallest prefix that reaches at least 0.80. Retain " noon", " email", " close" and " 18".
- Renormalise those four probabilities over their total of 0.85. Their sampling weights become about 49.4%, 28.2%, 12.9% and 9.4%.
- Sample one token using those adjusted weights. At the following step, the model produces a new distribution and constructs a new nucleus.
Result: " noon" is the likeliest selection, but any of the four retained tokens can be sampled. The number of candidates at the next step may differ because top-p applies to the new probability distribution.
In a United Nations workplace
A desk officer in a small field presence with intermittent connectivity is using an approved, locally hosted model to tighten talking points before informals resume in 20 minutes.
The desk officer compares top-p settings to keep the drafting reasonably consistent without forcing identical wording each time. A lower value narrows the candidate set; a higher value may admit more varied phrasing. The setting affects variation, not whether the output is accurate or cleared.
Before you try thisTalking points may contain uncleared positions or attribution-sensitive details. Do not enter them into any tool not approved for that material. Even in an approved system, top-p is not a factuality or confidentiality control. The desk officer must check the draft against the cleared file.
Where you may hear it
- A vendor demo during a procurement unit's technical evaluation
- An ICT governance board reviewing settings for an approved assistant
- A model evaluation session in an information management office
- A prompting workshop before a pilot is cleared
Avoid confusion
Avoid confusion
Often confused with
Temperature
A setting that controls how predictable or varied a language model’s next chosen words are during generation.
Sampling (choosing the next token)
A stochastic procedure that draws one next token from the model’s probability distribution, appends it to the output, and repeats until a stop condition.
Deterministic output
The same full input and generation conditions produce the same output every time, rather than varying across repeated runs.
Next-token prediction
A language model produces text by repeatedly estimating probabilities for small pieces of text that could come next in the current sequence.
Common misconceptions
Common misconception: Top-p and top-k are the same setting with different names.
Correction: They are related but not the same. Top-k keeps exactly k highest-probability tokens every time. Top-p keeps as many tokens as needed to reach a probability threshold, so the number kept changes from step to step. Some tools list both because they can be combined.
Common misconception: A top-p value of 0.9 means the chosen token has a 90 percent chance of being correct.
Correction: It does not measure correctness. It only defines the pool of allowed tokens by cumulative probability. The selected token can still be wrong, awkward or misleading, because the model may assign high probability to poor continuations in some contexts.
Common misconception: Lowering top-p always improves factual accuracy.
Correction: Lower top-p usually reduces randomness, but it does not make the model reliable by itself. If the highest-probability continuation is inaccurate, a lower top-p can make that inaccurate continuation more likely to appear again. Accuracy depends on the model, the prompt, the context and any retrieval or verification steps.
Common misconception: Top-p changes what the model has learned.
Correction: Top-p does not retrain the model or alter its weights. It only changes the decoding stage, which is the procedure used to pick the next token from the probabilities the model already produced.
Quick check
Which statement best distinguishes top-p from top-k during text generation?
Top-p clips by cumulative probability, so the kept set can grow or shrink depending on the distribution. Top-k clips by rank, so it always keeps exactly k tokens. Neither setting retrains the model, and neither guarantees factual accuracy.
Go deeper
Go deeper
The simple version
Top-p tells the system to choose the next piece of text only from the most likely options whose probabilities add up to a chosen amount, such as 90 percent.
The technical version
At each decoding step, the model outputs a probability distribution over the vocabulary for the next token. Under nucleus sampling, tokens are sorted by probability in descending order, and the smallest prefix whose cumulative probability is at least p is retained. The distribution is then renormalised over that retained set, and one token is sampled from it. Because the retained prefix depends on the shape of the distribution, the effective candidate count varies by step. In a peaked distribution, the nucleus may contain only a few tokens; in a flatter distribution, it may contain many. This distinguishes top-p from top-k, which clips by rank rather than cumulative mass. In practice, top-p also interacts with temperature, because temperature reshapes the distribution before or during sampling, which changes how many tokens are needed to reach the threshold.
Reviewed external resources
EncyclopaediaTop-p sampling (opens in a new tab)Wikipedia
This Wikipedia page gives a broad overview of top-p sampling and suits readers who want a quick orientation before deeper study.
Intermediatelink checked 2026-07-31
CourseLLM Course (opens in a new tab)Hugging Face
This Hugging Face course material places top-p among other generation settings and suits readers who want practical context for model use.
Deeperlink checked 2026-07-31
VideoLarge language models, explained visually (opens in a new tab)3Blue1Brown
This 3Blue1Brown video gives a visual explanation of how language models generate text and suits readers who learn best from intuition and diagrams.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Top-p (opens in a new tab)YouTube search
This YouTube search results page helps readers find multiple video explanations of top-p and suits people who want to compare teaching styles, but it is not a single vetted source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Top-p (opens in a new tab)Reddit search
This Reddit search results page helps readers see community questions and examples about top-p and suits readers looking for informal discussion, but it is a search page rather than one authoritative explanation.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Top-p retains the smallest set of likely next tokens whose combined probability reaches the threshold, then samples from that set. Its size can change at every step.