Learn AI /Large Language Models /Temperature
Temperature
- Intermediate
- Large Language Models
- Technical readers
Back to the milestone you came from
In five seconds
What it is
A setting that controls how predictable or varied a language model’s next chosen words are during generation.
In plain words
Think of temperature as a creativity dial for word choice. Lower settings stick closer to the most expected wording. Higher settings allow more unusual wording.
An everyday example
Two colleagues in Beirut need to cross the city for a meeting. Their map app shows several routes, with one route clearly fastest, two slightly slower and a few unusual detours. If they set the app to “play it safe”, it almost always picks the fastest route. If they set it to “explore more”, it sometimes chooses one of the less usual routes instead.
A map app may use real traffic rules and destination goals, while a language model is not planning a journey or checking road conditions. It is repeatedly sampling tokens from a probability distribution. Also, one route choice happens once, but model generation makes a new choice at every token, so small differences can grow across a whole response.
Remember thisTemperature reshapes the probabilities used to sample each next token. It changes variation, not factual accuracy, judgement or compliance.
Understand it
Understand it
Why it matters
Temperature is a practical control. It helps you choose between steadier output and more variety. That matters when you compare drafts, test prompts, build services or evaluate risk. For tasks like structured extraction or standard wording, a lower setting often makes behaviour easier to reproduce. For brainstorming, a higher setting may produce more diverse options. But neither setting guarantees factual accuracy, safety or usefulness, so people should not treat temperature as a truth control.
How temperature changes token choice
After using this explainer, a reader should be able to say that temperature changes how strongly a model favours high-probability next tokens, affecting consistency versus variety.
The sentence so far
“The mission report was delivered to the …”
A slider changes temperature from low to high. The screen shows the same prompt and the same candidate next tokens with bars for their probabilities. As the slider moves, the bars flatten or sharpen, and example continuations update to show more predictable or more varied outputs. Read it left to right: prompt, candidate tokens, adjusted probabilities, sampled continuation.
The reader drags a single temperature slider and can press a replay button to sample several continuations at the current setting for comparison.
Describe this without the diagram
Screen-reader version: You hear one prompt, then a list of candidate next tokens with their relative probabilities. The control lets you choose low, medium or high temperature. At low temperature, the top token becomes much more likely than the others, so repeated samples are similar. At high temperature, the probabilities become more even, so repeated samples vary more. The prompt does not change. The lesson is that temperature changes randomness in selection, not the model’s underlying knowledge.
With reduced motion: Reduced-motion version: no animated bar movement. The page shows three static states side by side or switchable by tabs: low, medium and high temperature. Each state lists the same tokens with fixed bar charts and one or two example continuations.
When a language model produces text, it does not pull one fixed answer from storage. At each step, it considers many possible next tokens, where a token is a small unit of text such as a word, part of a word or punctuation. Temperature changes how sharply the model prefers the most likely options over the less likely ones.
Low temperature makes the output more predictable. The model stays close to its highest-probability next tokens, so answers are usually more consistent and repetitive. High temperature makes the output more varied. Lower-probability tokens get more chance, so the text may become more creative, surprising or unusual.
What goes in is the model’s scored list of possible next tokens plus the temperature setting. What happens is that the scores are adjusted before one token is sampled, meaning chosen from the adjusted probabilities. What comes out is one next token, then the process repeats until the response is complete.
Temperature does not add knowledge, truth or reasoning ability. It only changes the randomness of token selection. It is often confused with top-p, which limits how much of the probability list is considered, while temperature changes the shape of that list.
How “Choosing a route across a city” maps onto the real thing
| In the story | In the AI system |
|---|---|
| list of possible routes | list of possible next tokens |
| fastest route ranked highest | most likely token ranked highest |
| play it safe versus explore more | lower versus higher temperature |
| route chosen each time | token sampled at each generation step |
A worked example
A reporting officer tests how temperature affects the first word proposed for a heading in a progress report.
Starting from: Prompt: "Draft a short heading about delivery against agreed milestones." Candidate first-token logits: Implementation 2.0, Delivery 1.5, Progress 1.0, Momentum 0.0.
- At temperature 1.0, softmax converts the logits to probabilities of about 47%, 29%, 17% and 6%, respectively.
- At temperature 0.5, the logits are divided by 0.5. The distribution sharpens to about 66%, 24%, 9% and 1%, making "Implementation" more likely.
- At temperature 2.0, the logits are divided by 2. The distribution flattens to about 36%, 28%, 22% and 13%, giving lower-ranked words a better chance.
- After sampling the first token, the model calculates new logits and repeats the same temperature adjustment for each subsequent token.
Result: A lower temperature will more often begin with the highest-scoring wording, while a higher temperature will produce a wider range of headings. Neither setting checks whether the heading accurately reflects the progress report.
In a United Nations workplace
A programme officer in a liaison office is testing an approved tool to suggest labels for open-ended answers in a needs assessment before the coding scheme is finalised.
A lower temperature can make suggested labels more consistent across repeated runs. A higher setting may surface less obvious candidate labels during exploration, but it can also produce unnecessary variation. Temperature does not make the coding scheme valid or ensure that similar answers receive the same code.
Before you try thisNeeds-assessment responses may contain personal or protection-sensitive information. They must not be entered into a tool that is not approved for that material. Identifiers should be removed or minimised as required, and a person must review the labels for consistency, bias and loss of context.
Where you may hear it
- During a vendor demo for an approved drafting tool
- In a procurement unit's technical evaluation meeting
- At an ICT governance board review of model settings
- While testing prompts before a tool is cleared for wider use
Avoid confusion
Avoid confusion
Often confused with
Top-p
A decoding setting that limits each next-token choice to the smallest group whose combined probability reaches a chosen threshold, such as 0.9.
Deterministic output
The same full input and generation conditions produce the same output every time, rather than varying across repeated runs.
"Understanding" (what models do and do not do)
In language models, apparent comprehension usually means producing text that fits patterns in data and context, not reliably building a grounded grasp of the world like a person does.
Common misconceptions
Common misconception: Higher temperature makes the model more intelligent or more knowledgeable.
Correction: It does not improve the model’s stored patterns or add new evidence. It only changes how boldly the system samples from less likely tokens, which can increase novelty but also increase mistakes or odd wording.
Common misconception: Temperature controls factual accuracy.
Correction: Not directly. Lower temperature can make answers more consistent because the model repeats its most likely patterns, but a confident repeated mistake is still a mistake. Accuracy depends on the model, prompt, data sources and checking process.
Common misconception: Temperature and top-p are the same thing.
Correction: They are related but different. Temperature reshapes the probability distribution across candidate tokens. Top-p keeps only the smallest set of tokens whose probabilities add up to a chosen threshold, then samples within that set.
Quick check
What does lowering temperature usually do in a language model?
Lower temperature sharpens preference for the most likely next tokens. That often makes outputs steadier and easier to reproduce. It does not add knowledge, increase context length or guarantee truth.
Go deeper
Go deeper
The simple version
Think of temperature as a creativity dial for word choice. Lower settings stick closer to the most expected wording. Higher settings allow more unusual wording.
The technical version
In autoregressive text generation, the model produces logits or scores for candidate next tokens. Temperature rescales these scores before the softmax step that turns them into probabilities, commonly by dividing logits by the temperature value. When temperature is below 1, probability mass concentrates more strongly on the highest-scoring tokens. When temperature is above 1, the distribution becomes flatter, so lower-scoring tokens receive relatively more probability. Sampling then draws the next token from this adjusted distribution. At or near zero, systems often switch to greedy or otherwise near-deterministic selection, though exact implementation details vary by provider. Temperature is therefore a decoding parameter, not a training parameter. It affects generation behaviour at inference time, not what the model weights contain.
Reviewed external resources
EncyclopaediaTemperature (opens in a new tab)Wikipedia
This link gives a general reference-style overview of temperature in machine learning and suits readers who want a broad starting point.
Intermediatelink checked 2026-07-31
CourseLLM Course (opens in a new tab)Hugging Face
This course link is useful for learners who want a practical explanation of generation settings in the wider context of how language models work.
Deeperlink checked 2026-07-31
VideoLarge language models, explained visually (opens in a new tab)3Blue1Brown
This video gives an intuitive visual explanation of language model generation and suits readers who learn best from animated concepts.
Intermediatelink checked 2026-07-31
VideoVideo explanations: Temperature (opens in a new tab)YouTube search
This search results page helps readers find multiple video explanations of temperature and suits people who want to compare teaching styles rather than rely on one source.
search results, not one pagelink checked 2026-07-31
DiscussionCommunity discussion: Temperature (opens in a new tab)Reddit search
This search results page helps readers see common questions, examples and debates about temperature and suits readers who want informal community discussion with caution about reliability.
search results, not one pagelink checked 2026-07-31
Where this came from
Drafted with a language model through an offline pipeline, checked against this site's schema, and every external link above fetched to see whether it answers. Treat it as a starting point and check anything that matters against the primary sources. How this site is built and what that means.
The one thing to remember
Temperature reshapes the probabilities used to sample each next token. It changes variation, not factual accuracy, judgement or compliance.