Lesson 12 of 28 · Language models and chat assistants

What is temperature in AI models?

Temperature is a setting that controls how much randomness an AI model uses when it picks each next word. At a low temperature the model almost always chooses the most likely word, so answers are focused and repeatable. At a high temperature it more often picks less likely words, so answers become more varied and creative, but also more prone to mistakes.

How temperature works, without the maths

A language model writes one token at a time (a token is a word or part of a word). At each step it does not know the answer; it calculates a probability for every possible next token. After 'The cat sat on the', the word 'mat' might be very likely, 'sofa' less likely and 'moon' very unlikely. The model then picks one, a step called sampling.

Temperature changes how those probabilities are treated before the pick. A low temperature sharpens them, so the favourite wins nearly every time. A high temperature flattens them, giving unlikely words a better chance. Think of ordering at a restaurant: at a low temperature you always order your usual dish; at a high temperature you sometimes try something from the bottom of the menu.

Which temperature for which task?

There is no single best value; it depends on whether you want consistency or variety. The allowed range and the default differ between providers, so check the documentation of the tool you use.

Temperature is often paired with a related setting called top-p, which limits the pick to the smallest group of tokens whose combined probability reaches a threshold (Holtzman et al., 2019). Some providers recommend changing one of the two, not both. As a rule of thumb for temperature:

Low: extracting data, classifying, answering factual questions, writing code, following a strict format.
Medium: everyday writing, emails, summaries and explanations.
High: brainstorming names, slogans, story ideas and other tasks where surprise is useful.

What temperature is not

Temperature is not an intelligence or quality dial. A low temperature makes answers more consistent, not more correct: a model can repeat the same wrong fact every time. It also does not add knowledge, change the model's parameters or have anything to do with how hot the computer runs; the name is borrowed from physics, where a higher temperature means more random movement.

Even at a temperature of zero, output is usually close to deterministic but not guaranteed to be identical on every run, because of small numerical differences in how the hardware carries out the calculations. Some reasoning models fix the temperature or ignore it.

Where you meet temperature in real tools

Most consumer chat apps set the temperature for you and do not show it, which is one reason the same model can feel different from one product to another. Developer playgrounds and APIs usually expose it as a number, and some writing tools present it as a 'creativity' or 'randomness' slider.

If you build with an API or an automation tool, set the temperature on purpose for each step: low for steps that feed data into other systems, higher only where variety helps.

Example

Illustration: the same prompt at low and high temperature

Worked example

Next-word probabilities after 'The cat sat on the' (made-up numbers): mat sofa roof moon T = 1.0 60% 25% 10% 5% T = 0.5 83% 14% 2% under 1% T = 2.0 43% 28% 17% 12% Prompt: 'Suggest a name for a coffee shop.' (3 runs each) Low temperature: The Daily Grind / The Daily Grind / The Daily Grind High temperature: Bean There / Moonlit Roast / The Velvet Cup

Illustrative only: invented numbers and outputs to show the effect, not results from a real model. T = temperature. To see it yourself, use a tool that exposes a temperature setting and run the same prompt several times at each value.

Key points

Temperature controls randomness when a model picks each next token.
Low means focused and repeatable; high means varied and creative.
A low temperature makes answers consistent, not more accurate.
Use low values for facts, data and code; higher ones for brainstorming.
Many apps set it for you; APIs and playgrounds let you change it.

Try it on Mindber

Glossary: temperature

Prompt engineering guide: control output beyond temperature

Compare AI models (Arena)

Find AI writing assistants

Frequently asked questions

What is a good temperature setting?

It depends on the task and on the provider's scale. For factual answers, data extraction and code, use a low value so results are repeatable. For brainstorming or creative writing, try a higher value and compare several outputs. If you are unsure, keep the tool's default, which is usually a middle setting, and change it only when results are too dull or too erratic.

Does a temperature of 0 stop hallucinations?

No. A temperature of zero makes the model pick its most likely answer every time, but the most likely answer can still be wrong. It reduces random variation, not factual errors. To reduce hallucinations, give the model reliable sources, ask it to cite them, and check important facts yourself.

Why do I get a different answer when I ask the same question twice?

Because most chat apps use a temperature above zero, so the model samples from several likely words instead of always taking the top one. Small differences early in a reply then grow into different wording or structure. This variety is intended; if you need consistent results, use a tool that lets you lower the temperature, or give a stricter prompt and format.