AI temperature: what the settings really change

A restaurant owner asks for ten names for his new fall menu and gets ten variations on "Autumn Flavors." The next day, he asks a tool to extract the amounts from his supplier invoices, and the format changes from one line to the next. In the first case, he'd like more boldness. In the second, he'd like the tool to stop improvising.

These two opposite needs depend on the same parameter: temperature. To understand what it's for, you need just one reminder: a language model calculates probabilities for the continuation of the text, then picks one. That's the principle described in our article on how a language model works.

Where to find these settings

Let's start with a clarification that saves you a pointless search: in consumer apps, these sliders are generally not offered. You'll find them instead in environments aimed at developers, the providers' "playgrounds" (Google AI Studio, for example), and in API calls. At the time of writing, this is the most common situation, but check in your tool, because interfaces change often.

Some advice going around says you can simply write "temperature 0.2" in a chatbot. Don't count on it. That sentence doesn't touch the setting: at best, the model interprets it as a wish for restraint, like any ordinary instruction. The last section shows a more reliable way to get that effect.

Temperature: caution or risk-taking

Let's go back to the mechanism. The model wants to continue the sentence "Our restaurant serves up ... cooking." It sees "traditional" as very probable, "generous" as fairly probable, "chatty" as very improbable.

At low temperature, the choice concentrates on the most probable continuations. The answer is sensible, consistent, sometimes flat.

At high temperature, the probabilities move closer together: less expected continuations get a chance. The answer becomes more varied, more surprising, and also more exposed to going off-topic or being incoherent.

The table summarizes the usual uses. These are tendencies, to be tested on your own case.

The scale varies by provider, from 0 to 1 for some, 0 to 2 for others, and a given value doesn't have exactly the same effect from one model to another. Some recent models even restrict or ignore this parameter. Check the documentation before copying a value found on a forum.

Two mistakes come up very often.

Believing that a temperature of zero makes the answer true. The setting influences variety, never accuracy. At zero temperature, the model takes its favorite continuation, including when it's wrong. An invented piece of information can therefore come back identically on every try, which gives it a false appearance of solidity.

Believing it makes the answer perfectly reproducible. A very low temperature greatly reduces variation, without always eliminating it. Some APIs offer a seed parameter to improve repeatability, with no absolute guarantee.

The other settings you'll come across

Top-p limits the choice to the most probable continuations, up to a given share of the total probability. At 0.9, the model picks from the continuations that account for 90% of that probability and ignores the others. The effect resembles that of temperature, which is why documentation generally recommends changing one or the other, not both.

Maximum length (often max_tokens) caps the size of the answer. It cuts the text off cold once the limit is reached: the model doesn't write a more concise answer as a result. For a short, complete text, ask for it in the prompt, following our advice on imposing a response format. If you work with long documents, also keep in mind the limit described in our article on tokens and the context window.

Stop sequences tell the model a string of characters at which it must stop. They are mostly useful to developers who integrate a model into an application.

Frequency and presence penalties, offered by some APIs, discourage repeating words already used. They can help with long texts that go in circles, but they aren't available everywhere and their effect remains hard to predict.

The system instruction isn't a numeric setting, but it is often found on the same screens. It's often the one that has the most effect on the result.

For everyday use, the order of action is simple: improve the prompt first, adjust the temperature next, and leave the rest at default until you have a specific problem to solve.

Getting the same effect without a slider

Most people don't have access to these parameters, and it isn't a big deal: the prompt lets you move in the same direction.

For variety, the weak prompt would be "Suggest names for my fall menu." The model then produces what it sees as most probable. Force it to explore distinct directions:

I run [TYPE OF RESTAURANT] in [CITY]. My customers: [DESCRIPTION]. I'm looking for a name for [THE MENU OR DISH], which features [INGREDIENTS OR IDEA].

Suggest 12 names divided into 4 very different approaches: poetic, humorous, local roots, and an unexpected approach of your choice that you will name.

Constraints:
- fewer than 5 words;
- avoid the words [OVERUSED WORDS, E.G. FLAVORS, DELIGHTS, TERROIR];
- easy for a server to say out loud.

For each name, say in a few words what it evokes. Finish by pointing out the two riskiest names and explain what might put people off.

The gain comes from being forced to look elsewhere: the imposed approaches and banned words rule out the most expected continuations. You can also generate several answers in separate conversations and compare them, which amounts to taking advantage of the element of chance instead of putting up with it.

For consistency, it's the reverse: reduce the freedom. Give a strict format, an example of the expected result, and an instruction for ambiguous cases.

Extract the information from the invoice below. Answer only with a table of these columns, in this order: Supplier | Date (DD/MM/YYYY) | Invoice number | Amount excl. VAT | VAT | Amount incl. VAT.

Rules:
- copy the amounts as they appear, without recalculating;
- if a value is missing or illegible, write "NOT FOUND";
- add no comments.

Invoice:
"""
[TEXT OF THE INVOICE]
"""

Even so, check the first extractions by hand, then a regular sample. If the need becomes repetitive and software has to read the result, switch to a structured JSON output.

The test to run with your next prompt

Run your usual prompt three times, each in a new conversation, and compare the answers. If they look too similar when you wanted ideas, impose different approaches. If they diverge when you wanted a stable result, tighten the format. Either way, you'll know what to fix without touching a single slider.