Getting AI to Reason Step by Step: When and How

A tiler has to lay 34 m². He figures 10% for waste, and his tiles are sold in boxes of 1.44 m². He asks the AI:

How many boxes should I order?

If the model just answers "26 boxes," the number is right, but nothing proves it. On another job, a bare answer like that could be wrong, and you'd have no way to see where it went off track.

Now ask for the detail: 34 × 1.10 = 37.4 m², 37.4 ÷ 1.44 = 25.97, so 26 boxes, or 37.44 m² delivered. You see every assumption, and you pick up a useful piece of information along the way: the remaining margin is 0.04 m². With the slightest extra breakage, you're short a box. The short answer hid that.

Why the detail often improves the answer

A language model produces its answer word by word. If it has to give the result on the very first line, it commits without having laid out the intermediate steps. By asking it to write them out first, each step builds on the previous one and the result comes at the end.

The technique is called chain of thought. A team of researchers (Kojima and coauthors, 2022) showed that the single phrase "Let's think step by step" noticeably improved results on the reasoning problems of the time. Models have changed a lot since, and several tools now offer a built-in thinking mode. Check yours as you read this: if that mode exists, turn it on for hard problems, and the magic phrase becomes less useful.

The second benefit remains intact, and it's the one that matters most day to day: written reasoning can be reread. You spot the wrong assumption, the badly chosen rate, the figure the AI made up to fill a gap.

The short phrase isn't always enough

"Think step by step" works, but you get better results by saying which steps you expect and what the AI should do when a piece of data is missing.

I need to solve this problem: [DESCRIBE THE PROBLEM WITH ALL THE NUMBERS]

Proceed in this order:
1. List the data provided, then the assumptions you have to make, flagging them as such.
2. If an essential piece of data is missing, stop and ask me instead of estimating it.
3. Do the calculation step by step, one operation per line, with units.
4. Check the result with another method or a reverse calculation.
5. End with a line starting "Answer:" and say what would change this result.

Step 2 prevents the most troublesome flaw: flawless reasoning built on an invented number. Step 4 forces a check, without guaranteeing it. For any amount involving your money or a client's, redo the calculation in a spreadsheet: the AI can still get arithmetic wrong, especially on long operations.

Where it helps, where it changes nothing

Asking for reasoning makes the answer longer. No need to do it everywhere.

Watch out for the last case. On a fact the model doesn't know, well-presented reasoning gives an impression of rigor without verifying anything. For that type of risk, read about how to reduce hallucinations instead.

Separate the reasoning from the answer

When the result needs to be reused as is, the reasoning gets in the way. You can keep it to yourself and get a clean output by giving the model two distinct zones. Here's an example for a nonprofit treasurer preparing a decision for her board:

Our nonprofit is torn between two options for [DECISION TOPIC].
Option A: [DETAILS, COSTS, CONSTRAINTS]
Option B: [DETAILS, COSTS, CONSTRAINTS]
Our criteria, from most to least important: [CRITERIA]

Answer in two parts.

REASONING part: examine each criterion one by one for both options. Write out the calculations. Flag what you can't assess with this information.

NOTE TO THE BOARD part: 6 lines maximum, no jargon, with the recommended option, the main reason, and the risk to watch.

You read the first part to check, and pass along the second. If you're working with a required format, like an output meant for a spreadsheet, keep the same logic and copy only the final zone.

Have a second pass review the reasoning

A model that has just produced a chain of reasoning tends to defend it if you simply ask "are you sure?" It may also change its mind for no reason, just because the question suggests doubt. A better check is to open a new conversation and present the work as someone else's.

Here's some reasoning written by a colleague. I don't know whether it's correct.

Original problem: [PROBLEM]
Reasoning: [PASTE THE REASONING]

Go through it line by line. For each line, say "correct" or explain the error. Look in particular for unjustified assumptions, unit errors, and data that's absent from the problem. If you find no errors, say so without inventing any.

The last sentence limits the temptation to find a problem at all costs. Reasoning reviewed twice can still be wrong, but the glaring errors often drop out at this stage. If the problem is long, break it into a prompt chain and have each link checked rather than the whole.

Try it on your next calculation

Take a calculation you already know from your work, with an answer you know is right. Run it through the first prompt above and read the assumptions in step 1. If the AI added one you hadn't given it, you know what to specify in your problem statement next time.