Devil's advocate: getting the AI to argue against your ideas

When you present a project to the AI, it agrees with you. How to ask it for the strongest objections, or to imagine the failure, before you commit.

3 min read

Two teams playing tug of war on a lawn
Photo: Ahmad Ali Karim · CC BY-SA 4.0, cropped

The owner of a pizzeria is thinking about opening on Sunday evenings. He explains his idea to an AI: a single pizza maker, a reduced menu, neighborhood customers with nothing else open. It answers that this is "an excellent opportunity" and offers him a communications plan. He's delighted. Nobody has asked about the cook, who already works six days out of seven.

Models tend to go along with you, especially when you present your idea with enthusiasm. Our article on bias talks about sycophancy. To make a decision, you need the opposite: someone who looks for what could go wrong.

Ask for the devil's advocate

The devil's advocate defends the opposite point of view to test an idea. With an AI, you have to ask for it clearly and forbid it from hedging in every sentence.

Prompt
I run a neighborhood pizzeria in [CITY] (30 seats). I'm considering opening on Sunday evenings, from 6 p.m. to 10 p.m., with a single pizza maker and a reduced menu of 8 pizzas.

Play devil's advocate. Give me the five most serious objections to this project, from the most serious to the least serious. For each one:
- explain the mechanism in two sentences;
- tell me what I could check or test before committing;
- say whether the objection would be fatal or only costly.

Don't soften anything, and don't start with compliments. If you lack the information to judge an objection, say so and ask the question.

The last sentence of the prompt gives the AI the right to say it doesn't know. Without it, it fills the gap with generalities.

Imagine it has already failed

Another approach works well for projects: the retrospective in reverse, often called a pre-mortem. You ask the AI to place itself a year later, in a world where the idea has failed, and to explain what happened. Placing yourself in the future makes it easier to write precise scenarios, whereas "what are the risks?" invites lists of precautions.

Prompt
It is [DATE + 12 MONTHS]. The Sunday-evening opening of my pizzeria was stopped after a few months because it wasn't working. Write three different, plausible stories of this failure, each in five sentences, covering what happened in the first month, the third month, and what decided the shutdown.

Then, for each story, say which sign could have alerted me as early as the first month.

The warning signs are the element to hold on to: they give you markers to track, such as the number of covers on the first Sunday, and a threshold beyond which you stop.

Have each option argued

When you're torn between two options, ask the AI to argue each one in turn, with equal force: "the strongest version that an intelligent person who believes it's right would defend." Describe the two options neutrally, without letting your preference show, because an AI that guesses your favorite tends to defend it.

What the debate won't give you

The objections are often generic: the AI knows neither your neighborhood, nor your competitors, nor your margins. If it puts forward a figure or a study, don't keep it without a source, and apply the article on checking an answer. It can also invent an objection that is brilliant and false, because it was asked to find one. For a decision involving money, a contract, health, or employees, this debate prepares your meeting with the accountant, the lawyer, or the doctor; it doesn't replace it.

After the debate

For each objection left unanswered, write down what you would do to resolve it, then decide: you go ahead, you test on a small scale (just three Sundays, for example), or you give up. To lay out a line of reasoning in detail, see step-by-step reasoning.

To try tonight

Take a decision you've half made: an investment, a price change, a hire. Describe it in ten lines without letting your view show, run the devil's advocate, then next to each objection note what you already know and what you don't. The "I don't know" column is your list of things to check.

Advanced techniques

Getting AI to Write in Your Style: The Complete Method

"Write like me" doesn't work. How to build a style sheet from your own texts, and hunt down the tics that give away generated text.

5 min read

Advanced techniques

Few-Shot Prompting: Guiding the AI by Example

Two or three well-chosen examples replace a long paragraph of instructions. How to select them, present them and avoid slavish copying.

5 min read

Advanced techniques

Scoring AI Answers With a Rubric

How to build a scoring rubric with four or five criteria to compare two prompts or two answers, and use it without fooling yourself.

4 min read