Meta-prompting: have the AI write your prompts

AI can write and critique your prompts, as long as you make it ask questions first. Three prompts to copy and the flaws to fix.

5 min read

Yellow typewriter
Photo: Suyash Dwivedi · CC BY-SA 4.0, cropped

Ask an AI "write me a prompt for creating math exercises" and you'll get a nice big block: "You are an experienced and passionate math teacher with twenty years of experience..." It looks serious, it's long, and it knows nothing about your class. You'll have gained a more impressive text, with no better result to show for it.

Meta-prompting, meaning using AI to design your prompts, works very well when you change your approach. A model knows the usual prompt-writing advice well, but it knows nothing about your need. Your job is to make it close that gap before you let it write.

Start by making it ask questions

When you ask for a prompt right away, the AI fills the gaps with average guesses. Reverse the process: it interviews you, you answer, and only then does it write.

Prompt
I want to build a reusable prompt for this task: [DESCRIBE THE TASK IN ONE OR TWO SENTENCES].

Don't write the prompt yet. First ask me the questions you need, one at a time, waiting for my answer to each. Six questions maximum. Focus on:
- what the result will be used for and who will read it;
- what varies from one use to the next;
- an example of a good result, if I have one;
- what would make the result unusable;
- the information the AI won't have and must not make up.

When you have enough information, write the final prompt with variables in [BRACKETS]. The prompt should be as short as possible. Delete any sentence that doesn't change the result, and don't use a flattering role or phrases like "world-renowned expert."

In practice, the questions are sometimes worth as much as the final prompt. A fifth-grade teacher whom the AI asks "what would make these exercises unusable?" may answer that her students haven't learned decimals yet, or that each worksheet must fit on one photocopied page. Without that question, those constraints would never have landed in the prompt.

Answer the way you'd talk to a colleague, and answer with details. Three precise lines are worth more than one word. If you have an example of a good result, paste it in: it helps more than any description (the article on few-shot prompting explains why).

Recognizing the flaws in a generated prompt

Prompts written by an AI share certain tics. Once you know them, you can remove a good portion of them in a few seconds.

FlawWhat it looks likeFix
Inflated role"World-renowned expert with 20 years of experience"A plain role that's useful for the task, or nothing
Politeness instructions"Make sure to deliver a result of the highest quality"Delete
Abstract criteria"Engaging and punchy text"A checkable constraint: length, audience, example
Invented rulesA constraint you never asked forDelete or validate
Too many sectionsSix headings for a simple taskMerge into a few lines

The test to keep in mind is the same as for a hand-written prompt: remove a sentence, rerun, compare. If nothing moves over two or three tries, the sentence wasn't doing anything.

Asking for a critique rather than a rewrite

Already have a prompt that works halfway? Don't ask "improve it": you'll get back a longer version with no diagnosis. Ask first for an analysis based on a real result, which forces the model to explain what's wrong.

Before:

Improve this prompt.

After:

Prompt
Here is a prompt I use, and a result it produced. This result doesn't work for me.

PROMPT:
[YOUR PROMPT]

RESULT OBTAINED:
[THE RESULT]

WHAT'S WRONG:
[DESCRIBE IN ONE OR TWO SENTENCES, FOR EXAMPLE: TOO GENERIC, TONE TOO SALESY, IGNORES THE LENGTH CONSTRAINT]

Proceed as follows:
1. Identify what in the prompt probably explains this flaw. Quote the sentence involved or point out what's missing.
2. Suggest the smallest possible change, in the form of sentences to add, remove, or replace.
3. Then rewrite the complete prompt with only those changes.

Don't change anything else and don't lengthen the prompt without reason.

The manager of a bed-and-breakfast who gets overblown replies to his guests may discover that the word "prestige," slipped into his description, is dragging everything else along. A one-word fix beats a full rewrite, because you understand what changed. You can chain these fixes as in any process for improving a prompt through iteration.

Testing the prompt with cases that can break it

A generated prompt looks finished. It isn't until it has faced real cases. The AI can help here too, by imagining the situations you wouldn't have considered.

Prompt
Here is a prompt I want to use regularly:

[YOUR PROMPT]

Suggest 6 test inputs to check its robustness, in a table with three columns: the input to use, what it's trying to break, the expected behavior.

Include: a normal case, a very short input, an incomplete input, an off-topic request, an ambiguous case, and a case where the right answer would be to refuse or ask a question.

Run these cases in a new conversation, with the prompt alone. Read the results yourself: the AI that wrote the prompt isn't a good judge of its own work. Note what fails and go back to the critique prompt. The testing method gives a simple framework for keeping a record of your attempts.

The limits to keep in mind

A generated prompt doesn't excuse you from knowing what you want. If your need is fuzzy, the AI will produce fuzzy text, nicely presented. It also doesn't contain your professional knowledge: a baker knows that a certain allergen wording is a problem; the model may not.

A prompt written by one model isn't necessarily optimal for another, either. If you switch tools, retest it. And for a prompt you'll reuse often, store the validated version somewhere: it deserves a place in your prompt library.

Run the interview on your most repetitive task

Choose a task you do at least once a week, copy the first prompt, answer the questions. Then compare the result with what you usually get on a recent case. If the gap is clear, keep the prompt. If not, look at which interview question helped you most and answer it in more detail next time.

Advanced techniques

Getting AI to Reason Step by Step: When and How

"Think step by step" sometimes helps and does nothing elsewhere. The cases where it changes the result, and prompts to check the math.

5 min read

Advanced techniques

Reducing AI hallucinations: the methods that work

An AI invents sources, figures, and functions with total confidence. The instructions that limit the risk, and what no prompt can ever fix.

5 min read

Advanced techniques

Rewriting a text without distorting it

You ask for a letter to be "better written" and the AI changes the meaning, adds promises, or erases a nuance. How to fix what must not move.

3 min read