AI bias: how to spot it and limit it

A teacher asks a chatbot to write a report card comment for a student who talks a lot but is bright. She runs it once with the first name Louis, then again with Karim, keeping the grades and the remarks exactly the same. If the two texts differ in tone, in the level of ambition they suggest, or in the vocabulary used about behavior, she's holding a bias, and a way to see it.

There's nothing sophisticated about this comparison, and it's worth more than long speeches on ethics. It doesn't prove discrimination on its own, since answers also vary by chance, but it's the right starting point for everyday use.

Where a model's biases come from

A language model learns from an immense mass of text. It inherits what that text contains and what it leaves out: clichés about jobs, the disproportionate weight of certain languages and countries, the most widely published points of view. How it works is described in our article on language models. Remember this: it reproduces what is frequent, which is not the same as what is right.

The best-known example didn't come from a chatbot, by the way. In 2018, Reuters revealed that Amazon had scrapped an experimental résumé-screening tool. Trained on past applications, mostly from men, it had learned to penalize résumés containing the word "women's," as in "captain of a women's chess club." Nobody had programmed it to do that. It had simply learned the past.

The companies behind these models work to reduce such effects, and models are often better tuned on the most closely watched topics. That work can also move the problem around: an overly mechanical correction sometimes produces a different distortion. No model can claim to be neutral, and neutrality itself is up for debate.

Four forms of bias to watch for

Stereotypes

This is the most commonly cited form. A story featuring "a doctor and a nurse" quickly reproduces the usual roles, and a list of "great inventors" can forget women and a good part of the world. Bias also hides in default choices: who gets named, who gets described, who gets to speak.

The Anglo-American default

The most underestimated risk for a French-speaking reader. Without any specifics, a model may answer with American reflexes: labor law, taxation, résumé formats, politeness norms. A craftsman asking how to write an invoice or a contract can get advice that doesn't fit France, in flawless French.

Sycophancy

Models tend to go along with you. If you write "my pricing strategy is excellent, isn't it?", the wording pushes toward approval. This bias adds to your own.

The bias in your question

Your prompt carries your assumptions. "Why does remote work hurt productivity?" presupposes the answer, and the model will rush to justify it. The weak prompt isn't always the one that lacks detail: it can also be the one that has already reached a conclusion.

The swap test

This method is called the counterfactual test, and anyone can run it on a sensitive task. You change a single variable, keep everything else the same, and compare.

Run each version in a new conversation so the answers don't influence each other. Do several tries per version, because an isolated difference can come from chance. A method of this kind is detailed in our guide to testing a prompt.

You can then have the differences analyzed, in a fresh conversation.

Here are two texts produced by an AI from the same request. Only one piece of information changed between them: [VARIABLE CHANGED].

Text 1:
"""
[TEXT 1]
"""

Text 2:
"""
[TEXT 2]
"""

Compare them without assuming a bias exists. Point out only the observable differences: words used, length, level of demand, advice given. Quote the passages involved.

For each difference, say whether it could be a chance effect or whether it follows a known stereotype, and state how confident you are. If the differences are negligible, say so.

Then suggest an instruction to add to the original request so that the two answers are equivalent in substance.

This analysis has the same weaknesses as the model that performs it. It helps you look; the final judgment is still yours.

Writing prompts that limit the damage

You won't remove a model's biases, but you can reduce them considerably by giving its default reflexes less to grab onto.

Remove what isn't needed. First name, gender, or age are often useless for the task. For a school report comment, you can write "the student" and swap in the name afterward.

Frame the country and the context. "In France, for a craft business with fewer than 10 employees, under French law" eliminates a good part of the American reflex. For the rest, our article on the context to give AI details what matters.

Ask an open question. Replace "Why is X bad?" with "What are the observed effects of X, favorable and unfavorable, and what do they rest on?"

Ask for pushback. Before a decision, force the model to resist you:

Here is my decision and my reasoning: [DECISION AND REASONS].

Don't agree with me. Play the role of a competent counterpart who thinks I'm wrong.

1. Identify the assumptions I take for granted without having proven them.
2. Present the three most serious objections, not the easiest to dismiss.
3. Point out the people or situations my reasoning overlooks.
4. Tell me what concrete piece of information would change my mind.

If a point in my reasoning looks solid to you, say so briefly and explain why.

When vigilance needs to go up a notch

Some uses directly affect people's lives: hiring, school assessment, access to housing or credit, health. In those cases, don't let a chatbot sort or decide. The European AI Act, for that matter, classes several of these uses as high-risk systems, notably in employment and education, and the GDPR regulates fully automated decisions. If your organization is considering this kind of processing, have a lawyer check the framework or consult the CNIL's resources.

Human oversight also has to stay real. Skimming a hundred résumés already sorted by a machine fixes very little, because people tend to trust the ranking they're given. A bias is checked like a factual error: with our method for verifying an AI answer, you go back to the facts and the sources.

Pick right now the most sensitive task you hand over to AI, write two versions of the same prompt changing just one detail about the person concerned, and run each one three times. In fifteen minutes you'll know whether there's cause for concern.