Reducing AI hallucinations: the methods that work

In 2023, in New York, lawyers filed a brief with a federal court citing court decisions that didn't exist. They had been generated by ChatGPT, and the judge sanctioned the lawyers (Mata v. Avianca). The references looked perfect: party names, case numbers, summaries.

This case sums up the problem well. A hallucination, the common term for false information produced with confidence, is hard to spot because it has the shape of a good answer. No prompt makes the phenomenon go away. You can, however, make it noticeably less frequent and, above all, easier to detect.

Why the AI invents without knowing it

A language model doesn't consult a database of facts when it answers on its own. It predicts a plausible continuation of text based on what it has learned. On a well-documented subject, plausible and true often overlap. On a rare detail, it keeps producing plausible text, because nothing forces it to stop. For the full mechanics, read how a language model works.

That explains where to look first:

• references: articles, books, rulings, web links;
• quotations in quotation marks attributed to someone;
• precise figures, dates, and amounts;
• local or recent information, such as a schedule, a rate, or a regulation;
• niche technical details: a library function that seems to exist, a software option.

If your request touches one of these points, the instruction needs to change. For rewording an email, you don't need any of this.

Explicitly allow "I don't know"

An assistant tries to answer. If your prompt provides no other way out, it produces one. Giving it the right not to know often reduces inventions, provided you say so precisely.

Before:

Give me 5 recent studies on the effect of the 4-day week on productivity, with the authors and key figures.

This prompt demands five specific results: it pushes the model to complete the list no matter what.

I'm looking for serious studies on [TOPIC].

List only those you're reasonably sure exist, even if there are fewer than five, or none. For each one, state what you know for certain (authors, year, institution) and write "to be verified" for everything else. Give no link, figure, or exact quotation if you're not sure of it.

End with the keywords and databases where I can check for myself.

The last sentence provides a useful output even when the model has nothing solid. An answer full of "to be verified" tells you more than a neat, false list.

Answer from a supplied text

The most effective method is to change the question. Instead of asking the AI what it knows, ask it what a document you give it says. The job becomes reading, an exercise far more reliable than memory.

Example: an association secretary wants to check what the bylaws provide for calling a general meeting.

Answer my question using only the document below.

Rules:
- Start by quoting word for word the passage or passages you rely on, giving the article or page.
- Then answer in 3 sentences maximum.
- If the document doesn't allow an answer, write "The document does not specify this" and add no outside knowledge.

Question: [YOUR QUESTION]

Document:
[TEXT OF THE DOCUMENT]

The order matters. By quoting first, the answer rests on the selected passages, and you can check the quotations with a simple search in the document. A quotation you can't find signals that you should be wary of the rest. For documents of several dozen pages, the method gets a little more complicated, and the article on long documents goes into detail.

Connect the AI to real sources

Several tools can search the web or your files before answering, and display their sources. This option reduces the risk on recent topics, without eliminating it. A displayed link can exist without saying what the AI claims it says, or come from a poor-quality page.

The habit to keep: open the link for any information you're going to reuse, and look for the exact sentence on the page. For developers, the equivalent is to supply the documentation for the right version of the library, run the code, and run the tests. A call to an invented function shows up immediately at runtime, as long as you run it.

Have it list the claims to verify

Once the text is produced, a second pass helps you know where to focus your review. Models judge their own reliability poorly: don't take this sorting as proof. It serves to prioritize the checks, and to keep you from rereading every line with the same attention.

Here is a text produced by an AI. I have to check it before publication.

Draw up a table of the verifiable claims it contains (figures, dates, names, quotations, cause-and-effect relationships). For each one, indicate:
- the sentence concerned;
- the type of source that would allow it to be verified;
- the risk of error: high, medium, or low, with a short reason.

Sort the table by decreasing risk. Don't correct anything and don't claim that any piece of information is true.

Text:
[PASTE THE TEXT]

You get a checklist. For the verification method itself, read how to verify an AI answer.

What no prompt fixes

These instructions lower the risk; they don't cancel it. A model can write "I'm sure" and be wrong, just as it can distort a document it has just read, especially when the answer requires combining several passages. On a medical, legal, or financial topic, treat the result as a starting point to check with an official source or a professional.

The question to ask before using an answer fits on one line: if this information is wrong, what happens? If the answer is "not much," move on. If the answer involves a client, money, or your signature, open the source yourself, then add to your usual prompt the line "write 'to be verified' for anything you're not sure of."