Summarizing and querying long documents with AI

You drop an 80-page requirements document into your assistant, ask "give me a summary," and get ten well-turned lines. What do they cover? Probably the most general ideas, the ones at the beginning. The late-delivery penalty clause on page 63, the one that matters to you, isn't in there.

A summary can be accurate and useless. With a long document, the question to ask before the prompt is about the reading you want to do: learn what it's about, find a piece of information, or extract a set of points. Each goal calls for a different method.

First check what the AI actually read

Before any question, make sure the text came through. Three problems come up often:

• The scanned PDF. An image of a page contains no text. Some tools can read it, others less well, and tables can come out jumbled.
• The document that's too long. Depending on the tool, part of the text may not be taken into account, or may be consulted only in excerpts. You're not necessarily told. The limits depend on the tool and your plan, and they change: check at the time you're working. The article on tokens and the context window explains the mechanism.
• The layout. Footnotes, columns, and repeated headers can cut sentences in two.

A quick check avoids many bad surprises.

Before working on this document, do a reading check:
1. Give the title and the outline with the section numbers as they appear in the document.
2. Copy word for word the first sentence of the last section, and the last sentence of the document.
3. Point out the passages you can't read or that seem badly extracted (tables, images, cut-off text).

Don't summarize anything for now.

Compare with the original. If the ending doesn't match, you know you'll have to split it.

Look for a precise answer, don't summarize

For a craftsman responding to a call for tenders, a summary has no value. He wants to know which documents to supply, by what date, which penalties apply. The right approach: ask closed questions and demand the source passage.

Before:

Summarize this tender package.

After:

You're going to help me analyze this package so I can decide whether to bid. My business: [YOUR TRADE, YOUR SIZE, YOUR AREA].

Answer in a table with the columns: Question, Answer, Exact quotation, Location (section or page).

Questions:
1. What is the deadline and the method for submitting the bid?
2. Which documents are mandatory?
3. What are the selection criteria and their weighting?
4. Are there penalties or unusual clauses? List them all.
5. What certifications, insurance, or references are required?
6. [YOUR SPECIFIC QUESTION]

If the document doesn't answer a question, write "Not found" and deduce nothing. After the table, list the contradictions between the different parts of the document.

The quotations let you verify in a few seconds with Ctrl+F. Treat them as a lead to check: a model can misread, and a "Not found" doesn't prove a clause is absent. For a decision with real stakes, like a contract or a public procurement contract, reread the original text and consult a professional if needed.

The logic of answers anchored in the document is detailed in the article on reducing hallucinations.

Splitting to really summarize

When you want a real summary of a very long text, say a federation's annual report, work in two stages. You summarize each part separately, then synthesize those summaries. Specialists call this map-reduce. Each part gets the same attention, and the end of the document no longer falls through the cracks.

The prompt for each part imposes the same format, so you can assemble them afterward:

This is part [NUMBER] of [TOTAL] of a document titled [TITLE]. I'm reading it for [YOUR GOAL].

From this part, extract only:
- the decisions or conclusions;
- the important figures, with their context;
- the points that could concern [YOUR BUSINESS OR YOUR ORGANIZATION];
- the cross-references to other parts of the document.

150 words maximum. Cite the section title for each point. Don't interpret, and add nothing that isn't in this excerpt.

Then, with all the summaries together:

Here are the successive summaries of all the parts of a document. Write a synthesis of [LENGTH] for [RECIPIENT].

Structure: what to take away, what changes compared with [PREVIOUS PERIOD OR VERSION, IF KNOWN], what to do or watch.

Keep the section references. If two summaries contradict each other, point it out instead of choosing. Add no information absent from the summaries.

Summaries:
[PASTE THE SUMMARIES]

Split the text by its chapters rather than by page count, so you don't separate a rule from its exception. As in any prompt chain, you can reread each intermediate summary before moving on.

Its weakness is well known: what wasn't retained in the first stage won't come back. So state your goal from the start, as in the prompt, so the selection is made according to what interests you.

Tools designed for querying documents

General-purpose assistants let you attach files, and some offer spaces where you keep several reference documents. There are also tools specialized in consulting sources, like Google's NotebookLM, which point back to the passages used. These features evolve fast: look at what your tool offers at the time you're reading.

The format you upload matters too. When you can, prefer clean text to a complicated PDF: a Word document or copied text often goes through better.

A reminder before sending anything: a contract, payslips, or a client file can contain sensitive data. Look at what not to paste into a chatbot, the terms of your plan, and, if you have a serious doubt, the CNIL's recommendations or the person in charge of data in your organization.

Prepare your questions before opening the file

The next time a long document arrives, first write five questions you need to answer, without reading a single page. Run the reading check, then the table with quotations. Finally, open the document at the locations indicated and read those passages yourself: you'll spend a few minutes on it, and you'll know exactly what you've verified and what you haven't.