How a Language Model Works, Explained Without Jargon

Ask a chatbot for the biography of a little-known craftsman from your town, without giving it web access. Chances are you'll get a smooth text with a date he set up shop, an apprenticeship under a master cabinetmaker and an anecdote about the workshop. Every bit of it could be false, and the tone won't give anything away.

This behavior looks like lying. It's actually explained by the very mechanics of these tools. Once you understand them, you know what to expect: why you need to paste in your sources, why a long conversation degrades, why two answers to the same question differ. No math formulas below, only the ideas that change how you use them.

A machine that predicts what comes next in a text

A language model does one thing, at an enormous scale: given a text, it estimates what is most likely to come next.

Take this sentence: "The baker takes the bread out of the." You complete it with "oven" without thinking. The model does the same, except it doesn't pick a single word. It calculates a probability for every possible continuation: "oven" very high, "bakehouse" fairly high, "bag" lower, "refrigerator" almost zero. It picks one, adds it to the text, then starts over with the lengthened sentence, and so on until the end of the answer.

Everything you see comes from this loop: the follow-up email, the Python code, the recipe, the contract summary. When you ask a question, the model produces the text that most resembles what follows that question in everything it has learned. It answers because, statistically, a question is followed by an answer.

The comparison with your phone's autocomplete is useful, but it has its limits. Your phone looks at the last two or three words. A language model takes into account all the text it has in front of it, and it has absorbed enough regularities to translate, reason through a problem or imitate a style. This difference in scale changes what it's capable of.

Does the model understand what it writes? Researchers debate it and the question remains open. For your purposes, a practical answer is enough: it handles the meaning of words very well, and it has no direct link to reality. It only knows what texts say about it.

What "training" a model means

Nobody writes the rules by hand. Nobody programmed that "oven" follows "bread out of the." The model works them out itself, by reading a colossal amount of text: web pages, books, code, forums, depending on what each company has gathered.

Training is like a fill-in-the-blank exercise repeated endlessly. The end of a sentence is hidden, the model tries to guess it, the guess is compared with the real continuation, then its internal settings are adjusted slightly so it gets it a little less wrong. Those settings are called parameters, and large models have billions of them. Picture a mixing board with an absurd number of knobs, each one turned a hair's breadth with every attempt.

What comes out of this stage is a model that continues any text, but isn't an assistant yet. If you write it a question, it may very well carry on with more questions, as in a questionnaire. A second phase therefore teaches it to behave like an assistant, relying notably on examples of good answers and on human evaluations. That's also when it learns to decline certain requests and to adopt a helpful tone.

Three very concrete consequences follow.

The model is not a database. It doesn't keep texts filed away in documents it could look up. It keeps regularities, dissolved in its parameters. A very frequently repeated sentence can come back almost word for word, but a rare piece of information, like the phone number of your village's town hall, may have left a trace too faint to be reproduced correctly.

Its knowledge stops at a certain date. Training is done on frozen data, and everything that happened afterward is unknown to it, unless the tool gives it web access or you supply the information.

It doesn't learn from you while you talk to it. Your messages don't change its parameters in real time. Some providers may reuse conversations for future training, depending on your settings, which is why it pays to check what you paste in. The "memory" features some tools offer are something else: notes saved on the side and fed back into the conversation.

You also need to distinguish the model from the product. The model is the prediction engine. The chatbot you use surrounds it with hidden instructions, web search, code execution, file reading. When a tool gives you a correct figure from a recent page, it's often that tooling that did the work, and the model that wrote it up. These features vary from one tool and plan to another, so check what's active in yours.

How words become numbers

A computer calculates; it doesn't read. So text has to be translated into numbers, and that translation happens in two steps.

First, the text is cut into pieces called tokens: a short word, a part of a long word, a punctuation mark. This splitting explains many practical limits, covered in the article on tokens and the context window.

Then each piece gets a long list of numbers, which you can picture as coordinates on an immense map. Training moves items that appear in similar contexts closer together on this map. "Invoice" ends up not far from "quote," "cat" not far from "dog," and the two groups stay far apart. Meaning becomes a matter of distance.

That leaves the case of words with several meanings. Compare these two sentences:

The bass was the loudest thing in the room.

The bass was too bony to eat.

Same word, two realities. The mechanism that handles this problem is called attention. To process each piece of the text, the model looks at all the others and weighs the ones that matter. Here, "loudest" pulls "bass" toward music, while "bony" and "eat" pull it toward the kitchen. The architecture that generalizes this idea, called the Transformer, was described in 2017 by Google researchers in a famous paper, "Attention Is All You Need." Most of today's large models descend from it.

This mechanism has a direct effect on your prompts: the model weighs everything in the text, including your digressions. An off-topic sentence or a misplaced detail influences the answer, even if only a little.

Why it makes things up with such confidence

Back to our imaginary craftsman. The model has read thousands of craftsman biographies. It knows what they look like: a date, training, passing the trade on, a devotion to the craft. When it has no information about the person, it still produces the most plausible continuation, because that's its function. Nothing in the prediction loop checks that a sentence corresponds to a fact.

This is what's called a hallucination. The word is misleading, because nothing was "seen." It's more of an extrapolation that takes the form of a confident answer.

An accounting request shows the trap well. If you ask for the article of the Code général des impôts that governs a specific deduction, the model knows that an answer of this kind contains a well-formed article number. It can produce one that looks right, that doesn't cover the topic, or that has changed since. The format is perfect and the content needs checking.

Two misconceptions deserve to be set aside:

• A confident tone says nothing about reliability. A model doesn't get more hesitant because it's wrong. The assertive style is part of what it has learned.
• Asking whether it's sure isn't enough. The model may stick to its error, or on the contrary drop a correct answer because your question suggests doubt. It tends to go along with whoever it's talking to.

Recent models are wrong less often, and many flag their uncertainty better. The risk goes down without disappearing. To reduce it, you can work on the prompt, as our guide to reducing hallucinations explains, then check the result.

Everyday oddities, decoded

Most strange behaviors are explained by what came before.

The point about different answers comes from a setting you can sometimes use, described in the article on temperature and other parameters. The point about memory often surprises people: with every message, the app sends the model the text of the conversation again. It doesn't remember the way you remember; it rereads.

What this changes in your prompts

Once you understand the mechanics, the good habit boils down to this: your prompt is the beginning of a text, and the model is going to continue it. The more that beginning looks like the start of a good document, the better the chances that what follows is good.

Here's a typical request from a nonprofit manager:

Write a message announcing our annual general meeting.

The model has nothing to hold onto in reality. So it will produce the average of all the annual meeting announcements it has read, with an all-purpose tone and probably a venue left blank or made up. The same need, written with its workings in mind:

Here is the exact information about our annual general meeting. Use only this information and don't add any.

Organization: [NAME AND ACTIVITY]
Date, time and place: [DETAILS]
Agenda: [ITEMS]
What members are expected to do: [VOTE, PROXY, CONFIRM ATTENDANCE]

Write an email of under 150 words for our [NUMBER] members, many of whom are retirees and read on their phones. The tone is warm and simple. Start with the date. End with what they need to do and the deadline.

If essential information is missing, write [TO BE COMPLETED] in its place instead of guessing it.

Each change answers a mechanism seen above:

• The facts are in the text, so the model doesn't have to reconstruct them from memory, where it's most fragile.
• The audience, length and tone reduce the number of plausible continuations. The answer moves away from the average.
• The [TO BE COMPLETED] instruction gives it an allowed way out. Without it, the plausible text wins out over a blank.

This prompt clearly improves the odds, without guaranteeing anything. The model can still slip in a detail you didn't ask for, and you'll need to reread every date and every name. For that proofreading, keep our method for checking an AI response handy.

Now redo the craftsman test: ask your chatbot, with web search turned off, for a biography of someone you know well who isn't famous. Read the answer, note what's wrong, then in a new conversation paste in three accurate facts about that person with the instruction "use only this information." The difference between the two texts is worth more than any explanation.