How ChatGPT works under the hood, explained simply

You type 'recommend me a film for tonight' and two seconds later you have three options with reasons, neatly ordered, sign-off emoji included. The feeling is always the same, there's someone in there who knows things. Well, no. In there is a machine that has only ever learned to guess the next word.

Its entire education was one fill-in-the-blank exam

ChatGPT works by predicting, word by word, the most likely continuation of a text. It was trained on a single exercise repeated trillions of times, it's shown a chunk of the internet with the ending covered and has to guess what comes next. Every miss adjusts the machine a little. That's it. There are no other subjects.

With every attempt its parameters get tweaked, the billions of little internal dials that keep adjusting until the machine guesses right more and more often. And to grade that many exams, text first gets chopped into tokens, the word-pieces we covered in their own explainer. After reading a colossal share of everything humanity has written, the machine hasn't memorised the internet, it has distilled its patterns, the same way you don't remember every sentence of your childhood but know perfectly well that 'Once upon a...' ends with 'time'.

The trick is that filling the blank forces you to learn everything else by accident. To guess the missing word in 'the capital of Australia is ___' you need geography. To continue a half-written program you need to know how to code. Nobody taught it those subjects, they slipped in through the blank.

Reinforcement learning

That student fresh out of the infinite exam isn't ChatGPT yet, it's an internet impersonator, with everything good and everything horrible that implies. Write it a question and it might answer with another question, because that's what people do in forums.

What turns it into the polite assistant you know is a second phase with human teachers, people who grade its answers, better marks for the helpful and courteous ones, worse for the invented or unpleasant ones, and the machine readjusts to please. Engineers call it reinforcement learning from human feedback; you and I would call it manners lessons. They work so well that it sometimes overshoots into flattery, which is exactly what some complain about.

Try it tonight, contradict something you yourself said two messages earlier. See how long it takes to agree with you. It isn't weak-willed, it has no will, it has grades from its teachers, and the grades reward being likeable.

Why it makes things up (and doesn't know it's making them up)

Now you can understand its most famous flaw. When you ask something outside its patterns, the machine doesn't look anything up or double-check, it just generates the most likely continuation, and the most likely continuation of 'what year did this author publish that book?' is a year that sounds right. The student never leaves an answer blank. We call that hallucination, and it isn't an occasional glitch, it's the same mechanism that writes its poems, just without a safety net.

Hence some advice: ask it for sources and check them, hand it the real document when you have one, and distrust on principle any figure, date or quote you can't verify. And remember its working memory, the context window, is the exam desk, only so many sheets fit on it, and whatever falls off the desk stops existing for it.

So how can it discover new mathematics?

The question asks itself, and it's the right one: if this thing only fills in blanks, how come the models of 2026 win maths olympiads, propose molecules for new drugs or find algorithms no human had written? The first part of the answer you already have, even if it doesn't look like it: guessing the blank well forces you to build an internal map of how the world works. There is no way to complete millions of physics problems without distilling, by accident, something that looks an awful lot like knowing physics.

The second part arrived with reasoning models, the ones that think before they speak: instead of blurting out the first continuation that comes up, they write a hidden draft, try paths, catch their own mistakes and only then answer. It's still the same blank-filling machine, but now it fills in the most important blank of all first, the one that reads 'which steps get me to the right answer?'.

And the leap to discovery has a third ingredient: a grader that isn't human. When a model proposes thousands of proofs, molecules or programs, nobody needs to take its word for it, because the proof gets verified with pure logic, the program gets executed and the molecule gets tested in a lab. The machine proposes at scale and reality prunes. That's how mathematical results and drug candidates that sat in no book have emerged: the student never stopped filling in blanks, it just fills in blanks nobody had filled before, with an infallible judge watching.

Next time it writes you a flawless cover letter, you'll know what's underneath, no encyclopedia, no consciousness, just the world champion of filling in blanks. If these plain-language explainers are your thing, we've also covered what ray tracing is and what stem cells are.