Ten thousand lifetimes of reading.

A note before we start. This post came out of a conversation between myself and Claude, Anthropic's AI. It didn't start with "write me a blog post about AI". One vague, general prompt gets you one vague, general answer, and that is the mush people call slop. I set out to understand the concepts behind these tools well enough to explain them here. That meant asking questions, pushing back, and testing the explanations until they were sharp enough to share. I get better output from a discussion than I do from a simple prompt.

And it doesn't arrive finished either. This post went through several rounds between us, with me steering and cutting and asking for changes, before it was done. Working with AI on something like this means drafts passed back and forth rather than one prompt and a finished result. I then ran the draft I was happy with past Gemini and ChatGPT for a second and third opinion. Most of their praise I ignored, for reasons the flattery section below makes clear, but the useful fixes made it in. That applies to blog posts and to most of the written work your business will use AI for. The questions in here are mine, the cuts are mine, and you can judge the answers for yourself.

Nobody labelled the internet

Most people's picture of machine learning is the handwriting example. You show a computer thousands of scrawled digits, a human labels each one ("that's a 1, that's a 7"), and the machine learns the pattern.

So how do you do that with the whole internet? Nobody sat and labelled it.

The trick is that text labels itself. Take the sentence "The cat sat on the mat" and hide the last word. When the model guesses, the right answer is sitting there in the text to mark it against. Nudge the model slightly towards the truth, move on, and repeat that trillions of times.

That's the whole training method. The next word is the label.

It doesn't predict a word. It predicts the odds.

My next question was the obvious one. Different words follow the same phrase in different sentences. Sometimes the cat sat on the mat, sometimes the sofa, sometimes the roof. How does the model cope?

It doesn't pick one word at all. It gives a probability to every word it knows, all at once: mat 40%, sofa 15%, roof 10%. The contradictions in the training text aren't a problem. They're exactly what's being learned.

And when the model writes, it rolls weighted dice on those odds at every single word. That's why you get a different answer when you ask the same question twice.

There's no backspace

Everything the model writes, even a full report, comes out through that same loop: one word at a time, with each new word predicted from everything before it, including its own. And here's the part that surprised me. Once the model has written a word, it can't delete it. If it's headed somewhere wrong, its only option is to write "actually, let me reconsider" and carry on. It cannot go back and change what it already wrote.

Worse, the most likely continuation of a confident wrong sentence is another confident wrong sentence, so early errors snowball. This is why AI can be wrong and still sound convincing all the way through. It's not lying. It's completing the pattern it started.

The newest "reasoning" models were trained specifically to catch themselves. Their hidden working is full of "wait, that's not right, try again". Everywhere else the revision pass has to come from you. Asking it to check the answer against the source, or to argue against its own conclusion, often catches errors it sailed straight past. A neutral challenge works better than "are you sure?", which a model tuned to please will often agree with.

Now the scale

The numbers here are the part I found hardest to picture.

A top model trains on roughly 15 trillion tokens, which is about 11 trillion words. A person reading a couple of hours every day for a whole lifetime gets through maybe a billion words. The training set is about ten thousand human lifetimes of reading.

That reading is compressed, over roughly three months, on tens of thousands of specialised chips drawing the power of a small town, into hundreds of billions of numbers: plain, adjustable numbers, like dials. Every dial gets nudged a microscopic amount, trillions of times. There is no database inside it, and "Paris is the capital of France" isn't stored in any one place: it is spread across billions of dials.

And the finished result is a file of a few hundred gigabytes. Ten thousand lifetimes of reading fits on a decent hard drive.

One more comparison, in the other direction: a child hears a few million words a year and speaks fluently by three, on maybe ten million words in total. The model needed roughly a million times more text to get there. And your brain runs on about 20 watts while the training cluster draws megawatts. Nobody has worked out how to copy whatever brains are doing.

• 10,000

  Human lifetimes of reading in one training set.

• 1,000,000

  Times more text than a child needs before they speak fluently.

• 20watts

  What your brain runs on. The cluster that trained the model drew megawatts.

The economics then flip. All that power and money goes into the training, which for a top model runs into hundreds of millions. Once the dials are set, asking the model a question is cheap: it is one quick pass through the frozen numbers, costing pennies at most. That's why a subscription costs about what a phone contract does.

Nobody ordered these features

The strangest part is that most of what these models can do was never designed. The training task was "predict the next word", and everything else fell out as a side effect. The builders were as surprised as anyone.

Some abilities appeared as the models got bigger. Smaller models couldn't do arithmetic, translate languages, or write working code. Then, at a certain scale, those skills appeared to switch on. Researchers still debate whether they arrive suddenly or grow steadily and only look sudden when they cross a test's pass mark. Either way, nobody added them. The best explanation for why they show up at all: at some point, the cheapest way to keep predicting text well is to learn the underlying skill for real.

Another surprise was learning from examples. Show a model two or three examples of a task in your prompt and it picks up the pattern and continues it. It was never trained to do that. It emerged because the internet is full of lists and repeated structures, and continuing patterns is what good prediction requires.

The flaws emerged the same way. Nothing in the training objective checks truth, only what text is likely to come next, so a confident wrong answer scores just as well as a right one. And the flattery you may have noticed, where AI agrees with you a little too readily, came from a later stage: models tuned on human feedback learned that people rate agreeable answers higher.

Even the personality is emergent, and nobody sat down and wrote one. The model absorbed every voice on the internet, and a final round of tuning settled it into one consistent character.

So the honest summary is this: the capabilities were not built, and neither were the flaws. Both fell out of the same simple objective, run at enormous scale. The impressive parts and the unreliable parts have the same cause, so you have to know which is which.

"It's just fancy autocomplete"

The sceptics are right that it is autocomplete. What changed how I think about it is what accurate prediction takes.

To predict the next word of a medical report, you have to model medicine. Text carries the structure of the world that produced it, and predicting text well means picking that structure up. The same is true of a contract, a set of management accounts or a tender response. The conventions are sitting in the text, along with the reasoning behind them and the things practitioners never bother to write down.

Autocomplete so good it had to learn your industry to keep doing it. That's what's sitting behind the chat box.

None of that structure tells the model whether the content is true, for the reason the last section gave. Learning the shape of a medical report is not learning whether this patient is ill. It can be fluent in the shape of your industry's work and still be wrong about the case in front of it.

What this means for your business

The practical conclusion should be a relief. For the work most SMEs do every day, drafting, summarising, research and reports, the current models cleared the bar a while ago. You are not behind because you're on the wrong tool or missing the latest release.

The bottleneck isn't the model. It's the questions, the context you give it, and the checking you do afterwards. This post exists because of an hour of good questions and several rounds of checking. That is the difference between AI slop and AI that is useful. Those skills can be taught, and that is what we do.

If your team is still typing one-line requests and getting mediocre answers back, that's fixable in a day.

Written with Claude, then steered, cut and checked by me. I answer for it.