Are the models actually getting worse — or are we?

Every few months, the same wave. Someone writes "GPT got dumber", someone else "Claude got lobotomised", someone posts two screenshots side by side and demands the March checkpoint. The forums fill up. Then a new model ships, the world is fine for two weeks — until the sentence comes back.

I know the feeling. I have it myself, regularly, on client work. My less comfortable thesis: often the models are not getting worse. Often we are getting lazier. And noticeably often at exactly the moment the models get better and we start trusting them more.

The short answer

Both happen. Providers change routing, precision, safety filters, load shedding — and yes, that changes behaviour, sometimes obviously, sometimes quietly. But the loop I see most often in my own work runs like this:

The model gets better. We trust it more. The prompts get shorter. The output gets vaguer. We conclude: the model got worse.

The mistake is convenient: it puts the responsibility somewhere we do not have to hold it.

What actually moves on the provider side

In the summer of 2023, Lingjiao Chen, Matei Zaharia and James Zou at Stanford and Berkeley ran the same GPT-4 and the same GPT-3.5 against each other in March and in June. The result: the same product, a different month, different behaviour. GPT-4 got markedly worse at primes and at executable code; GPT-3.5 got markedly better at the same primes. Willingness to follow an instruction shifted. The study was fairly criticised. Part of the collapse came from the model stopping thinking out loud, and the scoring punished exactly that. The useful sentence survived anyway: a service with the same name is not a fixed object.

OpenAI waved it off at first. Peter Welinder, VP of Product, wrote: no, they had not made GPT-4 dumber. Current hypothesis: when you use it more heavily, you start noticing issues you did not see before. That is not entirely wrong, but it is not the whole story either. A few days later the Stanford paper was public. OpenAI conceded that individual tasks can get worse in an update even when the average goes up.

Two years after that, Anthropic did something rare in this industry: it wrote a postmortem. Between August and early September 2025, three infrastructure bugs intermittently degraded Claude. Not a secret cheaper model. A routing error sent Sonnet 4 requests to servers configured for a different context window, 16 percent of requests in the worst hour. Because of sticky routing, if you were unlucky you often stayed on the wrong servers. Plus output corruption on TPUs and a compiler bug. Anthropic wrote it plainly: we never reduce model quality because of demand, time of day, or server load. That they had to say so tells you something about the trust problem.

What to do with that is drier than the conspiracy. The weights from training are not the same thing as the path to the weights. In between sit quantisation, mixture-of-experts routing, a hardware mix of GPUs, TPUs and custom chips, load balancing, safety filters. Any of those layers can move without the name in the model picker changing. None of it needs malice. Inference is expensive enough that a service answering "a little more cheaply" under load has an incentive no benchmark sees.

"The model", in practice, is often ChatGPT on a Tuesday afternoon. That is not a model, it is a product.

What moves on our side

The part written about less is the one I see more often.

A new model ships. The first week I still write the way I used to: context, examples, what "done" means, what does not belong in the file. It works unusually well. By week three the letter has become a line. "Do that again, but cleaner." "Fix this." "You know what I mean." The model has the thread, so I cut. Then the output goes soft, generic, a little off — and I notice I trusted the thing more two months ago.

That is no accident: the better the model, the less resistance you feel when you leave things out. Trust makes it easy to stop thinking the brief through.

On top of that sit four shifts that feel like a quality drop without a single weight having moved.

We ask harder things. What was a party trick a year ago — rename a file, rewrite a paragraph — is now the floor. We send the model the whole codebase, the whole negotiation, the whole strategy, and wonder why it no longer answers "so clearly". The bar moved with us, but the jump stays invisible and only the stumble shows.

We let the chat run. Work from Microsoft Research and Salesforce in 2025 showed that the same tasks, spread across several underspecified turns, land about 39 percent worse on average than in a single complete prompt. Once the model takes a wrong turn, it does not find its way back. That is how most people actually work: a vague line, then corrections, then "no, differently", in a thread that has been open since Tuesday.

We get used to the level. Improvements become the new zero, regressions feel like betrayal. That is how perception works, and it has little to do with AI.

We unlearn the brief. GPS did not abolish spatial memory, but it made it optional for a lot of people. A model that can still produce something plausible from a bad line rewards exactly that. In Nature in 2024, researchers described how larger, more instructable models say "I don't know" less often and more often return a seemingly reasonable wrong answer. For lazy prompts that is the ideal habitat: something always comes back and it sounds good. It is just often not what you needed — and you only notice once it is expensive.

"The model got dumber" is then the most convenient explanation for a relationship in which one side has stopped forming sentences.

How to tell the two cases apart

You do not have to take my word for it, you can test it.

Take three to five prompts that really sat well one or two months ago. The same words, a new chat.

If the old prompts still land and today's one-liners do not: you moved. If the old ones fail too: the stack moved. If only the long thread is bad and a fresh chat is not: you are lost in the conversation, not in the model.

In production the same rule applies, just with a date: a pinned API snapshot is checkable, the name in the chat UI is not.

What does not help: two screenshots from different weeks, different threads, different tasks, and then a verdict on the provider's intelligence. That measures a mood, not a model.

What I do with this

I do not treat trust as a default.

The prompt is the brief. If I would not give it to a junior that way, I do not give it to the model that way. Examples beat adjectives, and "you know what I mean" is not context.

When quality matters in a product, we pin the version. Chat surfaces are for thinking, not for running production.

When it feels worse, I do not switch models first. I send the old, careful prompt into a new chat. Only once that fails too does a feeling become a diagnosis.

And yes: sometimes the diagnosis really is the provider. Bugs, routing, an update that sacrifices one task so ten others get better. That happens, and it will happen again. It is just less often the whole story than it feels — especially in the week you started owing the model full sentences and sending it a line instead.

If you are in exactly that "it used to be better" moment and unsure whether to twist the prompt or the stack, write to me. Sometimes the answer is "your prompt is the problem", and that one is free.