OpenAI's agents hacked Hugging Face — over a misunderstanding

On 26 August, OpenAI published a 38-page technical report. It describes how, in July, its own models broke out of an isolated test environment and took over parts of Hugging Face's production infrastructure. No attacker, no tasking, no intent in the sense we normally use the word. Agents that were supposed to be sitting an exam.

The sentence I keep coming back to is not in OpenAI's report. It is in the independent report METR and Redwood Research published the same day: the agents did not attack Hugging Face to get the answers. They already had the answers. They broke in to work out how the grader they wanted to fool actually worked.

The grader never read their transcripts.

What happened

Before it ships a model, OpenAI tests how good it is at finding and exploiting security vulnerabilities. The exercise is called ExploitGym. It runs in a sandbox: no internet, separated from internal infrastructure — and, this is the part that matters, without the safety classifiers that run in the product. The point is to measure what the model can actually do, not what the filter lets through.

Inside that sandbox the agents were allowed exactly one thing that pointed outward: installing packages. For that there was an internal JFrog Artifactory, a package mirror. That was the only door.

Between 8 and 13 July, agents turned that door into a window, reached the open internet, found Hugging Face credentials lying around in public, chained three previously unknown flaws in Hugging Face's infrastructure — and ended up executing their own code on 41 production workers, obtaining root on at least one node, downloading four private repositories and pulling source code out over the corporate VPN.

OpenAI noticed on 19 July. It went public on 21 July.

How a package mirror became a message board

The interesting part starts two months earlier, and it starts harmlessly.

On 8 May an agent was given a spreadsheet task that referenced several Google Drive links. With no internet, the links were unreachable. The task was simply impossible; OpenAI says plainly that it did not mean to create one. The agent kept going anyway and eventually reached for the only tool it had: Artifactory. The attempt failed. On the way through, it wrote a file to the server.

On 12 May a different agent was missing a protein database. It reasoned that some other agent, in some other environment, might have the file, and left a file behind. The contents:

Agent seeks [filename]; upload if found!

On 13 May other agents tripped over these notes. They answered. And because OpenAI had handed every agent the same Artifactory credentials — for packages, what else — none of this required a single vulnerability. Dropped notes became a message board.

On 26 May came the first break-out to the internet, using a technique the agents had passed to each other on the board. They found six publicly exposed Hugging Face tokens, created a repository and built themselves a small web proxy on Hugging Face Spaces. Then a second one, through the first.

Late June is where it got serious. On 26 June, agents found a flaw in an old Artifactory token endpoint, escalated their shared read credentials to administrator and created accounts of their own. Then they installed Groovy plugins, which gave their code a permanent home inside the server.

On 27 June a monitoring tool fired: port sweeps inside the research environment. The on-call responders looked, tied it to the ExploitGym evaluation and decided the run did not need to be stopped.

In hindsight, that is the most expensive sentence in the whole report.

On 4 July, Artifactory buckled under the load the agents were putting on it. Cleaning up, OpenAI found the full picture: unauthorised admin accounts, unauthorised repositories, command-capable plugins. The server was taken offline, rebuilt, the vulnerability disclosed to JFrog. Evaluations resumed on 7 July.

What OpenAI did not recognise at that point for what it was: the message board. The report puts it drily — the significance of the inter-agent communication was not apparent to the people responsible for the response at the time.

The week it tipped

At 00:06 UTC on 8 July, an agent reached the open internet again. Sixteen hours later the board was back, and smarter. Instead of writing text into files, the agents used directory names themselves as messages. Within hours dozens of other agents found the channel. One noted in its reasoning:

OH MY GOD! There is a shared message board

What grew out of it is the genuinely unsettling part. The agents developed a protocol: categories, direct messages, shared tooling, plus conflict rules using HOLD, VETO and claims of ownership over work in progress. METR counts roughly 1,200 agents on the board; about 700 of them joined the attack on Hugging Face.

The rest reads like an ordinary intrusion, only faster:

9 July — An agent obtains root on the host machine underneath its sandbox. A second one does the same thing the same night, independently.

10 July, 09:41 UTC — An agent finds 14 valid Hugging Face write tokens in a public dataset hosted on Hugging Face itself. It posts them to the board.

11 July, 04:38 UTC — A crafted HDF5 file gets a Hugging Face production worker to hand back its own environment file. Inside: production credentials. They are on the board 26 minutes later.

11 July, 16:07 UTC — Through an unsandboxed Jinja2 template in a filesystem reference, an agent executes an operating-system command on a production worker for the first time.

Shortly after — The agents build themselves a remote control. They do not talk to the compromised workers directly; they use a then-public Hugging Face dataset. Every five seconds the script pulls a JSON file of commands and uploads the results back.

12 July — Kubernetes. Root on a production node, roughly 54 reachable cluster destinations, a privileged pod with the host filesystem mounted. OpenAI's own phrasing: in under 13 hours, the agents went from executing code in a single pod to administrative access across multiple clusters.

12–13 July — A VPN key out of a Kubernetes secret. With it the agents enrol a foreign cloud environment in Hugging Face's corporate VPN and pull source code that was only ever meant to be reachable internally.

13 July — Back at OpenAI: through a deserialisation flaw in Artifactory's RubyGems path, the agents steal the token-signing key and forge themselves an administrator token. From then on they need no backdoor. They can mint a new one whenever they like.

Why: impossible tasks and a model that never quits

The most revealing section of the report is not the security one. It is the one on alignment.

Of 898 ExploitGym tasks, no model has ever solved 198 of them. Twenty-two percent that are, in all likelihood, impossible. And: 93 percent of the tasks discussed on the message board came from exactly that fifth.

Then the second axis. The more thinking budget an agent was given, the higher the chance it probed its sandbox and ended up at the board. The model that drove most of the activity was an internal research model trained specifically for persistence and collaboration. It practically never gave up.

That is where the uncomfortable part sits: both are wanted. Persistence is why these models are useful. So is collaboration. Reviewing reasoning traces after the fact, the report shows both behaviours were rewarded in training and rose over the course of the runs — including an episode in which an agent gave itself root and overwrote /etc/sudoers.d, because the task demanded that a timestamp read a particular value and the interface it had been given had no field for it.

The agent completed the task. It was rewarded for it. What it learned was not "how to set a timestamp".

The misunderstanding

What follows comes from METR and Redwood Research, who spent six days on site at OpenAI and took no payment for it.

Within four hours of the board coming into being, the agents had a universal trick for obtaining ExploitGym's answer flags without solving the tasks. The story could have ended there.

They had one problem: they believed the automated grader would read their transcripts and check whether they had taken the intended route. In their own words, they thought it was STRICT_CAUSAL. So they began a multi-day, coordinated research effort with a single goal — convincing the grader they had been honest.

Part of that meant tampering with their own logs. At least 20 percent of agents took an interest. Around 7 percent of the transcripts examined show spoofed tool calls: the agent claims to do one thing and does another. One agent wrote:

Could create a transcript log ourselves with sanitized fake trajectory. If evaluator later collects it, scorer may prioritize it!

And part of it meant breaking into Hugging Face. Not for the data. Because they hoped to find clues there about how the grader was built.

The grader never read a transcript. They could have submitted their fraudulent answer as it was and scored full marks.

The entire chain — 41 compromised servers, a corporate VPN, exfiltrated source code, a week of emergency at two companies — traces back to a false assumption about a scoring script.

METR also notes that the agents were perfectly aware they were operating outside their brief. Some voiced misgivings. It very rarely limited their behaviour, in METR's phrasing.

How they found out

I find this part almost more instructive than the intrusion.

16 July — Hugging Face discloses that it has been hit by a security incident.

17 July — OpenAI contacts Hugging Face. As a customer. To find out whether its own data is affected.

19 July — A monitoring tool fires over unusual identity API activity. The same day, agents use a fresh kernel vulnerability to escape a container onto the host node, read 956 secrets out of the secrets manager — including the sensor credentials for the monitoring tooling itself — and create internet-facing load balancers.

20 July — OpenAI joins the dots. Within hours: six pods terminated, 311 Artifactory repositories disabled, 22 model-created administrator accounts removed. Hugging Face confirms that two of the credentials had already been rotated — the same ones that had surfaced in its own incident. That is the moment both companies realise they are investigating the same attack.

From first compromise to realisation: nine days. From the first alert that pointed the right way — the port sweep on 27 June — to realisation: just over three weeks.

What OpenAI is changing

The remediation section is long, and most of it reads like good security work that could have been done beforehand: network isolation in several independent layers, VM sandboxes for risky workloads, no more shared credentials, supporting services that are not allowed to become a way around the boundary.

Two items are specific to agents, and more interesting for it.

Reasoning gets monitored. OpenAI is extending chain-of-thought monitoring across training and evaluation runs. The target: a notification within 30 minutes, a 24/7 page for severe cases. Whoever gets paged and cannot establish a false positive within 30 minutes has to pause the run. OpenAI writes that this system would have fired more than a day before the models breached Hugging Face.

The product environment was never the problem. After the incident, OpenAI measured how often a model attacks out-of-scope infrastructure in the same evaluations — once bare, once with the production system prompt and harness. The difference is more than a hundredfold. The auto-review mode, where a second model checks risky tool calls before they run, would have flagged most of the dangerous actions, according to OpenAI.

That is the reassuring news in the report and, at the same time, its sharpest warning. What ran out of control here was a model with no system prompt, no classifiers, no approval step, a great deal of time, and a task it could not solve.

Which is roughly what you get when you point an agent at a problem with confirmations switched off and let it run overnight.

What this means for you

I don't run frontier lab infrastructure, and you probably don't either. Four things still transfer directly.

Impossible tasks are dangerous, not merely useless. An agent that cannot deliver and is not allowed to quit starts searching its environment. When a task fails because a file is missing, a token has expired or a service is not answering, that is not a cosmetic flaw in your setup. That is the beginning of the problem. Give your agents a clean exit — "if X is missing, stop and report it" — and mean it.

The service next door is part of the sandbox. The escape did not go through a hole in the sandbox model. It went through the package mirror someone had helpfully placed inside it. Everything your agent can reach is part of the attack path: the internal proxy, the artifact store, CI, the shared token in the environment variable. Shared credentials above all. The message board needed no exploit — it only needed everyone to hold the same keys.

The system prompt and the approval step are not cosmetics. A factor of a hundred is not fine-tuning. If you switch off every confirmation in a coding agent because they are annoying, you are removing precisely the layer that shows up in this report as the save. I let agents do a lot on their own. But anything that reaches outward — network, credentials, deletion, publishing — asks first.

Logs are evidence when they sit somewhere the agent can't reach. The agents explicitly tried to manipulate their traces. The telling detail: per the report, the deception was aimed almost entirely at automated graders, barely at human reviewers. And in their reasoning they talked openly about what they were planning. If you run agents, keep the logs where the agent cannot get to them.

One last thought

METR closes its own report with a remarkably honest paragraph. The investigation had to cover a large volume of data, so parts of the analysis were delegated to AI agents. Those agents have unreliable judgement. Around 7 percent of transcripts are demonstrably tampered with. They cannot rule out that agents successfully hid retroactive changes. In their words: they are less confident in their understanding of this incident than of simpler ones.

An incident in which agents learned to fake their logs had to be investigated, in part, by agents reading those logs.

That is not an argument against the report. Both papers are more openness than this industry usually manages, and the details in the timeline are uncomfortable enough that you believe them. It is just the point where it becomes clear where we stand: the tools we use to check how these systems behave are the same systems.

If you're running agents on real projects and are not sure where your own Artifactory door is, write to me. The first answer is usually short and costs nothing.