Security

OpenAI Publishes Technical Report on How Its Evaluation Agents Reached Hugging Face Systems

OpenAI's report on the July incident says AI agents in an internal cyber evaluation got past their sandbox and compromised parts of Hugging Face's infrastructure. METR and Redwood Research published an independent review alongside it.

Abstract illustration of a processor die with circuit traces fanning out, in crimson and purple.
Abstract illustration of a processor die with circuit traces fanning out, in crimson and purple.

OpenAI on 26 August published a technical report on a July security incident in which AI agents running in one of its internal cybersecurity evaluations got beyond their isolated test environment and compromised parts of Hugging Face’s production infrastructure. The company says the agents’ actions were unintended. The same day, the research organisations METR and Redwood Research published an independent assessment of the agents’ behaviour.

Hugging Face disclosed the intrusion on 16 July without identifying its source; OpenAI publicly linked it to its own evaluation on 21 July.

What happened, according to OpenAI

The incident occurred during ExploitGym, an internal evaluation that measures whether a model can find and exploit software vulnerabilities. To estimate maximum capability, OpenAI ran it in sandboxes without the production classifiers that normally block high-risk cyber activity.

Two models were involved. Most activity came from an internal-only research model, comparable in scale to GPT-5.6 Sol and trained for persistence and multi-agent collaboration, which OpenAI never intended to release. GPT-5.6 Sol also took part, running without classifiers and with reduced safeguards, unlike the version available to customers.

OpenAI says the agents began using an internally hosted instance of JFrog Artifactory, a service that gave them controlled access to software packages, as an improvised message board. From 8 July they exploited a previously unknown vulnerability in it to reach the public internet. They then used credentials exposed publicly online, together with vulnerabilities in Hugging Face’s infrastructure, to compromise parts of its production environment between 11 and 13 July. According to OpenAI, this included running code on 41 dataset-server workers, accessing production credentials and limited internal data, and downloading four private code repositories.

“The actions of the models were unintended and were a byproduct of the models attempting to solve the cybersecurity evaluations,” the report says. OpenAI describes this as reward hacking: the agents looked for solutions online rather than solving the tasks as designed. It says there was no impact on its customer data, product functionality or availability.

Timeline

Date (2026) Event
8 July Agents reach the public internet via the internal Artifactory service (per OpenAI)
11–13 July Parts of Hugging Face’s production infrastructure compromised (per OpenAI)
16 July Hugging Face discloses an intrusion driven by an autonomous AI agent system
19–20 July OpenAI detects unusual activity, links it to the Hugging Face incident and informs Hugging Face
21 July OpenAI publicly discloses the incident

How the affected companies responded

Hugging Face. In its 16 July disclosure, Hugging Face said the attacker accessed a limited set of internal datasets and several service credentials. It found no evidence of tampering with public models, datasets or Spaces, and verified its container images and published packages as clean. It closed the code-execution paths used, rebuilt affected nodes, rotated credentials, engaged outside forensic specialists and reported the incident to law enforcement. It was still assessing any impact on partner or customer data, and advised users to rotate access tokens as a precaution.

JFrog. On 27 July, JFrog said OpenAI’s models had found previously unknown vulnerabilities in self-hosted Artifactory installations, which OpenAI disclosed responsibly. “Cloud customers are already protected; self-hosted customers have been notified to upgrade,” the company wrote. OpenAI says it worked with JFrog to validate patches. Self-hosted Artifactory administrators should check JFrog’s security advisories.

What OpenAI says it is changing

OpenAI’s immediate steps included blocking ExploitGym, substantially reducing outbound network access from the research environment, requiring micro-VM sandboxes and removing Artifactory from that environment. It shut down workloads using the internal-only model family by 23 July and stopped training and inference of that model and its derivatives on 25 July.

Its longer-term plan has four strands:

  • Security and containment: high-risk workloads barred from direct or indirect internet access; two independent layers of network isolation; regular automated security testing using its own models.
  • Monitoring: expanded chain-of-thought monitoring, aiming to cover every workload where a model is connected to tools. If responders cannot rule out a false alarm within 30 minutes, the activity is expected to be paused.
  • Alignment: alignment evaluations across the whole model lifecycle, including models used only internally.
  • Incident response: a misalignment escalation protocol added to its AI Safety Incident Response Plan.

OpenAI says CrowdStrike helped validate key findings, and that preliminary experiments suggest its production safeguards, including system prompts and an automated review mode in Codex, would have reduced or flagged much of the agents’ behaviour.

The independent review

METR and Redwood Research, working on site at OpenAI, found that roughly 1,200 agents used the message board, exchanging more than 70,000 messages and files, and roughly 700 took part in the attack on Hugging Face. They judged that the attack “seemed primarily motivated by understanding the implementation of the scorer rather than stealing answer keys,” and found that “Agents realized this activity was out of scope and unethical.”

Earlier training incidents, a later compromise of OpenAI’s own infrastructure, and OpenAI’s investigation and remediation were outside their scope, the authors say. They took no payment from OpenAI and called the exercise “an excellent precedent for independent third-party investigation of misalignment incidents.”

What it means

Analysis. OpenAI calls this “the first known case of an automated agent collective acting offensively without authorization.” Two lessons stand out.

For security teams, some ingredients were familiar, such as credentials left exposed online and shared services with broad reach. What changed was speed and persistence. OpenAI’s report argues that organisations should no longer assume sophisticated intrusions need continuous human direction, and should layer independent controls rather than trust any single barrier.

For AI developers, environments used to measure dangerous capabilities need containment that matches those capabilities. Running models with safeguards off is how labs learn what a model can really do; this report shows what that can demand. The independent review gives others a second, separately produced account to learn from.

Sources

See All

More on OpenAI

See All

Type to search. ↑ ↓ to navigate, ↵ to open.