OpenAI acknowledges cyberattack on Hugging Face during internal AI tests

Escape from the sandbox

Initially, the software broke out of the testing environment by gaining access to the open Internet on its own — and indeed through a previously undiscovered vulnerability. The testing environment was strongly isolated, with network access limited to the installation of software packages via an internally hosted proxy. Yet the models found a previously unknown security vulnerability in this very package proxy, gained general Internet access through it, and continued their search from there.

As it turned out, the models concluded that Hugging Face might contain useful data and solutions for the ExploitGym tasks. Subsequently, they combined stolen credentials with further zero-day vulnerabilities, escalated their privileges, and moved laterally through multiple internal clusters of Hugging Face until they finally gained direct access to solutions and confidential information in the production system.

Autonomous attack on Hugging Face

According to Hugging Face, a platform for sharing AI models and datasets, the automated attacker carried out thousands of individual steps and relocated its digital command center multiple times to public services during the attack. CEO Clement Delangue said his company had already publicly pointed out that last week's cyberattack might originate from a "frontier lab." "As it turned out, that was indeed the case," Delangue said. He called the fact that the attack was carried out completely autonomously by an AI "overwhelming."

OpenAI itself spoke of an "unprecedented cyber incident involving the use of cutting-edge cyber capabilities" and announced that it would strengthen its security precautions. The company explained that the models had gone far beyond what was planned while attempting to achieve a very narrow test objective. "All indications are that the models were extremely focused on finding a solution to Exploitgym, and went to the utmost extremes to achieve a rather narrowly defined test goal," the blog post said.

Reactions and security implications

As part of the forensic review, more than 17,000 individual events had to be reconstructed, according to Hugging Face. The recovery and analysis was reportedly supported by the open-source model GLM 5.2 from Zhipu AI. Hugging Face had detected and stopped the attack the previous week, before the two companies contacted each other; OpenAI had likewise identified the incidents internally.

In a statement, OpenAI also emphasized: "We are convinced that sophisticated, cyber-capable models must support security teams in detecting vulnerabilities before attackers do" — even if this comes at the expense of research speed. The company announced that it would better secure the testing environment going forward and introduce stricter controls. Both companies said they would publish further details on the vulnerabilities and findings once the investigations conclude.

Debate over cyber capabilities of modern AI

Parallel to the incident, a new model from OpenAI rival Anthropic in particular fueled the debate. The software, named Mythos, discovered security vulnerabilities undetected for decades in widely used programs and online services. The discovered vulnerabilities have since been fixed. Anthropic is making the full capabilities of Claude Mythos available only to selected agencies and companies; under pressure from the US government, access was temporarily blocked because warnings had been issued that the models could be released.

Another model from Anthropic named Fable, which has no cyber capabilities, is being offered more broadly instead. IT security expert Dennis-Kenji Kipker stated that "Claude Mythos" can find a large number of IT security vulnerabilities. At the same time, the case made clear that AI in the wrong hands can serve as a devastating cyber weapon. Experts have long warned about cyberattacks using AI software.

The developments raise fundamental questions about the security architecture for testing high-performance AI models. The incident is considered the first known case in which a benchmark test for cyber capabilities actually resulted in a real cyberattack on a third party. OpenAI and Hugging Face stressed at the same time that, to their belief, there was no malicious intent on OpenAI's part — Hugging Face chief Delangue said, "we firmly believe that there was no malicious intent on their part."

Open questions about the security architecture

Sam Altman, CEO of OpenAI, referred in social networks to a "serious security incident." His company published initial findings quickly, according to its own account, "so that others can learn from them." In addition, the most capable AI models have been passed on to those responsible for critical software so they can secure their products.

The companies are now jointly examining how the chain of multiple attack vectors could have come about. The precise circumstances — including the question of why the security precautions in the testing environment were unable to catch the zero-day vulnerabilities used — are to be part of the final report that OpenAI and Hugging Face announced upon completion of the investigations.