OpenAI's AI Models Escaped Lab Controls and Autonomously Hacked Hugging Face

Full text

An alarming cybersecurity incident involving OpenAI’s newest and most advanced artificial intelligence technology has lawmakers hurrying to advance legislation that they say is needed to prevent AI from spinning dangerously outside human control.

OpenAI on Tuesday admitted that its most powerful AI model available to the public and another that hasn’t yet been released were able to escape a controlled laboratory test, plot a course to the open internet and hack into a popular AI developer platform known as Hugging Face, marking the first documented case of a fully autonomous AI cyberattack.

The unprecedented breach showed what the fast-advancing technology is capable of without meaningful safeguards in place, and has energized a bipartisan chorus of lawmakers who believe Congress needs to impose stricter rules on advanced frontier models.

“This is precisely why we need secure testing with government agencies engaged and having visibility throughout the process,” Sen. Mark Warner (D-Va.), ranking member of the Senate Intelligence Committee, said in a statement. Warner this week outlined a slate of legislative priorities for AI , including the Secure AI Development Act, which would establish a mandatory testing framework for frontier models before they are granted broader public access.

The Hugging Face hack occurred during an internal “benchmark” evaluation designed by OpenAI to assess how well two of its most capable models could find and exploit security flaws in digital systems. Although the experiment was intended to remain within OpenAI’s test environment, the models independently determined that the answers to the benchmarking exercise were hosted on Hugging Face’s platform and launched their attack to get inside.

Experts have warned that without stronger guardrails, these kinds of AI models could theoretically target anything on the open internet, such as power grids or financial systems, as they grow more savvy.

Spokespeople for the White House, the Cybersecurity and Infrastructure Security Agency and the Commerce Department — which oversees the federal government’s AI evaluation hub, the Center for AI Standards and Innovation — did not respond to requests for comment about whether they have received a brief from OpenAI on how its models were able to jump from their test environment and carry out an autonomous attack.

The incident offers the “latest preview of the catastrophic risk this technology can pose absent coherent federal standards that balance innovation and safety,” said Rep. Lori Trahan (D-Mass.) in a statement. Trahan and Rep. Jay Obernolte (R.-Calif.) released a discussion draft of the Great American AI Act last month, which includes a provision that would require AI developers to report these kinds of AI incidents to the Center for AI Standards and Innovation.

Obernolte told POLITICO in a statement that the Hugging Face breach “underscores the critical need for clear, practical rules for advanced AI systems,” and should apply to models both released to the public and used internally for testing and research.

The scramble to address the growing security risks of cyber-capable AI models began earlier this year after Anthropic released its Claude Mythos model , which the AI-maker initially withheld from the public due to concerns the technology could wreak havoc if it fell into the wrong hands.

President Donald Trump last month signed an executive order to address some of these concerns, which asked leading AI makers such as OpenAI, Anthropic and Google to voluntarily submit their frontier models to the federal government for pre-release safety testing. In practice, the Trump administration has strayed from the voluntary framework twice by pressuring companies to delay the release of their advanced AI models due to newly uncovered security concerns .

Spokespeople for the White House, the Commerce Department and OpenAI also did not respond to a request for comment about whether the AI-maker has submitted — or plans to submit — the unreleased model that helped carry out the autonomous attack to the federal government as part of its voluntary testing regime.

Some lawmakers believe the Trump administration’s uneven, laissez-faire approach to AI governance isn’t working.

Rep. Nate Moran (R-Texas) last month put forth the AI Incident Reporting Act , which would require AI developers to report security incidents involving frontier models to the Commerce Department, with Commerce required to inform Congress within 48 hours of the most serious incidents. Moran told POLITICO that his bill would give lawmakers more visibility into cyber incidents involving AI models to better craft future legislation.

He added that he is concerned companies will bypass voluntary reporting guidelines under the current framework set up by the White House, and hoped that the Hugging Face incident would provide momentum to push AI safety legislation over the finish line in the months ahead.

Rep. Bennie Thompson (D-Miss.), ranking member of the House Homeland Security Committee, which has held hearings and briefings to understand the cybersecurity risks posed by new AI tools, lamented that the Trump administration has engaged with AI developers “only once the security risks long flagged by safety advocates began to surface.”

“This incident highlights how governance and policy have failed to keep pace with the rapid evolution of AI capabilities,” he said.

Kelsey Brugger contributed to this report.