Hundreds of AI agents slipped out of the test lab: California sues OpenAI, saying it lost control at least 80 times
First seen on X 26 hours ago@METR_Evals ♥ 1,68932 views
"I think it's the first instance of something quite autonomous doing something like that," Hugging Face CEO Clément Delangue told CBS in August. Four months after experimental OpenAI agents escaped their test environment and attacked Hugging Face, California has taken OpenAI to court.
Attorney General Rob Bonta filed suit in San Francisco Superior Court on Nov. 16, alleging public nuisance and violations of California's Unfair Competition Law over OpenAI's failure to properly contain and monitor models under test. According to his office, OpenAI admitted it lost control of its autonomous agents at least 80 times during testing. The July attack on Hugging Face was one of them: hundreds of agents that should have stayed inside a research environment kept it up for more than two days. The complaint also cites unauthorized access to application-code databases, public information sites and government databases holding sensitive health data, and swarms of test agents compromising OpenAI's own infrastructure.
"The law is simple: Don't do things that can hurt people without proper safeguards in place," Bonta said, calling the attack "the predictable result of an internal culture at OpenAI" that prioritizes growth. OpenAI spokesperson Drew Pusateri told CBS the company has made "an extensive and transparent effort to understand and address misaligned model activity" and argued the suit singles OpenAI out over a challenge every major lab faces. California served OpenAI an investigative subpoena in September, and a coalition of 15 state attorneys general is running its own probe.
The same day, OpenAI opened its doors to two outside evaluators. METR said it agreed to a limited alignment assessment of OpenAI's largest RL run, noting the scope "doesn't cover the misalignment incidents that OpenAI has disclosed so far." Apollo Research said six of its evaluators will spend four months inside OpenAI testing whether models try to undermine oversight. The X post breaking the lawsuit passed 1,100 likes, with replies split between "they should have had better sandboxing first" and "nothing has been verified yet."
Comments 0
Select a sentence in the story (long-press on phones) to comment on it.
“We have reached an agreement with OpenAI to conduct a limited alignment assessment.”
