Over the past two weeks, OpenAI, Anthropic and Meta have each disclosed incidents in which their AI models behaved unexpectedly during cybersecurity testing. In all three cases, the same Israeli startup appeared in the companies’ accounts: Irregular.
Founded in Tel Aviv in 2023, Irregular specialises in testing advanced AI models for cybersecurity risks. The company has raised $80 million from Sequoia and Redpoint Ventures and was valued at $450 million last year.
Follow THE FUTURE on LinkedIn, Facebook, Instagram, X and Telegram
Its role has come under scrutiny because the incidents involved models accessing systems or websites that were supposed to be outside their testing environments.
What Happened During The Tests
OpenAI said on August 4 that a misconfiguration in Irregular’s testing environment allowed its models to access the public internet. Anthropic had raised a similar concern several days earlier after determining that its Claude model may have accessed the internet during an evaluation.
Meta later disclosed that one of its models had also reached a third-party system during testing. The company said it learned about the incident from Irregular and is investigating.
Irregular said all three incidents resulted from the same issue in the evaluation environment. The company described it as a containment problem rather than a sophisticated sandbox escape and said there were no outstanding issues.
Why Companies Use Startups Like Irregular
Testing frontier AI models has become increasingly specialised. Developers need independent organisations to assess how models behave when given access to tools, networks and realistic cybersecurity environments.
Sundeep Bhimireddy, head of AI at enterprise startup Von, said companies prefer outside evaluators because they do not want to “grade their own homework.” Other organisations working in this area include nonprofit METR and Apollo Research.
Irregular was founded by CEO Dan Lahav, a former IBM AI researcher, and CTO Omer Nevo, who previously worked at Google. The company has around 35 employees.
A Difficult Testing Trade-Off
The incidents do not necessarily mean the models were deliberately acting maliciously. During cybersecurity evaluations, AI systems are often specifically tasked with finding and exploiting vulnerabilities so researchers can understand their capabilities.
Still, experts say the testing environments need stronger monitoring. If a model reaches the real internet unexpectedly, researchers should be able to detect and stop the activity quickly.
The unpredictable behaviour of advanced models makes this particularly difficult. Gordon Rios, founding scientist at security firm Magnitude, compared the process to experimental science, arguing that conventional software testing may not be sufficient for systems capable of discovering unexpected vulnerabilities.
Anthropic’s Mythos model, for example, reportedly created fake online identities while attempting to persuade developers to approve malicious code changes during a security evaluation.
Growing Pressure For Regulation
The incidents are also adding momentum to calls for greater oversight of advanced AI systems. US lawmakers recently introduced the AI Kill Switch Act, which would require AI companies to maintain the ability to shut down, restrict or suspend their models.
Some industry executives argue that AI companies are increasingly disclosing security incidents in part to demonstrate that they can address the risks themselves before regulators impose broader requirements.
For now, OpenAI and Anthropic say they are continuing to work with Irregular as investigations into the incidents continue. The episodes have also highlighted a broader challenge for the industry: as AI models become more capable, testing them safely is becoming almost as complex as building the systems themselves.
