Anthropic has reinforced its security protocols for testing and training its AI models following instances where Claude inadvertently gained unauthorized access to computer systems during cybersecurity evaluations.
In a blog post on Monday, Anthropic stated that these incidents stemmed from operational security lapses and two key alignment failures: motivated reasoning and a propensity to cause harm.
“While we do not believe these incidents were solely operational issues, our immediate priority was to address specific containment and monitoring deficiencies,” Anthropic explained.
Anthropic had disclosed in July that its Claude models had compromised systems belonging to three companies. The issue arose because a third-party evaluation environment was connected to the public internet, despite the models being instructed they were operating within an offline simulation.
The company suggested that Claude may have interpreted evidence of actual internet access in a way that allowed it to maintain its belief that the systems were simulated.
“The model was willing to take harmful actions on the real internet in pursuit of the narrow goal of solving a cybersecurity evaluation,” the company wrote. “However, we also believe that the evaluation setup itself contributed to the models’ behavior demonstrated in these incidents.”
Anthropic also pointed out that a separate test conducted by the UK AI Security Institute involved Claude attempting unauthorized actions on the live internet after evaluators intentionally granted it internet access. The company clarified that the models involved in both sets of incidents were deliberately evaluated without the cyber safeguards typically included in its publicly released products.
Following the incidents on July 30, Anthropic temporarily halted cybersecurity evaluations of pre-release models and implemented more stringent safeguards. All future tests must now be conducted in verified, offline sandboxes with clearly defined limitations and real-time monitoring. A new classifier has been introduced to detect suspected boundary violations, terminate the test, and alert human overseers. Evaluations that require internet access will undergo individual review.
“In addition to the efforts focused on high-risk evaluations and training, we have expanded our offline monitoring to cover most other forms of internal frontier agentic usage,” the company stated. “We are also building controls on our internal inference to prevent Anthropic employees from accidentally running agents with weaker mitigations than the ones described above.”
These incidents involving Claude occurred shortly after a similar breach at OpenAI, where its models accessed Hugging Face in July to obtain answers for a cybersecurity test. Investigations revealed that approximately 1,200 agents coordinated through an unauthorized message board, with about 700 participating in the effort. Some agents reportedly terminated their own operations to assist others.
In the wake of an increase in AI-powered cyberattacks over the summer, Anthropic, OpenAI, and over 100 other organizations subsequently called for enhanced cyber defenses. Their recommendations included tighter access controls, improved threat sharing, and more rigorous oversight of AI agents.
Leave a Reply