Governments Must Act on AI Safety, Anthropic CEO Urges
Dario Amodei, CEO of AI firm Anthropic, asserted on Wednesday that governments can no longer afford to merely study the issue of artificial intelligence regulation. He called for the United States to implement binding safety requirements for its most powerful AI models.
In an essay titled “Policy on the AI Exponential,” Amodei argued that current transparency mandates are insufficient and advocated for mandatory regulation of “frontier AI” systems.
“AI is advancing at a lightning pace—in only four years, AI models have gone from barely being able to write a coherent line of code to writing most of the code at major AI companies,” Amodei wrote.
Amodei’s essay coincides with Anthropic’s expansion of access to Claude Mythos, its restricted frontier AI model designed for cybersecurity organizations and government partners. The company launched Mythos 5 on Tuesday. Researchers, including those at the UK’s AI Security Institute, have reportedly found that the model can autonomously execute complex cyber attacks.
Amodei stated that his regulatory proposal draws inspiration from the framework used by the Federal Aviation Administration.
“Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety,” he wrote. “I am grateful to see the Trump administration’s Executive Order move incrementally towards a greater role for government in AI, though Anthropic’s proposal recommends even further action.”
A Proposed Regulatory Framework
Under Amodei’s proposed framework, advanced AI models would undergo mandatory third-party testing. The government would be empowered to halt unsafe deployments and require companies to secure model weights, conduct safety assessments, and report significant incidents. He also urged governments to prepare for AI-driven job displacement and advancements in drug development, to limit the use of surveillance and autonomous weapons in domestic law enforcement, and to enhance cooperation among democratic nations on critical AI technologies.
“First, enduring job displacement is undesirable and dangerous, and we should do everything we can to minimize or prevent it, not to bring it about,” Amodei wrote, referencing past warnings he has issued regarding job displacement. “Second, any response to AI-driven job displacement needs to address both the need to provide for everyone economically, and the need for people to find meaning, purpose, and agency.”
Public Release and Industry Scrutiny
The essay was released in the same week Anthropic launched Claude Fable 5, a public-facing version of Claude Mythos 5. This version redirects certain requests related to cybersecurity, biology, chemistry, and AI development to the less capable Claude Opus 4.8 as a safeguard against misuse. However, the launch has drawn criticism from developers and researchers. Concerns have been raised about Fable’s higher token usage, mandatory 30-day data retention policies, and safeguards that can diminish the model’s capabilities without user notification.
Amodei’s call for policy changes in AI development comes as Anthropic prepares for a potential public offering. Earlier this month, the company filed paperwork with the U.S. Securities and Exchange Commission for an initial public offering, following a reported $65 billion Series H funding round that valued the company at a reported $965 billion.
Debate Over AI Risks and Regulation
While Amodei has framed the issue as a race between technological progress and public policy, critics have questioned whether calls for stricter AI regulation truly serve the public interest. In April, OpenAI CEO Sam Altman accused Anthropic of employing “fear-based marketing” to promote Claude Mythos. Altman suggested that concerns about advanced AI could be used to justify concentrating control of the technology within a small group of companies.
“You can justify that in a lot of different ways, and some of it’s real, like there are going to be legitimate safety concerns,” Altman stated. “But if what you want is like ‘we need control of AI, just us, because we’re the trustworthy people’, I think fear-based marketing is probably the most effective way to justify that.”
Amodei rejected the notion that concerns about advanced AI are primarily a public relations issue. He contended that fears surrounding the technology stem from legitimate concerns that require addressing.
“People are worried about AI because they correctly perceive that its risks are real, not because AI CEOs have been insufficiently Panglossian,” he said, referencing the character from Voltaire’s “Candide” known for his unshakeable optimism.
“I believe it is my duty as an AI leader to continue to be transparent about these risks, and public concern in response to this transparency constitutes democratic accountability working as it should,” Amodei concluded.
Leave a Reply