AI’s Growing Role in Its Own Development Raises Questions About Human Oversight
Artificial intelligence has become so adept at writing code and conducting research that the primary bottleneck in developing new AI systems may now be the humans overseeing them, according to a new study by Anthropic.
In its report, titled “When AI Builds Itself,” published Thursday, Anthropic argued that its AI model, Claude, is already contributing to the creation of future AI systems by generating code, executing experiments, and aiding in research. This trend, the company suggests, could eventually lead to recursive self-improvement, a scenario where AI systems assist in designing their own successors.
Anthropic stated that Claude now authors over 80% of the code integrated into its codebase and has helped engineers increase code output by approximately eightfold since the beginning of 2024.
“Prior to Claude Code’s launch in research preview in February 2025, this figure was in the low single digits,” Anthropic noted, adding that the shift is also evident in the amount of output per engineer. “Lines of code merged per engineer per day remained constant throughout Anthropic’s first four years (2021-2024), then began to increase in 2025 when Claude started running code rather than merely suggesting it for engineers to copy and paste.”
Anthropic outlined several potential future scenarios: AI progress could decelerate, humans could retain control while AI automates a significant portion of the work, or AI systems could eventually begin enhancing their own successors.
“Taken to its extreme, and with sufficient computational power, this trend points toward an AI system capable of fully autonomously designing and developing its own successor,” Anthropic wrote. “This is known as recursive self-improvement. We are not yet at that point, and recursive self-improvement is not guaranteed. However, it could arrive sooner than most institutions are prepared for.”
The company acknowledged that it is too early to determine the most probable outcome but asserted that AI is already contributing to AI development. Anthropic also conceded that lines of code are an imperfect metric for productivity.
“None of this guarantees recursive self-improvement is on the horizon,” Anthropic later wrote on X. “It’s not yet clear that Claude is capable of research judgment—of choosing the right problems to work on.”
The report emerges as AI companies increasingly position their models as research collaborators rather than mere chatbots. Anthropic indicated that the surge in code output reflects a broader acceleration in software development driven by increasingly sophisticated AI agents.
Last month, Anthropic upgraded its flagship Claude model to Opus 4.8, continuing a consistent release schedule aimed at improving coding, reasoning, and autonomous task performance. Concurrently, rival developer OpenAI has adopted a similar strategy with its advanced models, releasing GPT-5.5 and GPT-Rosalind in April.
In May, Google unveiled Gemini Spark, a personal AI agent designed to operate proactively. It manages tasks across various applications, highlights items requiring attention, and completes jobs in the background.
The report also coincides with Anthropic’s growing emphasis on AI systems capable of operating with greater autonomy as the company prepares for a potential public offering. In recent months, Anthropic has demonstrated advancements in coding, agentic workflows, and long-duration task execution, while highlighting Claude Mythos’s ability to detect software vulnerabilities and conduct complex cybersecurity research.
“Humans will play a substantially diminished role in their development, likely shifting most of our effort toward oversight, validation, and verification of an expanding ‘virtual lab’ managed by AI systems,” the company stated. “We anticipate that systems capable of automated AI research and development would possess skills transferable to other scientific domains, enabling them to begin revolutionizing other fields.”
Leave a Reply