Microsoft Discloses Vulnerability in Anthropic’s Claude Code AI Agent
Microsoft researchers have revealed a now-patched vulnerability in Anthropic’s Claude Code GitHub Action that could have allowed attackers to expose credentials stored within software development pipelines. The exploit involved manipulating the AI agent through malicious GitHub content.
In a blog post on Friday, Microsoft highlighted that AI coding agents operating within Continuous Integration/Continuous Deployment (CI/CD) workflows present new security risks. These environments frequently hold access to sensitive information such as API keys and cloud credentials.
“We initiated this research after observing prompt injection attempts in public repositories utilizing AI-assisted GitHub workflows across various vendors,” Microsoft stated. “In these instances, attacker-controlled issues or pull requests, along with their content, were processed by the AI agent, potentially influencing its tool usage.”
On GitHub, a pull request is a mechanism for developers to propose changes to a code repository, subject to review before being merged.
This disclosure comes as prompt injection attacks have emerged as a significant security threat to AI agents. Such attacks involve an attacker embedding hidden instructions within content like emails, documents, websites, or code comments, causing the AI system to execute these unauthorized commands instead of the intended user actions.
Claude Code, launched in October, is Anthropic’s AI coding agent designed for software development tasks. The tool previously drew attention in March when Anthropic inadvertently leaked over 500,000 lines of its source code, revealing details of its internal architecture and prompting extensive analysis by researchers and developers.
According to Microsoft, attackers could leverage prompt injection attacks embedded in GitHub issues, pull requests, or comments to manipulate Claude Code into accessing files containing sensitive credentials.
To demonstrate the vulnerability, Microsoft constructed a GitHub workflow. They disguised malicious instructions within content hosted on a domain they controlled, enabling them to bypass Claude’s safety measures. The prompt injection attack successfully tricked Claude into reading sensitive credentials and altering them in a way that evaded both Claude’s safeguards and GitHub’s secret-scanning tools. Microsoft explained that an attacker could then reconstruct these credentials and exfiltrate them through methods such as issue comments, workflow logs, web requests, or shell commands.
“To circumvent Sonnet’s refusal safety mechanisms, we obscured the shell payload behind a response from our controlled domain,” the firm noted. “We also configured the workflow to be triggered by users without ‘write’ permissions, ensuring that Anthropic’s environment variable scrubbing mitigations were active during our tests.”
Anthropic addressed the flaw on May 5 with the release of Claude Code version 2.1.128. Microsoft had initially disclosed the vulnerability through HackerOne on April 29.
Despite the implementation of multiple layers of built-in security controls, Microsoft’s findings indicate that a determined attacker could potentially manipulate an AI agent into exposing sensitive information.
“We are entering an era where natural language is executable code, and untrusted inputs like GitHub issues must be treated as hostile by default,” Microsoft concluded. “A single, carefully crafted comment, combined with a misunderstood trust boundary, is all it takes to walk away with production credentials.”
Leave a Reply