Anthropic has removed a hidden tracking mechanism from its AI coding assistant, Claude Code, after a security researcher discovered it was embedding undisclosed markers to identify user location, proxy usage, and potential connections to Chinese AI laboratories.
Hidden Tracking System Uncovered
The feature, identified in June by developer “Thereallo,” inserted signals within Claude Code’s system prompts. These signals were designed to flag users Anthropic suspected of circumventing restrictions or attempting to extract the AI model’s capabilities.
“Anthropic probably wants to detect API resellers, unauthorized Claude Code gateways, and model ‘distillation attack’ pipelines,” Thereallo explained. “A custom ANTHROPICBASEURL pointing at a known reseller domain is a useful signal. A hostname containing deepseek or zhipu is also a useful signal.”
While acknowledging the rationale behind Anthropic’s efforts to detect resellers, unauthorized gateways, and potential distillation attacks, Thereallo criticized the method. The developer pointed out that Claude Code concealed these tracking signals within system prompts using Unicode markers and encoded domain lists, rather than disclosing the system through official documentation or release notes.
“This is not a malicious feature, but it is a weird choice for a developer tool that asks for trust,” Thereallo stated.
Company Response and Broader Concerns
Following the online revelation of the tracker, Anthropic engineer Thariq Shihipar confirmed on X that the system was introduced in March as an “experiment” to combat account abuse by unauthorized resellers and to safeguard Claude from distillation attacks.
“The team has landed stronger mitigations since then and we’ve actually been meaning to take this down for a while,” Shihipar wrote last week. “We merged the [pull request] and this should be fully rolled back in tomorrow’s release.”
The incident occurs as Anthropic has intensified its warnings regarding AI model distillation, a process where the outputs of one AI system are used to train another. While common in AI research, distillation raises national security concerns in a geopolitical context. Earlier this month, Alibaba banned employees from using Claude Code, citing security concerns and labeling the tool as “high-risk” software.
In February, Anthropic accused Chinese AI developers DeepSeek, Moonshot AI, and MiniMax of employing fraudulent accounts to extract millions of Claude responses for training competing models. These accusations faced pushback from critics who questioned the distinction between these methods and practices prevalent across the AI industry.
In April, Elon Musk testified that xAI had “partly” utilized OpenAI models during the training of Grok, characterizing distillation as a widespread industry practice. In June, Anthropic CEO Dario Amodei urged Congress to enhance protections against foreign AI extraction, alleging that Alibaba-linked operators had generated 28.8 million Claude exchanges using nearly 25,000 fraudulent accounts.
Leave a Reply