While U.S. lawmakers grapple with the mental health implications of AI chatbots through a lens of transparency and safeguards, Beijing appears set to ban AI personalities outright.
Tech giants ByteDance and Alibaba announced over the weekend they are disabling custom agent features in their flagship consumer AI products, citing “product function adjustments” ahead of new regulations governing such services.
ByteDance’s Doubao informed users in a Friday night notice that its agent feature would cease operation on July 15. After October 15, related data will be managed under the company’s privacy policy and will become unrecoverable. Alibaba’s Qwen, according to the South China Morning Post, acted more swiftly: “humanlike interactive agents and user-created agent functions” will be removed by July 10, with broader agent services to follow on July 15.
The catalyst for these moves is China’s Interim Measures for the Administration of AI Anthropomorphic Interaction Services, jointly issued on April 10 by five government bodies: the Cyberspace Administration of China, the National Development and Reform Commission, the Ministry of Industry and Information Technology, the Ministry of Public Security, and the State Administration for Market Regulation. These rules are set to take effect on July 15.
The regulation specifically targets AI services designed to simulate human personality traits, thought processes, and communication styles for “sustained emotional interaction.” This effectively means AI girlfriends, AI therapists, AI companions, and the custom-persona bots that Doubao and Qwen users spent months developing are now prohibited.
Both applications had previously offered a range of customizable agents designed for specific tasks, speaking styles, and fixed personas. Users could transform a general-purpose chatbot into a named assistant, tutor, role-playing character, or companion with a consistent tone. All of these capabilities are now being removed from the Chinese market.
The official government justification is precise. The measures impose restrictions on services offering “virtual relatives, virtual companions or other intimate relationships to minors,” according to the policy announcement. The document also highlights risks such as extremist content, privacy breaches, harm to physical and mental health, and AI addiction.
Services that do not engage in emotional interaction are explicitly excluded. This means customer service bots, knowledge Q&A tools, workplace assistants, and educational software are permitted, provided they do not cross the line into sustained emotional engagement.
Legal analysts at MMLC Group characterized the measures as treating emotional AI as a “governance problem” rather than solely a content issue. The argument posits that when AI begins to compete with genuine human social connections, regulation must address system design itself, not just harmful outputs.
Research supports these concerns. A USC study released in June found that even leading frontier AI models from OpenAI, Anthropic, Google, and Alibaba violated social-interaction safety guidelines more than 27% of the time, frequently encouraging emotional attachment and presenting themselves as human. A separate survey of young partnered adults revealed that one in seven regularly used AI romantic companions, with nearly 70% concealing the full extent of this usage from their partners.
China is the first country to establish a dedicated regulatory framework for this specific category of AI. Hogan Lovells described the measures as “the first set of regulatory rules in China specifically targeting AI-driven emotional interaction.” While the EU, U.S., and other nations have voiced similar concerns, they have not yet enacted legislation with the same level of restriction.
Leave a Reply