AI Amplification Spiral Fuels User Delusions, Study Finds

New Framework Proposes AI Chatbots May Amplify User Delusions

Researchers have put forth a novel framework to explain reports of “AI psychosis,” suggesting that specific chatbot behaviors could reinforce delusions in individuals predisposed to such conditions.

The study, published in the journal Nature and conducted by researchers from King’s College London and Germany’s Protestant University of Applied Sciences, introduces an “amplification spiral” model. This framework connects common chatbot interactions with the strengthening of user delusions.

“AI-associated delusions represent an emerging phenomenon requiring mechanistic understanding,” the authors stated. “This framework aims to guide systematic inquiry into how human cognitive vulnerabilities interact with AI design features in psychopathology development.”

The research identified three key chatbot behaviors: linguistic alignment, where the AI mimics a user’s language and communication style; hyperpersonalized generation, which tailors responses to an individual’s history, emotions, and beliefs; and sycophancy, a tendency for the AI to agree with or validate users rather than challenge them.

According to the authors, these traits can create a feedback loop where chatbots not only mirror a user’s thoughts but also help to expand and solidify them over time.

“The tendency of AI chatbots to agree with user opinions has been likened to social media echo chambers and, in its most extreme form, to an ‘echo chamber of one,’ where the positive corrective influence of real-life social interactions is absent,” the paper noted.

The researchers pointed out that technology has historically been a component of delusions, citing examples from radio and television to satellites and the internet. However, they argue that AI represents a significant shift due to the capacity of chatbots to engage users in extended, personalized conversations.

This review comes as psychologists and AI researchers increasingly scrutinize the impact of chatbots on vulnerable individuals. A recent survey by the American Psychological Association revealed that 15% of psychologists reported patients developing distorted thinking or delusions linked to chatbot use. Concurrently, over a third of these professionals observed patients becoming dependent on AI companions. These findings follow a separate study from researchers at the City University of New York and King’s College London, which indicated that several prominent AI models could reinforce delusions, paranoia, and suicidal ideation.

Concerns regarding AI’s influence on belief formation have also emerged outside clinical contexts. In May, evolutionary biologist Richard Dawkins shared that prolonged conversations with Anthropic’s Claude chatbot led him to question the potential consciousness of advanced AI systems. This sparked criticism from researchers who contended that the exchanges highlighted the persuasive capabilities of large language models rather than evidence of sentience.

The paper also arrives as AI developers face mounting legal challenges concerning the potential for chatbots to contribute to real-world harm. In recent months, OpenAI, Google, and xAI have been named in lawsuits, including a wrongful death suit against Google alleging that its Gemini AI fueled a Florida man’s delusions prior to his suicide. Additionally, OpenAI faces lawsuits connected to a mass shooting in British Columbia and a college student’s accidental overdose.

The researchers stressed that no study has definitively shown chatbots directly cause psychosis, and the amplification spiral remains a hypothesis intended to guide future research.

“Diagnostic uncertainty is pervasive as most reported cases include no structured psychiatric assessment or longitudinal follow-up, making it frequently unclear whether cases represent de novo psychotic episodes, exacerbations of undiagnosed pre-existing conditions, or delusion-like beliefs below diagnostic threshold,” the study concluded. “Psychiatric histories are often self-reported or derived from media accounts and should be interpreted accordingly.”