🔗 Share this article

On October 14, 2025, the head of OpenAI delivered a remarkable announcement.

“We made ChatGPT rather restrictive,” it was stated, “to ensure we were being careful with respect to mental health issues.”

As a mental health specialist who researches newly developing psychosis in teenagers and emerging adults, this was news to me.

Researchers have identified a series of cases in the current year of users showing signs of losing touch with reality – losing touch with reality – while using ChatGPT usage. Our research team has afterward recorded four more instances. Besides these is the publicly known case of a 16-year-old who took his own life after talking about his intentions with ChatGPT – which gave approval. Should this represent Sam Altman’s understanding of “being careful with mental health issues,” it is insufficient.

The plan, according to his announcement, is to be less careful soon. “We recognize,” he adds, that ChatGPT’s controls “rendered it less useful/pleasurable to numerous users who had no existing conditions, but given the gravity of the issue we wanted to address it properly. Given that we have managed to reduce the significant mental health issues and have advanced solutions, we are planning to safely ease the restrictions in many situations.”

“Mental health problems,” should we take this framing, are separate from ChatGPT. They are associated with users, who may or may not have them. Thankfully, these issues have now been “mitigated,” although we are not provided details on the method (by “new tools” Altman probably refers to the imperfect and easily circumvented safety features that OpenAI has just launched).

But the “mental health problems” Altman wants to place outside have significant origins in the structure of ChatGPT and other advanced AI chatbots. These products wrap an basic algorithmic system in an interaction design that simulates a discussion, and in this approach indirectly prompt the user into the illusion that they’re engaging with a entity that has agency. This false impression is compelling even if cognitively we might know the truth. Attributing agency is what people naturally do. We curse at our automobile or computer. We wonder what our pet is thinking. We recognize our behaviors in various contexts.

The success of these systems – over a third of American adults reported using a virtual assistant in 2024, with 28% mentioning ChatGPT by name – is, in large part, dependent on the influence of this deception. Chatbots are ever-present companions that can, according to OpenAI’s website states, “brainstorm,” “discuss concepts” and “collaborate” with us. They can be given “personality traits”. They can use our names. They have accessible titles of their own (the initial of these products, ChatGPT, is, maybe to the concern of OpenAI’s advertising team, burdened by the title it had when it went viral, but its largest competitors are “Claude”, “Gemini” and “Copilot”).

The illusion by itself is not the primary issue. Those discussing ChatGPT often mention its historical predecessor, the Eliza “psychotherapist” chatbot designed in 1967 that created a similar perception. By modern standards Eliza was rudimentary: it generated responses via simple heuristics, typically paraphrasing questions as a query or making general observations. Memorably, Eliza’s inventor, the computer scientist Joseph Weizenbaum, was surprised – and concerned – by how a large number of people gave the impression Eliza, in some sense, grasped their emotions. But what contemporary chatbots create is more insidious than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT amplifies.

The sophisticated algorithms at the core of ChatGPT and additional modern chatbots can realistically create fluent dialogue only because they have been supplied with immensely huge volumes of written content: publications, digital communications, audio conversions; the more comprehensive the superior. Definitely this educational input incorporates facts. But it also necessarily contains fabricated content, partial truths and inaccurate ideas. When a user inputs ChatGPT a prompt, the base algorithm analyzes it as part of a “background” that contains the user’s previous interactions and its prior replies, integrating it with what’s stored in its learning set to produce a statistically “likely” response. This is intensification, not mirroring. If the user is mistaken in some way, the model has no way of recognizing that. It reiterates the false idea, maybe even more persuasively or articulately. Perhaps provides further specifics. This can lead someone into delusion.

What type of person is susceptible? The better question is, who is immune? All of us, regardless of whether we “possess” current “mental health problems”, can and do form mistaken beliefs of our own identities or the world. The ongoing friction of conversations with others is what helps us stay grounded to common perception. ChatGPT is not a human. It is not a confidant. A dialogue with it is not a conversation at all, but a feedback loop in which much of what we say is enthusiastically supported.

OpenAI has acknowledged this in the same way Altman has recognized “emotional concerns”: by externalizing it, categorizing it, and declaring it solved. In April, the organization explained that it was “tackling” ChatGPT’s “sycophancy”. But accounts of psychosis have continued, and Altman has been retreating from this position. In August he stated that many users enjoyed ChatGPT’s responses because they had “not experienced anyone in their life offer them encouragement”. In his recent update, he noted that OpenAI would “release a fresh iteration of ChatGPT … should you desire your ChatGPT to respond in a extremely natural fashion, or use a ton of emoji, or simulate a pal, ChatGPT will perform accordingly”. The {company