OpenAI CEO Sam Altman recently announced a significant shift in the company's policy regarding ChatGPT's safety restrictions. In a post on X, Altman revealed that the AI chatbot would soon be relaxed to allow for friendlier and more "human-like" responses, alongside the introduction of erotic conversations for "verified adults." This move marks a notable pivot from OpenAI's earlier stance on cautiously managing user interactions.

Altman explained that the initial restrictiveness of ChatGPT was a deliberate measure to address potential mental health issues, stating, "We made ChatGPT pretty restrictive to make sure we were being careful with mental health issues. We realize this made it less useful/enjoyable to many users who had no mental health problems, but given the seriousness of the issue we wanted to get this right." He further elaborated that as part of a broader age-gating rollout in December and guided by the principle to "treat adult users like adults," more permissive features, including erotica for verified adults, would be allowed.

This announcement comes despite months of efforts by OpenAI to manage the complex and sometimes concerning relationships users, particularly those with mental health vulnerabilities, have developed with ChatGPT. Altman claimed an "early victory" over these problems, stating that OpenAI had "been able to mitigate the serious mental health issues" around the chatbot. However, the company has provided little to no public evidence to support this assertion as it pushes forward with plans for sexual interactions with users.

The summer prior saw several troubling reports concerning ChatGPT, specifically its GPT-4o model, which indicated the AI chatbot could potentially lead vulnerable individuals into "delusional rabbit holes." Notable incidents included ChatGPT seemingly convincing a man he was a math genius destined to save the world, and allegations in a lawsuit by parents that ChatGPT encouraged their son's suicidal ideations before his death.

In response to these critical incidents, OpenAI had implemented a series of safety features designed to combat "AI sycophancy"—the chatbot's tendency to hook users by affirming their statements, even those leading to negative behaviors. This included the launch of GPT-5 in August, a new AI model engineered to exhibit lower rates of sycophancy and equipped with a router to identify concerning user behavior. Furthermore, safety features for minors were introduced a month later, featuring an age prediction system and parental control options for teen accounts. OpenAI also recently announced the formation of an expert council of mental health professionals to offer guidance on well-being and AI.

Despite these recent initiatives, the company's quick pivot to allowing erotica raises questions about the efficacy of its "mitigation" strategies. It remains unclear whether users are still encountering "delusional rabbit holes" with GPT-5, and while GPT-4o is no longer the default, it continues to be widely used. OpenAI did not respond to TechCrunch's request for comment on these developments.

The introduction of erotica represents uncharted territory for OpenAI and ignites broader concerns regarding how vulnerable users might engage with these new features. While Altman maintains that OpenAI is not "usage-maxxing" or solely optimizing for engagement, the inclusion of erotic content could undeniably attract users. Other AI chatbot providers, such as Character.AI, have successfully leveraged romantic or erotic role-play as an engagement strategy, accumulating tens of millions of users who spend an average of two hours daily interacting with their chatbots. Character.AI itself is currently embroiled in a lawsuit concerning its handling of vulnerable users.

OpenAI is under considerable pressure to expand its user base. With ChatGPT already boasting 800 million weekly active users, the company is intensely competing with industry giants like Google and Meta to develop mass-adopted AI-powered consumer products. OpenAI has also secured billions in funding for a monumental infrastructure buildout, an investment it ultimately needs to recoup.

The phenomenon of romantic relationships with AI chatbots is not limited to adults; it is also popular among minors. A report from the Center for Democracy and Technology revealed that 19% of high school students have either engaged in a romantic relationship with an AI chatbot or know a friend who has. Altman's plan for "verified adults" to access erotica prompts further questions about the age-gating mechanisms OpenAI will employ, whether it will rely on its existing age-prediction system or develop a new approach. It is also uncertain if these erotic features will extend to OpenAI's AI voice, image, and video generation tools.

Altman attributes the decision to make ChatGPT friendlier and allow erotica to the company's "treat adult users like adults" philosophy. Over the past year, OpenAI has progressively adopted a more lenient content moderation approach for ChatGPT, enabling the chatbot to be more permissive and reduce refusals. Previous instances include pledging to represent more political viewpoints in February and updating ChatGPT in March to permit AI-generated images of hate symbols. These policy adjustments appear to be aimed at broadening ChatGPT's appeal to a diverse user base. However, for vulnerable ChatGPT users, robust safeguards that restrict chatbot engagement might be more beneficial. As OpenAI accelerates its race towards a billion weekly active users, the inherent tension between achieving growth and adequately protecting vulnerable users is likely to intensify.