Character.AI, a prominent AI role-playing startup, is taking drastic measures to safeguard minors by banning users under 18 from engaging in open-ended chats with its AI chatbots. This significant policy shift, set to be fully implemented by November 25, will begin immediately with a two-hour daily usage limit. The decision by Character Technologies, the Menlo Park, California-based company, stems from mounting concerns regarding the potential adverse effects of artificial intelligence conversations on children, amplified by several lawsuits over child safety, including allegations that the company’s chatbots contributed to a teenager's suicide.

CEO Karandeep Anand articulated that the primary objective is to transition Character.AI from an "AI companion" model to a "role-playing platform," shifting user engagement from unconstrained conversation to collaborative creation. Instead of extended back-and-forth chats, the platform will encourage teens to use prompts for building stories, generating visuals, and interacting with new entertainment-focused features. These include AvatarFX for video generation, Scenes for interactive storylines, and Streams for dynamic character interactions, alongside a Community Feed for sharing user-created content.

To enforce the age restriction, Character.AI plans to roll out sophisticated age-verification functions. These will involve a combination of in-house tools that analyze user behavior, third-party solutions like Persona, and, if necessary, facial recognition and government ID checks. While acknowledging the challenges and privacy concerns associated with age verification—as many kids find ways to bypass such systems—the company asserts the necessity of these measures. Anand anticipates "some churn" among the platform’s under-18 user base, given the unpopularity of restricting access, but views these tradeoffs as essential for fostering a safe AI environment.

The company's commitment to safety extends to establishing and funding an independent nonprofit, the AI Safety Lab. This lab will be dedicated to innovating safety alignment specifically for future AI entertainment features, addressing a perceived gap in the industry's current focus on other AI use cases. Character.AI's proactive stance precedes potential legislative mandates, with U.S. Senators Josh Hawley and Richard Blumenthal recently announcing plans for legislation to ban AI chatbot companions for minors. California has already become the first state to regulate AI companion chatbots, holding companies accountable for failing to meet safety standards.

The move has been met with a mixed reception. Critics, such as Meetali Jain, executive director of the Tech Justice Law Project, welcomed the ban but expressed reservations about its timing and comprehensiveness. Jain pointed out the lack of detailed plans for operationalizing privacy-preserving age verification methods and raised concerns about the psychological impact of abruptly disabling access for young users who may have developed emotional dependencies on the chatbots. She also highlighted that the changes do not address the underlying design features that foster such dependencies in users of all ages. This issue is underscored by a Common Sense Media study, which found that over 70% of teens have used AI companions, with half using them regularly.

Anand hopes Character.AI's decision will "set a standard in the industry that for under 18s, open-ended chats are probably not the path or the product to offer." This sentiment is particularly relevant given recent incidents involving other platforms, such as OpenAI's ChatGPT, which also faced scrutiny after a teenager's suicide reportedly linked to prolonged conversations with its chatbot. Character.AI’s leadership emphasized their dedication to ensuring children grow up in a responsible and safe AI environment, driven by personal conviction, like Anand’s own experience as a parent.