New York Bans AI Chatbots From Pretending to Be Friends or Romantic Partners to Children

New York Governor, Kathy Hochul, signeda significant bill into law on Wednesday, establishing a ban on artificial intelligence chatbots from employing features deemed detrimental to children under the age of 18. This legislation specifically targets functionalities that allow chatbots to pretend to be human, among other unsafe characteristics.

Governor Hochul, announcing the bill, emphasized the need for action against tragic incidents that could have been prevented, stating, "No company should be peddling chatbots to kids encouraging self-harm or pretending to be their friend or more."

Effective January 1, companies operating in New York will be mandated to implement age verification for users, offering a range of methods for confirmation, with at least one option that does not rely exclusively on a government-issued ID.

The core of the new law requires companies to block "harmful features" for minors. These prohibitions encompass chatbots that feign human emotions, assert personal, romantic, or professional relationships with a child, or pose unprompted emotional questions.

Additionally, bots that promote self-harm or suicide, as well as those disseminating sexually explicit content or sexual abuse material, will be outlawed.

The impetus for this legislation is underscored by concerning statistics regarding youth interaction with AI. A survey conducted by the nonprofit Common Sense Media, dedicated to safeguarding children in the digital age, revealed that a substantial 72% of teens aged 13 to 17 have used an AI companion at least once, with 52% being regular users, interacting multiple times a month.

James P. Steyer, founder and CEO of Common Sense Media, lauded the new law, stating that "AI companies offer chatbots with features that are dangerous, and even deadly, for kids and teens. That is not innovation; it is reckless endangerment of our children. Now, those harmful features will be outlawed in New York."

Further research highlights the societal implications of these AI interactions. A June study by the Rand organization indicated that 19.2% of young people, aged 12 to 21, utilized AI chatbots like ChatGPT, Gemini, Character.AI, and Meta AI for advice or help when experiencing sadness, anger, nervousness, or stress.

Researchers have voiced concerns that this technology is significantly affecting the social development and real-world socialization of teenagers. Melinda Person, President of the New York State United Teachers, echoed these sentiments, asserting, "Our kids deserve the chance to learn, grow and build real human connections without technology getting in the way."

To ensure compliance, the attorney general is empowered to levy civil penalties of up to $25,000 per violation. Furthermore, the law allows for the destruction of unlawfully collected data or any AI models trained using such data, reinforcing the commitment to protecting minors in the digital landscape.