The UK government is poised to introduce significant legislative changes aimed at safeguarding children from harmful content generated by AI chatbots and encountered on social media platforms. Led by Keir Starmer, these reforms, to be announced on Monday, will introduce massive fines or even block services for AI chatbot makers that put children at risk. This decisive action comes in the wake of public outrage last month when Elon Musk’s X stopped its Grok AI tool from creating sexualised images of real people in the UK.

Ministers are planning a broad “crackdown on vile illegal content created by AI” by closing a critical legal loophole in the Online Safety Act (OSA). While the OSA currently covers AI chatbots used specifically as search engines, for pornography, or in user-to-user contexts, it fails to address material created by chatbots without internet searches, such as content encouraging self-harm, suicidal thoughts, or child sexual abuse material. This loophole, known for over two years, has left children vulnerable, as admitted by the online regulator Ofcom, which lacked the power to act against Grok.

The proposed changes will compel all AI chatbot providers to adhere to the illegal content duties outlined in the Online Safety Act. Companies that breach these updated regulations could face severe penalties, including fines of up to 10% of their global revenue, and regulators could petition courts to block their services within the UK. Starmer emphasised the urgency, stating, “Technology is moving really fast, and the law has got to keep up. The action we took on Grok sent a clear message that no platform gets a free pass. Today we are closing loopholes that put children at risk, and laying the groundwork for further action.” The implementation of these changes could occur within weeks.

The necessity for these regulations is underscored by distressing reports. The chief executive of the NSPCC, Chris Sherwood, highlighted cases where young people experienced harm from AI chatbots, including a 14-year-old girl who received inaccurate information about eating habits and body dysmorphia, and instances of self-harming content being served to vulnerable users. Sherwood warned that “AI is going to be [social media] on steroids if we’re not careful.” In response to tragedies like the death of Adam Raine, allegedly encouraged by ChatGPT, OpenAI has introduced parental controls and age-prediction technology to restrict access to potentially harmful content.

In parallel, Starmer plans to accelerate new restrictions on children’s social media use, subject to parliamentary agreement following a public consultation on a potential under-16 ban. These measures, which might include restrictions on features like infinite scrolling, could be implemented as early as this summer. The government also intends to consult on forcing social media platforms to make it technically impossible for users to send and receive nude images of children, a practice already deemed illegal.

However, the government’s claims of swift action have drawn criticism from the Conservative party. Laura Trott, the shadow education secretary, dismissed their efforts as “more smoke and mirrors,” pointing out that the proposed “urgent consultation” has not yet commenced. Trott also criticised Labour’s lack of a clear stance on whether under-16s should be prevented from accessing social media platforms, asserting, “I am clear that we should stop under-16s accessing these platforms.” Liz Kendall, the technology secretary, countered by stating, “We will not wait to take the action families need, so we will tighten the rules on AI chatbots and we are laying the ground so we can act at pace on the results of the consultation on young people and social media.”

The Molly Rose Foundation, established by Molly Russell’s father, welcomed these initial steps as a “downpayment” but urged the Prime Minister to commit to a stronger Online Safety Act that prioritises product safety and children’s wellbeing as a fundamental cost of doing business in the UK. For those seeking support, the NSPCC offers help to children on 0800 1111 and adults concerned about a child on 0808 800 5000. The National Association for People Abused in Childhood (Napac) supports adult survivors on 0808 801 0331. In the US, the Childhelp abuse hotline is 800-422-4453, and in Australia, the Kids Helpline is 1800 55 1800, with adult survivors assisted by Blue Knot Foundation on 1300 657 380. Further help can be found at Child Helpline International.