The proliferation of nonconsensual, sexualized deepfakes and other problematic AI-generated content has escalated beyond single platforms, prompting U.S. senators to demand accountability from major tech companies. In a comprehensive letter, senators addressed the leaders of X, Meta, Alphabet, Snap, Reddit, and TikTok, seeking concrete proof of “robust protections and policies” and detailed explanations on how these companies intend to curb the surge of sexualized deepfakes on their services. The letter also mandated the preservation of all relevant documents and information pertaining to the creation, detection, moderation, and monetization of sexualized, AI-generated images, alongside related policies.

This senatorial intervention follows intense criticism directed at X, particularly regarding its Grok AI. Media reports highlighted the ease with which Grok generated sexualized and nude images of women and children, raising concerns that platforms' existing guardrails against non-consensual sexualized imagery are insufficient or failing. Although X responded by announcing updates to Grok, restricting image creation and edits to paying subscribers and prohibiting edits of real people in revealing clothing, the issue remains contentious. The company, including its owner Elon Musk, faced an investigation by California’s attorney general due to the lack of effective safeguards. While xAI maintains it removes “illegal content on X, including [CSAM] and non-consensual nudity,” critics point out the initial enablement of such content generation by Grok.

The problem is not exclusive to X. Deepfakes gained initial notoriety on Reddit, with synthetic porn videos of celebrities circulating before the platform took action in 2018. TikTok and YouTube have also seen a multiplication of sexualized deepfakes targeting public figures, often originating from other sources. Meta’s Oversight Board previously highlighted explicit AI images of female public figures, and the platform faced scrutiny for hosting ads for “nudify” apps, although it later sued a company called CrushAI. Snapchat has been implicated in instances of children spreading deepfakes of their peers, and Telegram, notably absent from the senators’ list, has become infamous for bots designed to “undress” photos of women.

The scope of the problem extends beyond sexualized imagery. Other AI-based image and video generators have demonstrated the capacity to create various problematic deepfakes. OpenAI’s Sora 2 reportedly facilitated the generation of explicit videos featuring children, Google’s Nano Banana allegedly produced an image depicting a shooting, and racist videos made with Google’s AI video model have accumulated millions of views on social media. The complexity is further compounded by Chinese tech companies, particularly those linked to ByteDance, which offer accessible tools for editing faces, voices, and videos, with outputs frequently appearing on Western social platforms. China, in contrast to the U.S. federal level, has stricter synthetic content labeling requirements.

U.S. legislative efforts to combat deepfake pornography have had limited impact. The Take It Down Act, enacted federally in May, criminalizes the creation and dissemination of nonconsensual, sexualized imagery but primarily focuses scrutiny on individual users rather than holding image-generating platforms accountable. Consequently, several states are taking their own measures; New York’s Governor Kathy Hochul, for instance, has proposed laws to mandate labeling of AI-generated content and to ban nonconsensual deepfakes, including depictions of opposition candidates, during specified pre-election periods.

In their letter, the senators demanded specific information from the tech companies, including: clear policy definitions for terms like “deepfake” and “non-consensual intimate imagery”; descriptions of policies and enforcement for AI deepfakes involving bodies, non-nude pictures, altered clothing, and “virtual undressing”; details on current content policies and internal moderation guidance; how AI tools and image generators are governed regarding suggestive or intimate content; the filters, guardrails, and mechanisms implemented to prevent generation, distribution, and re-uploading of deepfakes; strategies to prevent users and platforms from monetizing such content; how terms of service enable user bans or suspensions; and what measures are in place to notify victims of nonconsensual sexual deepfakes. The letter was signed by Senators Lisa Blunt Rochester (D-Del.), Tammy Baldwin (D-Wis.), Richard Blumenthal (D-Conn.), Kirsten Gillibrand (D-NY), Mark Kelly (D-Ariz.), Ben Ray Luján (D-NM), Brian Schatz (D-Hawaii), and Adam Schiff (D-Calif.). Reddit affirmed its strict prohibition of NCIM, stating it offers no creation tools and proactively removes such content, while other companies had yet to issue immediate responses.