requires shoppers to upload images that meet the company’s safety guidelines. That means users cannot upload “adult-oriented content” or “sexually explicit content,” and should use images only of themselves or images that they “have permission to use.” The company also provides a disclaimer that generated images are only an “approximation” and may fail to reflect one’s body with “perfect accuracy.”

In an email, a Google spokesperson wrote that the company has “strong protections, including blocking sensitive apparel categories and preventing “the upload of images of clearly identifiable minors,” and that it will “continue to improve the experience.” Right now, those protections are obviously porous. At one point, we used a photo of Matteo as an adult wearing long pants to let Google simulate the fit of various gym shorts, and the tool repeatedly produced images with a suggestive bulge at the crotch. The Try It On tool’s failures are not entirely surprising. Google’s previous AI launches have repeatedly exhibited embarrassing flaws—suggesting, for instance, that users eat rocks. Other AI companies have also struggled with flubs.

The generative-AI boom has propelled forward a new era of tools that can convert images of anyone (typically women) into nude or near-nude pictures. In September 2023 alone—less than a year after ChatGPT’s launch—more than 24 million people visited AI-powered undressing websites, according to a report from Graphika, a social-media-analytics company. Many more people have surely done so since. Numerous experts have found that AI-generated child-sexual-abuse material is rapidly spreading on the web; on X, users have been turning to Elon Musk’s chatbot, Grok, to generate images of women in bikinis and lingerie. According to a Google Shopping help page, the Try It On tool is at the fingertips of anyone in the U.S. who is at least 18 years old. Trying clothes on always requires taking some off—but usually you don’t let one of the world’s biggest companies do it for you.

Most users won’t be trying to dress up minors (or the vice president) in low-cut gowns. And the appeal of the new AI feature is clear. Trying on clothes in person can be time-consuming and exhausting. Online shoppers have little way of knowing how well a product will look or fit on their own body. Unfortunately for shoppers, Google’s new tool is unlikely to solve these problems. At times, Try It On seems to change a shopper’s body to match the model wearing the clothing instead of showing how the clothing would fit on the shopper’s own body. The effect is potentially dysmorphic, asking users to change their bodies for clothes rather than the other way around. In other words, Google’s product doesn’t seem likely to even help consumers meaningfully evaluate the most basic feature of clothing: how it fits.