The CEO of People, Inc. (formerly Dotdash Meredith), Neil Vogel, has publicly accused Google of being a "bad actor" for using its web crawler to extract content from publishers' websites to support its artificial intelligence products. Vogel, who leads a digital and print publisher operating over 40 brands including People, Food & Wine, and Better Homes & Gardens, stated that Google is not playing fair. He highlighted that Google utilizes a single bot for both indexing websites for its search engine and for feeding its AI features. This dual-purpose crawling means that while Google Search still sends some traffic to publishers, its AI products are perceived as stealing content.

Vogel shared statistics illustrating a significant decline in traffic from Google Search, which accounted for approximately 65% of his company’s traffic three years ago but has since fallen to the "high 20s." He emphasized that despite this reduction, People, Inc. has managed to grow its audience and revenue, indicating the complaint is not about financial distress but about the principle of unfair competition. Vogel firmly asserted, “you cannot take our content to compete with us.”

In response to this perceived imbalance, Vogel believes publishers need greater leverage in the evolving AI landscape. His company has adopted a strategy of blocking AI crawlers that do not compensate for content, utilizing web infrastructure company Cloudflare’s latest solution. This approach aims to compel AI developers to negotiate content licensing deals. People, Inc., for instance, has successfully established a deal with OpenAI, which Vogel lauded as a "good actor" due to its willingness to engage in such agreements. While no specific names of other large LLM providers were disclosed, Vogel indicated that discussions are progressing significantly since implementing the crawler-blocking solution.

However, a critical challenge remains with Google. Vogel explained that Google's crawler cannot be blocked without simultaneously preventing the publisher's websites from being indexed in Google Search, thereby cutting off the remaining "20%-ish" of traffic it still delivers. This technological constraint, coupled with Google's refusal to split its crawler, led Vogel to explicitly declare Google an "intentional bad actor" in this context.

Janice Min, editor-in-chief and CEO at Ankler Media, echoed Vogel's sentiments, labeling large tech companies like Google and Meta as long-standing "content kleptomaniacs." Her company also blocks AI crawlers, noting a lack of perceived benefit in partnering with any AI company currently.

Matthew Prince, CEO of Cloudflare, who was part of the same panel discussion, offered a broader perspective. He speculated that changes in how AI companies operate are inevitable, possibly driven by new regulations. Prince also questioned the efficacy of relying on pre-AI copyright law to combat these issues, suggesting it might be a "fool’s errand." He argued that AI companies create derivatives, and under copyright law, the more derivative something is, the more it might be protected under fair use. Prince referenced a settlement involving Anthropic and book publishers, which aimed to preserve a positive copyright ruling for derivative use.

Furthermore, Prince provocatively stated that "everything that’s wrong with the world today is, at some level, Google’s fault," attributing to the search giant the practice of teaching publishers to prioritize traffic over original content creation, leading to behaviors like writing for clicks. Despite this critique, he acknowledged Google's current competitive predicament. Prince optimistically predicted that within the next year, Google would likely begin compensating content creators for crawling their content and integrating it into AI models, suggesting significant internal debates are already taking place within the company regarding this issue.