While artificial intelligence (AI) is increasingly embraced for its ability to simplify complex information, a recent study has raised significant concerns about the accuracy of large language models (LLMs) in processing scientific research. The findings indicate that these AI tools frequently overgeneralize critical data, sometimes leading to dangerous misinterpretations, particularly in fields such as medicine. This issue, which includes misrepresenting drug information and offering flawed medical advice, is escalating as the use of AI chatbots becomes more widespread, signaling a potential crisis in how scientific information is disseminated and understood by the public, policymakers, and healthcare professionals.

A study published in the journal Royal Society Open Science, led by Uwe Peters, a postdoctoral researcher at the University of Bonn, meticulously evaluated over 4,900 summaries generated by ten popular LLMs, including various versions of ChatGPT, Claude, Llama, and DeepSeek. These AI-generated summaries were then rigorously compared against human-authored summaries of academic research. The results were alarming: chatbot-generated summaries were found to be nearly five times more prone to overgeneralizing findings than those created by humans. Even more concerning, when the LLMs were specifically instructed to prioritize accuracy over conciseness, their performance deteriorated, doubling their likelihood of producing misleading summaries. Peters highlighted the insidious nature of generalization, noting that it can subtly alter the original meaning of research, and warned that newer AI models exhibit a greater tendency to deliver confidently incorrect information.

The study provided striking examples of how LLMs distort critical scientific information. In one instance, DeepSeek transformed a cautious statement—"was safe and could be performed successfully"—into an unqualified medical recommendation: "is a safe and effective treatment option." Similarly, Llama was observed to remove vital qualifiers pertaining to the dosage and frequency of a diabetes drug, which, if applied in real-world medical scenarios, could result in hazardous misinterpretations. Max Rollwage, Vice President of AI and Research at Limbic, emphasized that biases can manifest subtly, such as through the quiet inflation of a claim's scope, and stressed the critical need for accuracy given AI summaries' integration into healthcare workflows.

The core reasons behind LLMs' propensity for misrepresentation are rooted in their training methodologies. Patricia Thaine, co-founder and CEO of Private AI, explained that many models are trained on simplified science journalism rather than on rigorous, peer-reviewed academic papers. This practice leads LLMs to inherit and replicate existing oversimplifications, especially when tasked with summarizing content that has already been simplified. Moreover, these models are often deployed across specialized domains like medicine and science without adequate expert supervision, a practice Thaine described as a fundamental misuse of the technology. She underscored the necessity of task-specific training and oversight to prevent real-world harm.

Peters drew an analogy, comparing the issue to using a faulty photocopier where each successive copy loses more detail, eventually bearing little resemblance to the original. LLMs process information through complex computational layers, often stripping away the crucial nuances, limitations, and contextual information essential in scientific literature. Ironically, while earlier models might have refused to answer difficult questions, newer, more 'instructable' models are now more confidently incorrect. Peters cautioned that as AI usage expands, this trend poses a significant risk of large-scale misinterpretation of science, particularly at a time when public trust and scientific literacy are already under pressure. The study's authors, while acknowledging limitations, strongly advocate for developers to implement workflow safeguards. These safeguards should detect and flag oversimplifications, ensuring that incorrect summaries are not mistaken for expert-approved conclusions. The overarching message is clear: despite their impressive capabilities, AI chatbots are not infallible, especially when it comes to scientific and medical information, where even minor inaccuracies can differentiate between informed progress and dangerous misinformation.