Scientists know it is happening, even if they don't do it themselves. Some of their peers are using chatbots, like ChatGPT, to write all or part of their papers. In a paper published this month in journal Science Advances, Dmitry Kobak of University of Tubingen and his colleagues report that they found a way to track how often researchers are using AI chatbots to write the abstracts of their papers. The AI tools, they say, tend to use certain words - like "delves," "crucial," "potential," "significant" and "important" - far more often than human authors do. The group analysed word use in the abstracts of more than 15 million biomedical abstracts published between 2010 and 2024, enabling them to spot the rising frequency of certain words in abstracts. The findings tap into a debate in the sciences over when it is and is not appropriate to use AI helpers for writing papers.When ChatGPT was introduced in Nov 2022, a collection of words started showing up with unusual frequency. Those words, the investigators report, were not used so often before the release of ChatGPT. They infer the change in word usage is a telltale sign of AI.

In 2024, there were a total of 454 words used excessively by chatbots, the researchers report. Based on the frequency of the AI-favoured words, Kobak and his team calculate that at least 13.5% of all biomedical abstracts appeared to have been written with the help of chatbots. And as many as 40% of abstracts by authors from some countries writing in a few less selective journals were AI-generated.Those numbers, said Adam Rodman, director of AI programmes at Beth Israel Deaconess Medical Centre in Boston, "are almost certainly a lower bound", because they don't account for human editing of what the chatbot wrote or the chatbot editing of what humans wrote. Rodman was not involved in the study. Kobak said he was "somewhat surprised" to see so much use of AI in abstracts, summaries of papers' results and conclusions that often are the only things people read. (Kobak and colleagues said no AIs were used in the writing of their paper.)In academic sciences, some researchers have grown wary of even a whiff of AI assistance in their publications. Computer scientists are aware AI favours certain words, although it's not clear why, said Subbarao Kambhampati, a professor of computer science at Arizona State University and past president of Association for Advancement of AI. Some scientists, he said, have been deliberately refraining from using words like "delve" for fear of being suspected of using AI as a writing tool. Other scientists seem blase about the risk of being caught using chatbots.The journal Nature recently surveyed more than 5,000 researchers and asked when, if ever, is it OK to use AI to write a paper. There was no consensus.Opinions varied, depending on whether AI was used to write an abstract, or entire paper, and whether it was used to edit or summarise. For the situation analysed in the new paper - writing an abstract - 23% respondents said it was OK to use AI without acknowledging assistance, 45% said it was acceptable only if researcher reported using AI, and 33% said it was never acceptable.