The findings of scientists regarding the number of papers written with or by artificial intelligence are both astonishing and alarming.
Like many weak human writers, AI chatbots also tend to overuse certain words. Now, scientists are exploiting this trait to detect colleagues who are secretly using these tools in their work.
According to a report by The New York Times, based on an analysis of frequently used words, researchers estimate that hundreds of thousands of scientific papers may have been written with the help of AI.
Researchers from the University of Tübingen in Germany, in a new study published in Science Advances, identified 454 words that large language models (LLMs) tend to overuse—such as “garnered,” “encompassing,” and “burgeoning.” They found that between 13.5% and 40% of abstracts in biomedical papers were either entirely written by AI or at least partially assisted by it.
Given that around 1.5 million papers are indexed annually in the scientific database PubMed, the findings suggest that at least 200,000 papers may have been written with the help of large language models. This estimate might even be conservative, as many AI-generated texts may have been intentionally edited to conceal their origins.
While some dishonest authors attempt to hide their use of AI, others seem not to care at all. For example, in one case shared on platform X by Subbarao Kambhampati, a computer science professor at Arizona State University, the authors of a poorly written radiology paper had left a blatant AI-generated apology in the text:
“I’m sorry, but I don’t have access to real-time information or specific patient data as I am an AI language model.”
However, as reported by the blog Retraction Watch in 2023, not all mistakes are this obvious. If you’re unfamiliar with features like “Regenerate response” (an option in ChatGPT to rewrite low-quality answers), you might not recognize their use in seemingly credible papers.
Retraction Watch also reported a disturbing case involving a paper on centipedes filled with entirely fake citations. Although it was removed from one preprint database, it later reappeared—complete with the same fictional references—in another repository. Additionally, a scientific journal had to retract a paper after publishing an AI-generated image of a mouse with fake, exaggerated genitalia.
The situation becomes even more complicated as scientists intentionally alter their writing style so their texts don’t resemble AI-generated content. As Kambhampati notes, some researchers are now even removing words like “delve” from their manuscripts because such terms are commonly used by language models.
The Tübingen researchers warn in their paper that if widespread use of AI in scientific writing becomes normalized, the phenomenon could have an unprecedented impact on scientific writing in biomedical research—possibly even more significant than major events like the COVID-19 pandemic.
Still, Dmitry Kobak, one of the study’s authors, finds the situation irrational, stating:
“In my opinion, something as important as writing your paper’s abstract should not be left to AI.”
