Can AI algorithms are not ready to replace science editors?
Anyone familiar with cell culture will instantly recognise the issue in this excerpt. “Roswell Park Memorial Institute 1640” (commonly “RPMI 1640”) refers to a cell culture medium named after the institute where it was developed in the 1960s. It’s highly unlikely that researchers based in Japan cultured their cells there!
So, how did such a statement find its way into the Methods section of a research manuscript? Well, this is just one of several glaring errors introduced by an artificial intelligence (AI)-based editing tool we tested.
AI: A Buzzword with Big Promises
AI is certainly the buzzword of the year, making its mark across sectors with promises of transformative potential. For scientists, one of the most exciting prospects is AI’s ability to extract meaningful insights from big datasets—a capability poised to revolutionise healthcare in the coming years. However, AI currently risks overpromising, with developers claiming their algorithms can handle tasks they’re not yet ready for, at least in untrained hands.
Exploring AI Tools for Scientific Editing
Over the past few months, I trialled several AI applications that made bold claims in the editing arena—perfect grammar, optimised sentences, native-level English, and even advanced features like paraphrasing, plagiarism prevention and editorial reports. These features are particularly appealing given that most scientific articles (up to 98%) are published in English, despite fewer than 20% of the global population being native speakers.
To fully explore these tools, I had to commit to paid subscriptions, as free versions were significantly limited. I approached this investment with optimism, hoping the results would justify the cost. To ensure fairness and avoid endorsing or discouraging specific tools, I’ve anonymised the names of the AI services tested. If you’re interested in learning more about any of them, feel free to reach out privately for details. Read on for an overview.
Anonymised AI Tools: The Good, The Bad, and The Tedious
Tool 1: Initially promising with extra features like editorial reports, but these turned out to be basic grammatical summaries, tediously listed one by one. Its MS Word add-in required manually accepting each change, massively slowing the process and limiting its utility.
Tools 2 and 3: Provided tracked changes in MS Word, making them more user-friendly than Tool 1. However, they introduced conceptual and grammatical errors, didn’t refine text for wordiness, and failed to follow conventions of scientific writing in the biomedical sciences. Overall, a disappointment.
Tool 4: The simplest of all, delivering quick edits to a document of thousands of words in under a minute, but limited to few, minor grammatical tweaks without addressing substantial issues.
Tool 5: A hybrid of AI and human editing at a premium cost. This one I thought had the potential to be a winner. Unfortunately, the human editing offered little beyond basic grammar corrections and even introduced errors without adding comments to flag unresolved issues. For example:
In this excerpt, edits were made without any accompanying comments, implying the final text was correct. However, the edited sentence remains grammatically incorrect, poorly constructed, and unclear in meaning.
The Risks of Over-Reliance on AI
These experiences highlight the need for caution when using AI for editing, as it seems that even costly premium services often fall short of the nuanced demands of scientific writing. While AI tools promise polished grammar and language, I found that they mainly address surface-level issues like punctuation or typographical errors, akin to standard grammar checkers, and require manual oversight to ensure accuracy. Errors introduced by AI—or even human editors—can critically alter meaning, underscoring the importance of thorough review and sound judgement in scientific editing. For example:
Without reading the edited text with the tracked changes showing, you might not readily spot this error in meaning. The authors intended to say that Telmisartan is a partial PPAR-γ agonist, but the text has been edited to say that Telmisartan was chosen partly because it is a PPAY- γ agonist. In this case, the sentence would be more accurate if it had been left alone!
Evolving but Not Yet Ready
So what’s my take home message? AI algorithms for editing scientific content are still evolving. While they show great potential for improving grammar and language accuracy, and many are financially accessible, the tools I tested are far from perfect. Polished grammar, while important, is rarely the deciding factor in effectively pitching your manuscript or funding proposal. Success lies in how you present and frame your findings—with accuracy, clarity, and a compelling narrative.
If you do choose to use AI for editing or writing, proceed with caution and thoroughly review the results. Additionally, when using AI generative tools, remember that the phrasing of your commands significantly influences the output. “Talking” to AI effectively is a skill that requires practice and precision.
—