Some writers worry that AI detection software could falsely flag their work, potentially ruining reputations or job opportunities before they have a chance to prove the detectors wrong.
karetoria/Getty Images
hide caption
karetoria/Getty Images
Ghostwriter Sally McGraw was hoping to land a job developing a book proposal for a prospective client. But after submitting a writing sample, she got an unexpected response: an email telling her that multiple artificial intelligence detection tools had flagged her work as machine-generated.
“I was floored because I had not used AI to generate the text,” McGraw said. “I’d written it all myself.”
Having spent more than 20 years as a professional nonfiction writer, McGraw said she was angry that her work had been dismissed as AI-written.
“You know where you see a cartoon character going red from their chin all the way up to the crown of their head? I felt my ears starting to burn,” she said. “I thought, what is going on?”
McGraw said she assured the client she had written the proposal herself, but decided not to press the issue because she considered the job “minor.”
Sally McGraw, a ghostwriter, said she’s changed her writing style after a prospective client suggested her work was AI-generated.
Sally McGraw
hide caption
Sally McGraw
In an email to NPR, the client confirmed that she had screened McGraw’s work for AI. But she said the results were not the determining factor in choosing to hire a different writer, instead citing experience and fit. McGraw says she was paid for her writing samples, but believes the detector results cost her the opportunity to write the full book proposal.
As editors struggle to keep up with a growing influx of AI-generated content, some are turning to detection tools to help sift through the mounting number of submissions. But these tools aren’t perfect: They’ve been known to falsely flag human-written work as AI, which can leave writers with little idea of what triggered the result and how to prove otherwise.
“Everyone knows that these screeners are not 100% accurate,” said McGraw.
Editors are overwhelmed by AI-written submissions
Neil Clarke, publisher and editor of Clarkesworld, says a flood of AI submissions at the science-fiction magazine has put pressure on the editing staff.
In 2021, the publication received roughly 1,100 submissions a month. By February 2023, after AI agents like ChatGPT, Claude and Gemini started to become widely available, that number had doubled, forcing Clarke to temporarily close its submission portal. He estimates his own workload has increased by about 30%.
“It’s introduced an element of mistrust into the process,” he said.
When AI writing detectors disagree
While AI detectors have gotten more sophisticated in recent years, the companies behind the detectors acknowledge that the tools aren’t foolproof. Researchers have found that different AI detection tools can reach dramatically different conclusions about the same piece of writing.
In a study published in June, in the International Journal for Educational Integrity, researchers compared the accuracy of four widely used detectors. They tested 160 academic papers comprising four different categories: entirely human-written, entirely AI-written, a combination of the two, and AI material rewritten by a human.
Among the four detectors, Pangram was the most consistently accurate, correctly identifying the use of AI in 37 of the 40 papers with the combined text and in 37 of the 40 papers where a human rewrote AI-generated text.
Pangram CEO Max Spero, in an emailed statement, said his company’s detector can make mistakes, particularly when there is not much writing to evaluate.
Previously published writing can pose an additional problem, says Chris Callison-Burch, a professor of computer and information science at the University of Pennsylvania.
“Many AI detectors work by asking a large language model how predictable a passage is,” Callison-Burch said. “The catch is that the language models also tend to find text from their training data unusually predictable.”
That could explain what science blogger Janelle Shane encountered in late 2022 when she tested excerpts from her 2019 book about how AI works, You Look Like a Thing and I Love You. One detector reported a 98.7% confidence that a passage was machine-generated.
“This is a clear, glaring, blinking-light false positive,” Shane said. “I had written this text myself.”
An unpublished draft like McGraw’s writing sample wouldn’t face that particular problem, Callison-Burch said, but could still be falsely flagged based on patterns in the writing style.
Detectors look for clues in word choice, punctuation and the way sentences are put together. The word “delve,” frequent em dashes, and ideas grouped in lists of threes can trigger an AI-positive result, he said. So can opening sentences with adverbs such as “notably” or “crucially,” or posing a question immediately followed by answering it.
At the same time, those are familiar habits for human writers. Callison-Burch says detectors weigh patterns across a whole piece, so an em dash or a particular word alone doesn’t necessarily mean that a bot was used.
What’s at stake for writers
The Authors Guild, an organization that advocates for writers’ rights, said in May that the use of AI detectors “creates an environment reminiscent of witch hunts,” with writers worried about false accusations and how to defend themselves.
“A false positive can cost an author their publishing contract and reputation, and a false negative, when AI-generated work slips through reputable publishing undetected, can cause readers to lose their trust in authors and the industry,” the Guild wrote.
Clarke, the sci-fi magazine publisher, sits somewhere in the middle. He says that while he doesn’t personally use AI detectors to screen incoming manuscripts, he thinks certain detection tools could help overwhelmed editors manage the volume, provided they check the detector’s findings. But he cautions against turning suspicions into accusations that could damage a writer’s reputation.
“Submissions should end in rejection, not an accusation,” he said.
McGraw says the fear of another accusation has changed how she writes. She says she now avoids using certain words and sentence constructions, and occasionally leaves out commas she used to include, hoping the inconsistency will help prove that a human wrote the text.
“I miss my em dash. It was something that I learned in journalism school and I’ve used it for decades,” she said.
The ghostwriter also sees the irony. Large language models learn from human prose, yet she says feels the pressure to make hers look less like AI.
“It’s like trying to fool a machine into thinking I’m not a machine,” McGraw said.
And she worries about facing another accusation when a bigger opportunity is on the line.
“I’m bracing for the time when it’s an assignment that I’m really gunning for and the outcome makes or breaks me,” she said.
