How to Judge Whether a News Story Was Machine-Written, and What a Detector Score Cannot Prove

Quick Answer

Judge a news story by what detectors cannot see: named bylines, on-record sources, first-hand detail and publisher disclosure. A detector score reads writing habits, never authorship.

One limit from the research I lead: scoring AI-drafted articles section by section shows which passages read machine-written. It cannot show who typed them.

Formulaic wire copy is the hardest case. A trained reporter following house style can score as AI, and an edited AI draft can score as human.

Key Points

• A detector score measures how predictable prose reads and holds no record of who wrote it; Turnitin stated a false-positive rate below 1%, yet a smaller Washington Post test found 50%.
• In the research I lead, four open detectors (desklib, Fakespot, Binoculars and Fast-DetectGPT) at a 1% false-positive rate flagged 0 of 223 held-out AI-assisted articles, so a clean score clears nobody.
• Origin rests on what a detector never sees: a verifiable byline, named on-record sources, first-hand detail and disclosures such as Substack’s “How I make this” statement.

Readers of wire stories, press releases and breaking-news briefs now face a plain question: did a person write this? A detector score looks like the fastest answer. On its own, it is the weakest one.

The question is spreading with the tools. In 2025, the Reuters Institute’s Digital News Report found that 7% of respondents used AI chatbots for news each week, rising to 15% among under-25s. The same report found that respondents expected AI to make news cheaper to produce by a net 29 points, yet less transparent and less accurate by a net 8 points each. Cheaper news means more copy to judge.

I built an AI content detector calibrated to avoid flagging honest human writers, and I would not let any detector settle that question alone. A detector reads writing habits. It cannot see a byline, a phone call or an on-record quote. When my research team scored AI-drafted client articles one section at a time, looking for the parts that read machine-written, the answer turned on what each section was built from, and that is something a reader can check without any software.

What follows sets out what a detector score measures, how edited AI drafts slip past it, and which checks on sourcing and disclosure still hold when the score points the wrong way, beginning with the habits a detector counts.

A publish date cannot show a page predates AI writing tools. My research team accepts human text only from Wayback Machine or Common Crawl snapshots captured on or before 31 December 2020.

A printed date is easy to trust and hard to check. Readers of news face the same trap from the opposite side. A detector score looks like proof, yet it reads writing habits and holds no record of who did the typing. A University of San Diego law library guide calls AI detectors “problematic and not recommended as a sole indicator of academic misconduct.” It also cites a 2023 Ars Technica report that detectors had flagged the US Constitution as AI-written. No chatbot wrote that text.

The opposite failure happens too. Edited AI drafts can score as more human than real human writing.

For a reader weighing a wire story or a press release, I keep two questions apart: who wrote it, and how it reads. A detector speaks only to the second. The first is settled by the byline, the named sources and the publisher’s disclosures, so the place to start is what a detector score is built to measure.

Top questions this article answers

• How can you tell if a news article was written by AI?
• What does an AI detector score actually measure?
• Can an edited AI draft pass an AI detector as human-written?

What does an AI detector score actually measure?

A detector score estimates how statistically predictable a passage reads compared with model output. It records no evidence of who typed the words, which sources were called or who edited the copy.

Before reading anything into a percentage, check three things:

• Which tool produced it, and what false-positive rate its maker claims.
• Whether the text is long and varied enough to measure, or a short formulaic brief.
• Whether the tool reports a confidence level or an estimated share of the text, since vendors define the number differently.

Turnitin has stated that its AI checker has a false-positive rate below 1%. A later Washington Post test, run on a much smaller sample, found 50%, according to a University of San Diego law library guide. The same guide notes that Turnitin accepts missing roughly 15% of AI-generated text to hold its false-positive rate down. Tuning a detector to spare human writers means letting more machine text through. For a closer look, see AEO Content on where AI detectors break.

The mechanism explains the spread. A Vegavid explainer describes a detector as a tool built to “calculate the statistical probability that the text was generated by an artificial intelligence model rather than a human.” A high score therefore means the text read as predictable. Rule-bound writing is predictable by design, and a wire brief built on an inverted-pyramid lede, attributed quotes and fixed phrasing is about as rule-bound as prose gets.

The common assumption is that a high score is a finding about authorship. AEO Content’s own testing points elsewhere. Its detector misread 4 of 2,868 pre-2021 Medium posts (0.14%) as AI, and the misses were neutral institutional prose, the register closest to syndicated news. Separately, four open detectors (desklib, Fakespot, Binoculars and Fast-DetectGPT), each held to a 1% false-positive rate, caught anywhere from 13% to 80% of ordinary AI text. Read together, these 5 sources point one way. The number tracks predictability, and predictability belongs to a genre as much as to a writer.

For a reader, that changes the question. A flagged market report or flood-watch brief may simply be doing its job well, which is why the harder case for any detector runs the other direction: a polished AI draft that someone has reworked by hand.

Can an edited AI draft pass an AI detector as human-written?

Yes, and by a wide margin. In one test, AI-assisted articles scored an AUROC between 0.22 and 0.40 across four open detectors, meaning the detectors ranked them as more human than human writing.

If a high score cannot convict a human writer, a low score might at least clear one. The evidence runs the other way. AUROC is a ranking measure: at 0.50 a detector cannot tell AI text from human text, and below 0.50 it places the machine text on the human side. The figures come from AEO Content’s tests of its own held-out articles, where desklib, Fakespot, Binoculars and Fast-DetectGPT, at the same strict false-positive threshold, flagged 0 of 223. Commercial tools such as GPTZero, Pangram, Originality.ai and Copyleaks were not part of that test. In plain terms, the open detectors were confidently wrong.

A section-level evaluation shows where the human reading comes from. Between 1 and 13 September 2026, the company scored 2,625 visible sections across 146 AI-drafted articles built for 13 client websites, one section at a time. In articles built on one client’s own sales records, record-heavy sections read AI 37% of the time, against 71% for everything else. Sections anchored in real records read far less machine-like.

Surface habits are even easier to change. AI-drafted articles carried 12.9 dash-punctuation marks per 1,000 words, against 0.8 in one firm’s pre-AI blog, and a dash budget brought new articles down to 1.69. Default model tone can also be prompted away, or aimed at samples of a writer’s own prose. Each of those is a habit a detector reads, and each can be edited out before publication.

Crude evasion still has limits. One overnight automated effort to beat a commercial detector reached a human verdict only by producing incoherent, ungrammatical text. Edited articles stay readable, and the human readings above came from the four open detectors alone.

For a news reader, the practical result is blunt. A clean score clears nobody. The useful evidence sits off the prose, in who reported the story, who said what on the record and where each number came from.

Sourcing, bylines and disclosure tell a reader more about a story’s origin than any detector score.

How can a reader tell whether a news story was machine-written?

Keep two readings apart: origin, meaning who wrote the story, and style, meaning how generic it reads. Detectors estimate style; only sourcing and disclosure speak to origin.

When I built a detector, I kept the origin reading and the style reading separate and never blended them into one number. A style reading helps an editor revise, because it shows which lines sound generic. It cannot settle authorship. Reviewers who screen freelance copy report that changing a single word can flip a whole piece between AI and original, which is a poor basis for any verdict about a person.

For any story, check five things the score never sees:

1. A byline you can verify. A named reporter with a findable record of earlier work.
2. Named sources and on-record quotes. People and documents you could contact or locate, quoted in words that appear nowhere else.
3. First-hand detail. Places, figures and records with a stated origin, which generic copy rarely carries.
4. A disclosure. A publisher note on how AI was used, or a writer’s process statement; Substack, for one, lets writers attach a “How I make this” statement.
5. Provenance on images and audio. Signed metadata under the C2PA standard attaches a record to the file itself.

Editors screening freelance submissions can go further. Ask for drafts with edit history, the sources behind each claim and a short call about the reporting. A writer who cannot explain the work beyond generic statements deserves a closer look.

Check the outlet, too. In 2023, NewsGuard found ads from more than 140 major global brands running on unreliable AI-generated news sites, and one of those sites averaged more than 1,200 articles a day with little apparent editorial oversight. That pace rules out phone calls and second sources. Volume, missing bylines and thin editing expose such a site faster than any single story’s score.

The opposite error has a cost as well. In 2024, Sam Gregory of WITNESS described a rapid-response task force that received three audio clips people claimed were deepfaked; its experts found the clip from Sudan authentic “almost without a shadow of a doubt.” Gregory warned that “the ease of calling “that’s deepfaked” on something real is increasing.” A detector flag on a reporter’s genuine copy works the same way, handing anyone who dislikes the story a ready reason to dismiss it.

Will a detector score ever prove who wrote a story?

Not on its own. Scores will keep reading style, so proof of authorship will rest on bylines, named sources and publisher disclosure, the checks readers already trust for facts.

Audiences already lean that way. In 2025, the Reuters Institute’s Digital News Report found that 58% of respondents were concerned about their ability to tell true from false in online news. Respondents also expected AI to make news less trustworthy by a net 18 points. When they wanted to check a claim, they named trusted news brands and official sources most often. Readers go to the source first.

The research I lead points the same way. Four open detectors (desklib, Fakespot, Binoculars and Fast-DetectGPT) ranked AI-assisted articles built by my company as more human than real human writing. Commercial tools such as GPTZero and Pangram were not part of that test. I expect a style score to separate fewer stories as drafting tools improve. So when a story reads unusually smooth, I would check the byline and the sourcing before running any scanner, then ask the publisher how the piece was made.

Frequently Asked Questions

What else do people ask about spotting machine-written news?

Readers ask how accurate detectors are, whether a prompt change can fool one, who gets wrongly flagged, whether old texts get flagged and how audiences feel about AI-made news.

How accurate are AI detectors on news articles?

Published accuracy figures are measured on each tester’s own samples, and none cited in this article came from wire copy. In a December 2025 thread on Reddit’s r/ChatGPT, one user reported that two popular detectors “gave random scores” on the same AI-assisted draft, and a commenter called all the detectors “essentially random number generators.” That was one informal test of one draft, with no sample size. It still shows the risk a reader faces: two tools can return different verdicts on identical text.

Can a small prompt change fool an AI detector?

Often, by one practitioner’s account. Cat Casey, chief growth officer at Reveal and a member of the New York State Bar AI Task Force, reports fooling detectors 80 to 90% of the time by adding the single word “cheeky” to her prompt, according to the University of San Diego law library guide. The guide lists paraphrasing, added emotion or anecdotes, and humanizer tools as other common causes of missed AI text. Nothing in a published story shows that one word, so a reader cannot see it either.

Why do non-native English writers get flagged as AI?

Their writing often relies on repeated phrases, terms and words, which detectors read as predictable. A false positive is human writing that a detector calls AI. The University of San Diego law library guide cites a May 2023 Stanford HAI article and a July 2023 paper by Liang and colleagues in the journal Patterns, and reports that students writing in English as a second language, along with neurodivergent students, are flagged at higher rates than native English speakers. I would expect the same risk for wire copy filed by correspondents writing in a second language, though none of these sources tested news text.

Can a detector flag writing that predates AI tools?

Users report that it can. In a December 2024 thread on Reddit’s r/writers, one commenter said detectors rated classics, and text the commenter had written, as containing “lots of AI text,” even when the writing predated AI. In a December 2025 r/content_marketing thread, another commenter said human writing gets flagged “because it is written properly.” These are user reports rather than controlled tests. I read them as the same warning that applies to news: formal, rule-bound prose is the kind most easily misread.

Do readers trust news written mostly by AI?

Many do not. A 2024 survey of 2,000 people in each country found that 52% of US respondents and 63% of UK respondents were uncomfortable with news “predominantly produced by AI.” I read that as a reason for publishers to state plainly how a story was made, since a score run by a reader will not settle it.