Does Grammarly AI detection actually work?

The 99% Claim vs. the 22% Reality: A Contradiction Worth Understanding

You paste in a paragraph, Grammarly tells you it's "99% accurate," and a competing test on the very same benchmark family says the detector catches barely one in five AI-written samples. Both claims are on the record — and both can't be right as stated.

Grammarly markets its detector as achieving 99% detection accuracy and ranking "#1 on RAID's independent benchmark," a claim its blog repeats under "large-scale standardized evaluation." Yet an independent-style test using the same RAID dataset found Grammarly scored an F1 of just 0.364, with recall of 0.222 — effectively 22.22% accuracy across 99 samples spanning 9 adversarial attack types and 11 AI models.

That gap isn't a rounding error; it's a 77-point chasm between marketing and measurement. The available sources don't resolve the discrepancy — different test conditions, dates, or methodologies may explain it — but the practical takeaway for anyone evaluating the tool is unavoidable: the headline number is contested, and you should know who's making each claim.

Here's what makes the contradiction matter most:

• The 99% figure is self-reported by Grammarly; the 22.22% figure comes from a competitor (Originality.ai) with its own commercial stake in the outcome.
• Both tests reference the RAID benchmark family, so a casual reader could reasonably assume they're measuring the same thing.
• No fully independent third party has publicly adjudicated the two claims in the available research.

The most honest voice in this debate may be Grammarly itself. Its own support documentation states plainly that "currently there is no AI detector that can conclusively or definitively determine whether AI was used to produce text," and that its score "should not be used as an objective source of truth." The company even describes its model as merely "directionally accurate."

That's a rare admission against interest, and it reframes everything. A Grammarly AI score is a screening signal, not proof — a flag worth investigating, never standalone evidence for an academic misconduct ruling or a hiring decision. Independent testing echoes this: as one reviewer put it, "a score of 92% does not mean there is a 92% chance the content is AI-written."

This is the same skepticism SkillzBench applies to AI tooling generally: verify claims with your own hands-on testing rather than trusting vendor benchmarks. If you work with AI agents, the same principle applies to the skills that power them — SkillzBench is building a free directory where every skill gets benchmarked and security-scanned so you can see real Lift, not marketing numbers.

Where the Detector Works — and Where It Falls Apart

Imagine a detector that nails 89% of raw AI text with zero false accusations against human writers — and then misses 94% of the AI content people actually submit. That's the paradox at the heart of Grammarly's AI detection, and the numbers explain why.

According to independent testing, Grammarly flags unedited AI-generated text 89% of the time while maintaining a 0% false positive rate on human writing. That second number is genuinely impressive — Grammarly deliberately optimizes for it, telling users that wrongly accusing a human writer is worse than missing AI text. The trade-off is a flood of false negatives, which one reviewer calls "the worst failure mode because they create a false sense of security."

The detector falls apart on paraphrased content. The same testing found human-paraphrased AI content was caught just 6% of the time — and as the reviewer bluntly puts it, "students never send raw ChatGPT output." Research on AI-generated college essays reinforces this: simple prompt editing reduced detection from 100% to 13%.

Scores also swing wildly depending on what you feed it:

• AI academic essay: 97% detected
• AI social media post: 92% detected
• AI creative writing: 57% detected
• Human-paraphrased AI article: 6% detected

You cannot apply one threshold across content types and trust the result.

Two deeper problems compound this. The same MIT Sloan research found 61% of TOEFL essays by non-native English speakers were wrongly flagged as AI across seven detectors, versus roughly 5% for native writers — a systemic bias Grammarly itself acknowledges. And the industry's best-funded player gave up entirely: OpenAI shut down its own detector due to poor accuracy.

Grammarly's own guidance is the honest takeaway: the score is "directionally accurate," not an objective source of truth. Treat it as a screening signal, never proof. The same skepticism applies to any AI tool you adopt — verify before you trust, which is exactly the principle SkillzBench is building its free, independent skills directory and MCP connector around: security-scanning and benchmarking every agent skill before it touches your workflow.

The Trap Nobody Warns You About: Grammarly's Own Rewrites Get Flagged

Here's the paradox at the heart of this tool: Grammarly's own AI rewriting features can produce text that gets flagged as AI-generated — sometimes by Grammarly's own detector. The company's support documentation admits it directly, noting that rewrites from its Paraphraser, Humanizer, and other agents come from its own LLM and will likely be flagged, and advising users to avoid those features if they're worried about detection.

The numbers back this up. A study of 500 files by Copyleaks found Grammarly's "Improve" feature produced text flagged as AI 31.6% of the time, while standard grammar corrections were detected as human in 1,000 out of 1,000 files. A smaller test of ten samples found the same split: light grammar edits had minimal impact on detection, but rephrase and rewrite features consistently triggered AI flags.

That distinction matters enormously for anyone using Grammarly at work or school. The safe features and the risky ones sit side by side in the same interface:

• Grammar and spelling corrections: tested safe, passing as human in every file examined
• Rephrase and rewrite agents: LLM output that detectors — including Grammarly's own — can flag
• Humanizer features: despite the name, these generate fresh AI text, not human-verified prose
• "Improve" suggestions: flagged as AI roughly one in three times in Copyleaks' testing

There's also a subtler trap hiding in the data: the false-negative trade-off. Grammarly deliberately tunes its detector to minimize false positives, explaining that wrongly accusing a human writer is more damaging than missing AI text. That's a defensible design choice, but it means a low score doesn't prove your text is clean — it may simply mean the detector is being conservative. As one reviewer put it, false negatives are the worst failure mode because they create a false sense of security. A writer who runs AI-drafted copy through Humanizer, sees a reassuringly low score, and ships it may never realize the detector simply missed what its own company created.

The practical takeaway is straightforward. If provenance matters — a client deliverable, an academic submission, a bylined article — treat Grammarly's rewriting agents as AI tools, because that's exactly what they are, per Grammarly's own guidance. Keep original drafts, be transparent about which features you used, and never let a low detection score substitute for knowing where your text actually came from. The same verification-first mindset applies across AI workflows generally: at SkillzBench, we're building free benchmarking and security scanning so you can see what agent skills actually do before you trust them with your work.

How to Use AI Detection Without Getting Burned: A Practical Playbook

A detector that catches only 6% of paraphrased AI content — while missing nearly everything students and employees actually submit — isn't a verdict machine. It's a smoke alarm: useful when it goes off, meaningless when it stays silent. Here's how to use AI detection without getting burned.

Treat scores as signals, never as verdicts. A 92% AI score does not mean there's a 92% chance the text was machine-written; it's a statistical pattern read, as independent testing notes. Grammarly itself says its detector is "not an objective source of truth" and should serve as "a starting point for review rather than as final proof of authorship," per its own support documentation. For high-stakes decisions — misconduct hearings, contract disputes — pair any score with plagiarism checks, citations, and version history.

Skip the rewriting tools if you need to sound human. Grammarly's own documentation warns that text from its Rephrase, Rewrite, and Humanizer features will likely be flagged as AI, because rewrites come from Grammarly's LLM. The numbers back this up: Copyleaks testing found the genAI "Improve" feature produced AI-flagged text 31.6% of the time, while standard grammar corrections were detected as human in 1,000 out of 1,000 files. Basic proofreading is safe; generative rewriting is not.

Protect yourself with paper trails and disclosure. Keep original drafts, clarify AI-tool expectations with clients or instructors upfront, and disclose what you used — best-practice guidance from Originality.ai recommends exactly this. Version history is the one form of authorship evidence that never produces false positives.

Write policies instead of policing scores. The most striking data point in this space isn't about detection at all: a Tyton Partners study of 3,000+ respondents across 750+ institutions found that only 13% of institutions relying on bans considered their AI policy effective, versus 30% at institutions that integrated AI into workflows. Clear rules about acceptable use consistently outperform enforcement through detection.

A practical playbook looks like this:

• Use detector scores as one input among several — never sole evidence for accusations or penalties.
• Be extra cautious with non-native English writers, whose TOEFL essays were wrongly flagged as AI 61% of the time across seven detectors.
• Stick to standard grammar corrections; avoid AI rewriting features when human-sounding output matters.
• Replace blanket bans with explicit, integration-friendly AI policies.

The same "trust but verify" logic applies wherever AI touches your workflow. If you use AI agents for writing or research tasks, the tools and skills behind them deserve the same scrutiny you'd apply to a detector score — SkillzBench is building exactly that kind of verification, with security scanning, version pinning, and sandbox benchmarking for agent skills, all free to browse and test.

Beyond Detection: Verifying AI-Assisted Work the Transparent Way

The deeper problem isn't whether Grammarly's detector works — it's that we're asking a statistical guess to answer a trust question. A detector score tells you what a model thinks about patterns in text, not what actually happened to your content. Even Grammarly admits the score "should not be used as an objective source of truth" and that no detector can definitively determine whether AI was used.

That admission matters more as AI moves from a single tool to entire automated workflows. When an AI agent drafts, rewrites, and edits your content using a chain of skills and tools, no after-the-fact detector can reconstruct what happened. The evidence has to be built into the process, not inferred from the output.

MIT Sloan's guidance points the same direction: instead of policing with detection, teams should establish clear AI policies and transparency about tool use. In practice, that means knowing exactly which tools and instructions touched your text — and being able to verify them with evidence rather than marketing claims.

For teams building agent-driven writing workflows, that verification layer is what SkillzBench is building. Agent skills — instruction packages that extend tools like Claude Code, Cursor, and Gemini CLI — are proliferating fast, but published research found roughly 26% of them carry at least one vulnerability. You wouldn't run unvetted software in your stack; the same standard should apply to the skills your writing agents use.

SkillzBench's approach replaces trust-me marketing with testable evidence:

• Verify — every skill is pinned to a specific version, license-checked, and security-scanned before it earns a place in the directory.
• Benchmark — each skill is tested in a sandbox with and without the skill, and the measured Lift in percentage points is published separately from any rating.
• Equip — a free MCP connector lets you toggle exactly which skills and tools each agent can see, so provenance is a choice you control, not a black box.
• Stay current — upstream changes trigger re-scans and re-benchmarks, so what you vetted is what you keep running.

The lesson from AI detection applies here too: scores without context create a false sense of security. A detector's 6% catch rate on paraphrased content shows what happens when verification is bolted on after the fact instead of designed into the workflow. Whether you're evaluating a detector, a rewriting agent, or a skill package, demand the same thing — reproducible evidence of what the tool actually does.

SkillzBench's free skills directory, benchmarking data, and MCP connector are being built now, with the waitlist live. If your team is automating writing workflows with agents, vetting what you connect is the transparency layer no detector score can replace.