How Can Teachers Tell If You Use AI?

The Detection Myth: Why Teachers (and Tools) Get It Wrong

If you asked a teacher to pick the AI-written essay out of a pile, you'd hope they'd have a better-than-even shot. They don't. In two randomized-controlled experiments, neither novice (N=89) nor experienced teachers (N=200) could identify ChatGPT-generated texts among authentic student essays — and both groups were overconfident in their judgments. A separate study of 289 teachers found detection ability no better than chance, regardless of experience level.

The software doesn't save the day, either. Across 805 controlled writing samples, detection tools averaged 39.5% accuracy on unmodified AI text, according to education researcher Leon Furze. Apply basic paraphrasing and accuracy collapses to 22.1% — one tool dropped a leading detector to 4.6%. Worse, the same testing found a 15% false accusation rate on essays written entirely by humans.

Part of the problem is that the "tells" teachers have been taught to look for don't hold up. Em dashes and the word "delve" are unreliable myths, per the UK's Jisc National Centre for AI, since plenty of human writers use both. Humans only spot AI work with any consistency when they already know the writer's usual style — which is rarely the case with a stack of unfamiliar essays.

And here's the uncomfortable twist: training makes it worse. JISC found that teaching staff to spot AI work "significantly increased false positive rates" — confidence goes up, accuracy doesn't. That confidence gap carries real consequences, because the people most likely to be wrongly flagged aren't cheaters at all:

• 61% of essays by non-native English speakers were falsely flagged as AI-generated, versus near-perfect accuracy on native-speaker essays
• 20% of Black teenagers reported having work incorrectly flagged, compared with 7% of white teens
• Free and lesser-known detectors showed false positive rates as high as 50%, while even the best tools sit at 1–2%

The scale math is brutal. Jisc estimates that a 20,000-student institution running 480,000 assessments a year at a 1% false positive rate would generate roughly 4,800 false accusations annually. It's no surprise that at least 12 elite institutions — including Yale and Johns Hopkins — have disabled AI detection entirely, with 60+ more across five countries following suit.

This is why the expert consensus has shifted from playing detective to redesigning assessment around visible process — revision histories, drafts, and reasoning rubrics. The same principle applies beyond classrooms: whether you're evaluating student work or AI agent skills, verified evidence beats gut feel. That's the thinking behind SkillzBench, which is building free security scans, version pinning, and published benchmark Lift scores for agent skills — join the waitlist to follow along as it launches.

The False Positive Trap: When Accusing the Wrong Student Is the Real Risk

Imagine being told your carefully written essay "looks AI-generated" — because English isn't your first language. For thousands of students each year, that's not a hypothetical. It's the reality of detection-first academic integrity policies, and the data shows the people most often wrongly accused share a few common traits.

The numbers are stark. Stanford research found that 61% of essays by non-native English speakers were falsely flagged as AI-generated, while native-speaker essays were assessed with near-perfect accuracy. Across seven detectors analyzing TOEFL essays, the worst performer flagged 97% of non-native writing as machine-made.

The disparities don't stop there. According to Common Sense Media data, 20% of Black teenagers had work incorrectly flagged as AI-generated, compared with 10% of Latino teens and just 7% of white teens. As education researcher Leon Furze argues, detection tools "privilege students who are English first language, have access to paid large language models, and are more digitally literate" — turning a supposedly neutral technology into an equity problem.

Even at low error rates, scale makes wrongful accusations inevitable. One Jisc analysis calculated that a 20,000-student institution running 480,000 assessments a year would generate roughly 4,800 false accusations annually at just a 1% false positive rate. Nationally, that math suggests around 223,500 wrongly flagged essays among U.S. first-year students each year.

That's why Jisc's conclusion inverts the usual framing: "a low false positive rate is far more important than an overall high detection rate." A missed detection costs little; a false accusation costs a student their trust, their record, and sometimes their future. The University of Pittsburgh's Teaching Center cited exactly those risks — loss of student trust and motivation, bad publicity, and potential legal sanctions — when rejecting detector use.

Here's the counterintuitive part: training teachers to spot AI writing makes things worse, not better. JISC reports that such training "significantly increased false positive rates," and controlled experiments show even experienced teachers detect AI text no better than chance. Sharpening suspicion without improving accuracy just produces more confident — and more frequent — wrongful accusations.

The safer path is to stop playing detective. Assessment designs that make thinking visible — process documentation, revision histories, and rubrics that reward reasoning over polished output — reduce reliance on unreliable judgment calls altogether. For educators and institutions building such workflows, tools like SkillzBench are emerging to help teams find and verify AI agent skills that automate supporting tasks such as rubric drafting and revision-history analysis, with every skill security-scanned and benchmarked before use.

Until detection improves dramatically, the calculus is clear: the cost of accusing the wrong student will always outweigh the cost of missing one who used AI.

From Detective to Designer: Reading Evidence Instead of Guessing

If you can't trust your gut or a detector score, what can you trust? The answer emerging from the research is surprisingly simple: evidence, gathered the way any good investigator gathers it — by comparing the suspect work against something known.

The most reliable starting point is the student's own writing baseline. Jisc's National Centre for AI notes that humans are only decent at spotting AI-written work when they already know the writer's usual style. That means the tell isn't "this sounds polished" — it's "this student who usually makes comma splices is suddenly producing flawless syntax." A dramatic, unexplained jump in sophistication relative to a student's in-class writing is legitimate evidence; a generic impression of polish is not.

There are also stylistic patterns worth noting, cautiously. AI-generated text tends toward uniform sentence length (clustering around 15–25 words, versus natural human variation of 5–10 and 30–40+), excessive hedging ("might," "could," "arguably"), overused transition words like "however" and "furthermore," and a "perfect but hollow" quality with no personal insight or natural errors. But Jisc cautions that these "tells" are largely unreliable myths — they also appear in authentic student work, so treat them as conversation-starters, never verdicts.

What about detector scores? Treat them as probability estimates, nothing more. Even Turnitin acknowledges that its detection "may not always be accurate" and is not proof of anything. The numbers back that up: across 805 controlled tests, average detection accuracy was just 39.5% on unmodified AI text, dropping to 22.1% after simple paraphrasing, with a 15% false accusation rate on human-written samples. Free and lesser-known tools fare worse — some showed false positive rates as high as 50%, while the best mainstream tools sit around 1–2%. In a school with 480,000 assessments a year, even a 1% error rate means roughly 4,800 false accusations annually.

Before trusting any tool, test it yourself. Pangram's advice is blunt: "Test, test, test — don't pay money until you've tested." Run samples of writing you know the true origin of, both human and AI, and check the false positive rate on the human samples first, since that's where the real damage happens. One tool in independent testing flagged 100% of human-written texts as AI.

• Compare suspect work against the student's known in-class baseline, not generic "tells."
• Treat stylistic patterns as signals to investigate, never as proof.
• Never act on a detector score alone — JCQ guidance says detection should form part of a holistic approach.
• Test any detector on verified samples before trusting it, prioritizing a low false positive rate.

The same verification-first mindset applies beyond the classroom. Anyone building AI agents or workflows should evaluate skills the same way — checking provenance, security, and real performance before trusting them. SkillzBench is building exactly that: a free directory where every agent skill is version-pinned, security-scanned, and benchmarked with published Lift scores, with the waitlist open now.

Redesigning Assessment So the Question Answers Itself

If you can't reliably catch AI-written work — and the evidence says teachers can't — the logical move is to stop playing detective and change what you're assessing. That's exactly where the field is heading. The mood in higher education has "shifted decisively" away from automated detection, with many institutions now relying on human judgment alone unless assessments are substantially redesigned, according to Jisc's National Centre for AI.

The most ambitious example is the University of Surrey, which is redesigning every degree program from September 2026 to assess student process rather than final output. The principle is simple: if the essay can be outsourced, make the thinking behind it visible and gradable.

Redesign takes several practical forms:

• Process-based tasks that require students to document hypothesis development and decision-making at each stage, not just submit a polished result.
• Revision history review, where the trail of drafts, edits, and timestamps becomes evidence of genuine work — or its absence.
• Rubrics that assess reasoning rather than the final product, rewarding how students justified choices instead of how fluent the prose is.
• Personal reflection components that tie the work to the student's own context and experience.

These approaches come from research synthesis of what actually works when detection fails. But redesign isn't a silver bullet. Scarfe et al.'s "Turing Test" study found AI submissions averaged just over half a grade boundary higher than real students' work, and Hardie et al. found 17 assessment types were "generally not robust in the face of GAI" — even the most resilient formats like reflection and observation could still be passed by AI. The lesson: redesign must specifically target visible thinking, not just novelty of format.

There's a practical workload problem here, too. Drafting AI-aware rubrics, building process checkpoints, and reviewing version histories for every assessment is real labor on top of existing teaching loads. This is where agent skills — instruction packages that extend AI coding agents and assistants — can help educators automate the routine parts, such as generating rubric drafts aligned to reasoning-focused criteria or summarizing revision histories into reviewable timelines. SkillzBench is building a free, security-scanned directory of these skills, with version pinning and benchmarking so educators can see which skills actually help before using them; the waitlist is live for those who want early access.

Redesign doesn't answer "did this student use AI?" — it makes the question less important. When the grade depends on demonstrated process and reasoning that unfolds in front of the assessor, a polished final document alone stops being sufficient evidence of anything.

A Practical Playbook for Teachers Facing Suspected AI Use

The evidence is uncomfortable: a peer-reviewed study found that neither novice nor experienced teachers could reliably identify ChatGPT-generated essays, yet both groups were confident they could. Acting on a hunch — or a detector score — is therefore the riskiest move a teacher can make.

The playbook starts with a hard rule: never accuse on a score alone. Turnitin itself acknowledges its detection scores are probability estimates, not proof. JCQ guidance says detection "should form part of a holistic approach," and the stakes justify that caution — false positive rates range from 1–2% on the best mainstream tools to 50% on some free ones, per Jisc's analysis.

A practical protocol looks like this:

• Start with a conversation, not a confrontation. Ask the student to explain their argument, define a key term, or walk through how they developed a section. Understanding the work is evidence; a detector percentage is not.
• Check the writing baseline. Jisc notes humans only spot AI reliably when they know the writer's usual style. A student who "usually makes comma splices suddenly producing flawless syntax" is a meaningful signal — generic tells like em dashes and the word "delve" are not.
• Use a detector as one signal, if at all. Test any tool before trusting it, and prefer low false-positive options. Across 805 samples, mean accuracy was 39.5% unmanipulated and 22.1% after simple paraphrasing, with a 15% false accusation rate on human-written work.
• Watch the equity risk. 61% of essays by non-native English speakers were falsely flagged as AI-generated in one study, and 20% of Black teenagers reported being incorrectly flagged. A false accusation damages trust far more than a missed case.
• Communicate the AI policy transparently. Around 53% of students are scared to use AI even for legitimate support, fearing accusations — and only 18% actually submit AI text directly. Clear rules reduce both fear and misuse.

One caution: don't rely on training yourself to spot AI prose. Research found that detection training "significantly increased false positive rates." The sustainable shift is toward assessment redesign — process-based tasks, revision history review, and rubrics that assess reasoning rather than final output, the approach the University of Surrey is applying across all degree programs from 2026.

For educators and administrators building AI-aware workflows — rubric drafting, revision-history processing, policy documentation — agent skills can automate much of the heavy lifting. SkillzBench's free directory lets you discover and evaluate skills that are version-pinned and security-scanned; the waitlist is open for educators who want vetted building blocks rather than guesswork.

ctaText: Join the SkillzBench waitlist — free, independent verification for the agent skills you build with. socialProofText: Every skill on SkillzBench is version-pinned, security-scanned, and benchmarked with published Lift scores — and it stays free.