Is This Picture AI Generated?

Why Spotting AI Images Is Harder Than Ever

The six-fingered hand is dead. If you're still using 2023-era detection tips to judge whether an image is AI-generated, you're not just behind the curve, you may be actively worse than someone who admits they don't know.

The classic tells have quietly disappeared. Major models like Midjourney and DALL-E now render anatomically correct hands and accurate text, according to the Global Investigative Journalism Network's detection guide. The guide recounts a seasoned fact-checker declaring an AI image "authentic" precisely because it showed a correct five-finger hand, misplaced certainty that is, in the guide's words, "more dangerous than honest uncertainty."

The numbers back this up. A large-scale academic benchmark spanning 2.6 million images, 291 generators, and 23 detector variants found the best detector achieves only 75.0% mean accuracy, and the worst manages just 37.5%. More striking: modern commercial generators like Flux Dev, Firefly v4, and Midjourney v7 evade most detectors entirely, with detection accuracy of only 18–30%.

• 64 of 291 tested generators (22%) defeat most detectors, scoring under 50% average detection accuracy
• Detector rankings are wildly unstable across datasets (Spearman ρ: 0.01–0.87), a top method on one dataset can drop nearly 20 positions on another
• Prior work shows accuracy can fall from over 95% to under 50% when detectors face unseen datasets

The benchmark's authors are blunt: no universal winner exists, and practitioners must select detectors based on their specific threat landscape rather than trusting published performance. Vendor claims deserve extra skepticism, Hive reports 98.03% accuracy on its own blog, but that figure is vendor-reported on its own product, a meaningful conflict of interest against the independent 75% ceiling.

The core problem is that detection has shifted from identification to risk assessment. Every verdict is a probability, not a certainty, and a "no signal" result from provenance tools like OpenAI's Verify means nothing, metadata can be stripped, watermarks degrade, and the tool only recognizes OpenAI content in the first place.

This mirrors a broader lesson for anyone deploying AI tools: published benchmarks rarely survive contact with your actual use case. It's why SkillzBench is building its directory to test every agent skill in a sandbox, with and without the skill, before publishing results, rather than taking upstream claims at face value. Honest uncertainty beats confident wrongness, and the same principle applies whether you're evaluating a detector or a workflow automation.

What Still Works: Physics, Geometry, and Provenance Signals

Here's a counterintuitive truth: while AI has learned to draw perfect hands, it still struggles to understand how light behaves in the real world. The reason is structural, AI assembles images like a collage artist, not a photographer, stitching together plausible fragments without a coherent physical model underneath.

That distinction gives you three checks that remain surprisingly effective, validated by academic GradCam analysis showing structural distortions in AI images:

• Vanishing point analysis: In real buildings, parallel lines, rooflines, window rows, floor edges, converge to a single vanishing point. AI often generates lines pointing in inconsistent directions.
• Shadow consistency: In any scene with one light source, all shadows must point away from that source. AI frequently renders shadows falling in multiple directions within the same image.
• Reflection verification: Lines connecting objects to their reflections should converge to a single vanishing point and meet reflective surfaces at right angles. AI-generated reflections often violate both rules.

These geometric checks work because diffusion models optimize for local plausibility, not global physics. A face can look flawless while the shadow beneath it defies the sun that supposedly created it.

Alongside visual inspection, provenance tools offer something rarer: a strong positive signal. OpenAI's free Verify tool checks for two signal types, C2PA Content Credentials, an open standard that attaches provenance metadata to images, and SynthID, an invisible watermark designed to survive cropping, filters, and lossy compression. When a signal is detected, it's reliable; OpenAI states that false positives are rare.

The critical caveat is asymmetry. A detected signal tells you the image came from a participating generator. A "no signal detected" result proves nothing: metadata can be stripped, watermarks can degrade, content may predate watermarking, and Verify only detects OpenAI-generated content, not images from Midjourney, Flux, or any other model. Absence of evidence is not evidence of authenticity.

This layered approach, physics checks plus provenance plus automated detectors, reflects where detection has landed: risk assessment, not identification. The same principle applies if you're building AI workflows that process images at scale. Rather than trusting any single published detector claim, teams should verify performance in their own context. SkillzBench, AIQ Labs' free directory of agent skills, is building exactly this kind of verification, benchmarking every skill in a sandbox, with and without the skill, so you can see the measurable lift before connecting anything through its free MCP tool. The waitlist is open now.

Treat every verdict as a probability, and stack independent signals. That's the discipline that separates careful verification from confident guessing.

Your Time-Boxed Verification Workflow: 30 Seconds to Deep Dive

The fastest way to triage a suspicious image is a tiered, time-boxed workflow, the kind practitioners at the Global Investigative Journalism Network recommend for working journalists who can't spend all day on one picture. The trick is matching your effort to your stakes: most images deserve 30 seconds, a few deserve five minutes, and only the highest-stakes ones warrant a deep dive.

Tier 1: The 30-second red flag check. Scan for "too good to be true" perfection, flawless makeup on a protest leader in a crowd, perfectly styled hair on a disaster victim. Reality is rarely that polished, and AI-generated images often are.

Tier 2: The five-minute technical zoom-in. Zoom to 100% on faces and scrutinize the details AI still struggles with:

• Skin texture, look for pores, asymmetries, and natural imperfections rather than plastic smoothness
• Hair strands, real hair has flyaways and individual strands; AI often renders clumps
• Teeth, count them and check for uniformity that looks dental-catalog perfect
• Clothing physics, fabric should drape, wrinkle, and respond to the body naturally
• Jewelry and accessories, reflections and small objects frequently betray the generator

Tier 3: Deep investigation. For images that could move markets or ruin reputations, move to comparative analysis, pixel-level magnification, and forensic consultation. This is also where provenance tools earn their keep: OpenAI's free Verify tool checks for C2PA Content Credentials and SynthID watermarks, which are strong positive signals when present, though a "no signal" result proves nothing, since metadata can be stripped and the tool only detects OpenAI content.

Whatever tier you reach, resist the urge to declare a final verdict. The largest academic benchmark of detection methods, 2.6 million images across 291 generators, found the best detector achieved only 75.0% mean accuracy, and detector rankings shift dramatically between datasets. That's why the workflow's real rule is to run multiple methods and treat every verdict as a probability, never a certainty.

This probabilistic mindset applies beyond images, too. Whether you're evaluating a detection tool or an agent skill for your verification pipeline, published performance claims deserve skepticism until tested in your own context, which is exactly the philosophy behind SkillzBench, which benchmarks agent skills in a sandbox with and without the skill so you can see the actual lift before you trust one.

One warning before you start: don't lean on 2023-era tips like counting fingers. Modern models render anatomically correct hands, and a fact-checker who trusted that outdated tell declared an AI image authentic, a mistake the GIJN guide warns is more dangerous than honest uncertainty.

Automating Detection, and Why You Should Test Before You Trust

Here's a number worth pausing on: Hive reports its AI-image detector achieved 98.03% accuracy, while the largest independent benchmark to date found the best detector managed only 75.0% mean accuracy. Both figures are technically real, and the gap between them is the most important lesson in this article.

The discrepancy isn't necessarily dishonesty. Hive's figure comes from a study summarized on the company's own blog, a clear conflict of interest, and reflects its specific test conditions. The independent benchmark, run across 291 generators, 12 datasets, and 2.6 million images, found detector rankings wildly unstable across contexts, with Spearman correlations ranging from 0.01 to 0.87. A detector that tops one dataset can drop nearly 20 positions on another. The benchmark's authors put it bluntly: practitioners should select detectors based on their specific threat landscape, not published benchmark performance.

That lesson extends far beyond image detection. It applies to any AI tooling you adopt, including the agent skills that extend tools like Claude Code, Cursor, and Gemini CLI. Published research found roughly 26% of community agent skills carry at least one vulnerability, yet almost nobody tests them before deployment. This is the problem SkillzBench, AIQ Labs' free skills directory, is being built to address: every skill gets benchmarked in a sandbox with the same model, tested with and without the skill, and the resulting Lift is published in percentage points so you can see what a skill actually does in practice, not what its README promises.

The same "test before you trust" discipline applies to your verification workflow itself. Rather than running every check manually, teams can automate parts of it with agent skills:

• Running provenance checks on images as they arrive, flagging anything with C2PA metadata or SynthID watermarks for review.
• Applying the GIJN guide's tiered checks, a 30-second red-flag scan for "too perfect" content, then zoomed inspection of skin texture, shadows, and reflections.
• Routing high-stakes images through multiple detectors in parallel, since no single method is reliable.
• Logging verdicts as probability scores with confidence levels, so human reviewers see honest uncertainty instead of false certainty.

You can browse and test free agent skills for workflows like these on SkillzBench, the directory, MCP connector, benchmarks, and alerts are all free, with skill versions pinned to commit SHAs and security-scanned before they're listed.

Treat every detection verdict as a probability, not a pronouncement. The tools that survive contact with reality are the ones you've verified in your own context, and the habit of verifying before deploying is worth more than any single accuracy figure.