Can AI Agents Actually Handle PDF? What Skills Can and Cannot Do in 2026

The PDF Gap: Why Your AI Agent Fumbles Documents

Your AI agent says it read the 200-page contract. It confidently summarizes clause 14, quotes the termination section, and then you discover it missed the indemnity language entirely — a failure it never mentioned. This is the PDF gap: agents don't fail loudly, they fail silently, and the document is exactly where it hurts most.

The first documented failure mode is context overflow. Practitioner research on PDF-reading agents found a practical ceiling of roughly 373 pages on dense documents — and the real limit arrives sooner than the published page cap suggests, because agents default to over-reading instead of inspecting first. When a 500-page filing gets loaded wholesale, content simply falls off the end of context, with no error message to warn you.

The second is scrambled reading order. A two-column financial report, extracted with naive positional sorting, interleaves the left and right columns into gibberish. The same research found that adding a column-detection gate lifted two-column fidelity from 0.564 to 0.816 — meaning nearly half the extracted text was corrupted before the fix. As the pdf-mcp maintainer puts it, "the text your model reads is not the text you see, until you prove otherwise," and averaged quality scores hide localized corruption.

The third is search-mode mismatch. Which retrieval strategy works depends on granularity:

• At section level, plain BM25 keyword search beat hybrid RRF fusion
• At page level, hybrid search was preferred
• Agents gravitate to the cheapest available action, so the tool layer's defaults decide the strategy — not your prompt

That last point is the crux: the model isn't the bottleneck — the tool layer is. As Anthropic's own engineering team acknowledges, Claude "already knows a lot about understanding PDFs, but is limited in its ability to manipulate them directly," which is exactly why agent skills exist: they're instruction packages that orchestrate external tools like pypdf, pdfplumber, and pytesseract. Prompt engineering cannot rescue a tool layer that forces agents into bad decisions.

This is also why bolting on a random PDF skill isn't automatically the fix. Skills extend capability, but they must be chosen and verified deliberately — the approach SkillzBench is building around, with every skill version-pinned and benchmarked with and without the skill before it earns a trust score. The gap between "agent claims it read the PDF" and "agent actually read the PDF" is closed by tooling you can verify, not vibes.

How Skills Actually Work: Instructions That Orchestrate Tools

The word "skill" sounds like it should mean a new ability inside the model. It doesn't. An agent skill is just an organized folder of instructions, scripts, and resources that an agent discovers and loads at runtime — prompt-based guidance that tells the model how to orchestrate the tools it already has.

Anthropic's own PDF skill is the clearest proof. Claude already understands PDFs but cannot manipulate them directly — it can't merge two files or fill a form field. The skill closes that gap by mapping each task to a specific library rather than teaching the model anything new.

• Merging and splitting route to pypdf; text and table extraction route to pdfplumber
• PDF creation calls reportlab; OCR converts pages to images with pdf2image, then runs pytesseract
• Form filling is complex enough to need its own bundled instruction file (FORMS.md) inside the skill

This matters because many document operations require the deterministic reliability that only code can provide. Asking a model to hallucinate a merge result invites corruption; handing it a script that performs the merge produces the same output every time.

Skills also follow a design principle called progressive disclosure: the agent reads only the top-level description first, then loads deeper instruction files only when a task demands them. That keeps context lean on a 373-page document where a naive read-through already risks overflow (pdf-mcp engineering notes) — a practical ceiling practitioners hit well before published page caps.

The trade-off is that executable code inside a skill is executable code, full stop. Security researchers have demonstrated skills exfiltrating SSH keys and achieving remote code execution through marketplace distribution (Orca Security research), which is why auditing and pinning skills to specific commits matters as much as picking the right tool mapping.

Since no independent with-versus-without benchmarks of skill performance exist publicly yet, SkillzBench is building exactly that: version-pinning to commit SHAs, security scanning, and sandbox Lift testing for every skill in its free directory. Skills, proven. Join the waitlist to browse, verify, and benchmark agent skills when we launch.

The Catch: Every Skill Is Untrusted Code

Here is the security section for your article.

The same mechanism that makes agent skills useful is what makes them dangerous: every skill is executable code running on your machine. Anthropic itself warns that malicious skills may introduce vulnerabilities or direct agents to exfiltrate data and take unintended actions. That is not a theoretical risk.

Orca Security built working proof-of-concept skills that ran arbitrary shell commands, including exfiltrating a user's SSH private key via curl, and demonstrated three end-to-end attack flows against a prominent skills marketplace. Each achieved confirmed code execution on end-user machines.

The distribution layer is just as exposed. According to Orca's marketplace research, skills are scanned only at creation and again at a popularity threshold, leaving an exploitation window of days or weeks. Worse, install counts were inflated to several thousand through a single scripted HTTP request — so the social proof you see can be manufactured.

OWASP has responded with the Agentic Skills Top 10, the first comprehensive security framework for AI agent skills, covering every major agent platform. Until that guidance matures into enforcement, the practical rule is simple:

• Treat every skill as untrusted code, because that is exactly what it is.
• Pin skills to a specific commit rather than tracking a moving branch.
• Audit the instructions and bundled scripts before installing — not after.
• Avoid blind bulk updates that silently pull in new code.
• Never trust platform badges or install counts as a security signal.

This is precisely the gap SkillzBench is being built to close. Every skill in its directory will be pinned to a commit SHA, security-scanned, license-checked, and quality-judged before it earns a trust score — and independent Lift benchmarks, measured with and without each skill, will always sit separately from ratings. Money never touches a score.

Skills, proven. Join the SkillzBench waitlist to browse, scan, and benchmark agent skills for free when we launch.

Your PDF Agent Playbook: Inspect First, Read Last

Most PDF failures in AI agents are silent: the agent reads the file, produces confident output, and nobody notices the context window quietly buckled or the two-column text arrived scrambled. The fix isn't a smarter prompt — it's a better playbook.

Inspect first, read last. Practitioner benchmarks show a dense document hits the practical context ceiling at roughly 373 pages, well before the published page cap suggests, and agents default to over-reading unless the tool layer makes inspection cheaper than extraction. Scout page count, metadata, and table of contents before pulling a single line of text. As the pdf-mcp maintainer puts it, "prompt engineering cannot rescue a tool layer that forces agents into bad decisions."

Gate multi-column extraction. Naive positional sorting silently corrupts reading order in two-column layouts, and corrupted input poisons everything downstream. The numbers are stark: adding a column-detection gate lifted two-column fidelity from 0.564 to 0.816. Run detection before extraction, every time, on anything that might be a report, paper, or form.

Match strategy to task economics. Cost decides which workflows actually get automated. LandingAI reports parsing below one cent per page in some tiers, with 25–80% cost reductions on mixed workloads — meaning simple pages shouldn't ride premium extraction models. Route cheaply where you can.

Demand source-verifiable answers in regulated workflows. A system can read a million pages, but if you can't check any single answer against its source, you can't deploy it where audits happen.

Your deployment checklist:

• Inspect page count, metadata, and TOC before extracting content
• Run column detection before reading multi-column documents
• Route extraction by document complexity, not one-size-fits-all
• Require every answer to cite its source page

The same discipline applies to the skills themselves. Orca Security demonstrated skills executing arbitrary shell commands, and advises treating every skill as untrusted code — pin versions, audit before installing, skip blind bulk updates. SkillzBench is building exactly this guardrail: pinning every skill to a commit SHA, security-scanning it, and benchmarking with-and-without-skill Lift in a sandbox, all free at launch. If you're assembling a PDF agent, that's the layer where "skills, proven" stops being a slogan and becomes a deployment requirement.

Choosing PDF Skills Without Getting Burned

The uncomfortable truth about PDF skills in 2026 is that installing one is a security decision as much as a productivity one. Anthropic itself warns that malicious skills may introduce vulnerabilities or direct an agent to exfiltrate data, and Orca Security demonstrated skills executing arbitrary shell commands — including exfiltrating ~/.ssh/id_rsa via curl — with confirmed code execution on end-user machines. Treat every skill as untrusted code, because that is exactly what it is.

Before you install anything, run through a short pre-flight checklist:

• Security scan — read the bundled scripts, not just the SKILL.md summary. Marketplace scanning runs only at creation and at a popularity threshold, leaving exploitation windows of days or weeks.
• Version pinning — pin to a specific commit SHA and avoid blind bulk updates, since a skill's behavior can change between releases.
• License check — confirm the terms permit your intended commercial use.
• Maintenance signals — release cadence, issue responsiveness, and whether the author tracks upstream tool changes.
• Provenance — install only from trusted sources, as Anthropic's own guidance advises.

Here is the gap that makes all of this harder than it should be: no independent, with-versus-without benchmarks of PDF-skill performance exist in any published source. Vendor claims are self-reported, and practitioner evidence shows silent failure modes — reading-order corruption in multi-column layouts, a practical context ceiling around 373 pages on dense documents — that averaged quality scores tend to hide.

SkillzBench is being built to close that gap. The platform pins every skill version to a commit SHA, validates it, license-checks it, security-scans it, and quality-judges it, then tests each skill in a sandbox with the same model, with and without the skill, publishing the resulting Lift in percentage points separately from ratings. Scanning, benchmarks, and a free MCP connector that lets you toggle exactly which skills each agent sees will all be free when the directory launches.

Skills, proven. Join the SkillzBench waitlist to browse, scan, and benchmark agent skills for free when we launch.