Can Skills Steal Your Data?

The Problem: Skills Are Someone Else's Instructions Running With Your Permissions

You installed a skill to save time, but you just handed over the keys to your house.

Agent skills are someone else's instructions running with your permissions. They inherit your shell access, file system rights, and credentials.

This isn't a hypothetical risk. It is a documented supply chain threat.

Snyk audited 3,984 skills and found 13.4% contained critical issues. Their report confirmed 76 malicious payloads designed for data theft.

A broader academic study analyzed 31,132 skills. It found that 26.1% carried at least one vulnerability.

These numbers represent a significant portion of the ecosystem. The attack surface is not just the code. It is the documentation layer itself.

Duke University researchers demonstrated "dynamic malicious skills." These embed harmful logic in natural language. The agent executes the malicious intent even if the code looks clean.

Static analysis often misses this. The model sees the instructions and follows them. It cannot distinguish between a user command and an attacker’s payload.

The industry is moving fast, but security is lagging behind. Daily skill submissions jumped from 50 to over 500 in weeks.

Publishing requires minimal effort. There is no default sandbox or code signing. The ecosystem resembles early package managers before security became standard.

You cannot rely on the model to protect you. Research shows 82.4% of frontier LLMs executed malicious commands from peer agents. They would have refused the same commands from direct user input.

Trust boundaries must be enforced by infrastructure, not by the language model.

Consider the specific risks you are accepting:

• Credential Exfiltration: Skills can read environment variables and send secrets to external servers.
• Silent Override: Name collisions allow attackers to replace benign skills with malicious versions.
• Indirect Injection: Skills fetching third-party content can execute hidden instructions from poisoned sources.

Popularity does not equal safety. Over 90% of high-popularity skills failed rigorous auditing.

You need to treat every skill as untrusted code. This means auditing before installation and pinning versions to specific commits.

Independent verification is essential. You need to know if a skill actually works and if it is safe.

SkillzBench is building a free, independent directory to address this gap. It verifies skills by pinning every version to a commit SHA. It security-scans and license-checks each package.

The platform benchmarks skills in a sandbox. It measures the actual performance lift versus running without the skill.

This ensures you know exactly what you are installing. You can toggle skills on and off per agent. Your agent only sees what you explicitly choose.

The waitlist for SkillzBench is live. Join to get access when the platform launches. You can browse and test free skills, ensuring your agent runs with proven, verified capabilities.

Why Your AI Can't Protect Itself: The Attacks That Bypass Every Defense

The uncomfortable truth about AI agent security is that the model itself — the thing you'd expect to catch a malicious instruction — is often the weakest link in the chain. Independent research teams keep arriving at the same conclusion: models process trusted instructions and attacker-controlled input identically, so they cannot enforce trust boundaries by default.

The numbers back this up. In one study of 17 frontier models, 82.4% executed malicious commands from peer agents that they would refuse if a user asked directly, according to security research from Alice. The GAP benchmark found similar divergence, with refusal rates breaking down under tool-encouraging prompts in up to 79.3% of conditional cases across six frontier models.

Datadog's testing made this painfully concrete. Their researchers watched a model explicitly state "I'm not going to execute this skill" — after the skill's exfiltration had already run. Because Claude Code's dynamic context executes shell commands before the model ever sees the skill's content, model-level defenses never get a chance to intervene. A later model version ran the same credential-stealing skill without detecting anything wrong. As Datadog's researchers put it, "No model will ever be 100% resistant to prompt injection."

Static analysis doesn't save you either. Duke University researchers demonstrated "dynamic malicious skills" that hide malicious instructions in a skill's natural-language documentation, inducing the agent to inject malicious logic into otherwise benign code at runtime — so the code on disk looks clean while the behavior isn't, per their published paper.

And the payloads themselves can be absurdly simple. Orca Security demonstrated a one-line exfiltration disguised as an update check:

• A single curl command that reads your SSH private key and sends it to an attacker's server
• Embedded in a skill that otherwise looks like routine maintenance
• Executed with whatever permissions your agent already has — shell, file system, credentials

If popularity were a proxy for safety, you could at least stick to well-known skills. Unfortunately, the opposite appears true. Researchers behind the SkillProbe study documented a "popularity-security paradox": over 90% of high-popularity skills failed rigorous auditing. High-risk skills also form a single giant connected component, meaning risk cascades systemically rather than staying isolated.

The takeaway across all these studies is consistent: security has to live in deterministic infrastructure around the model — sandboxing, least privilege, version pinning, human approval points — not in the model's judgment. That's the principle behind how SkillzBench is being built: every skill version pinned to a commit SHA, security-scanned, and benchmarked in a sandbox before it reaches your agent, with a free MCP connector that only exposes the skills you explicitly choose. If you want to evaluate what's out there while the ecosystem matures, the waitlist is open.

The Solution: Enforce Security in Infrastructure, Not in the Model

The good news buried in all this research: the defense doesn't depend on the model behaving well. It depends on the infrastructure around the model behaving deterministically. Across six independent studies, the consensus is identical — treat every skill as untrusted code, because that's exactly what it is, as Orca Security puts it.

Why infrastructure, not the model? Because the evidence shows models can't enforce trust boundaries. Datadog researchers observed a frontier model state "I'm not going to execute this skill" when it already had — the dynamic context shell commands ran before the model ever saw the skill. In separate testing, 82.4% of state-of-the-art LLMs executed malicious commands from peer agents that they would refuse if a user asked directly. "No model will ever be 100% resistant to prompt injection," the Datadog team concludes.

The strongest proof that infrastructure works comes from Duke University. Researchers built a kernel-enforced read-only mount for skill files — the operating system itself prevents any skill from modifying agent configuration or code — and the attack success rate dropped to zero while benign skills kept working normally. No clever prompting required.

The framework gaining traction formalizes this as the "lethal trifecta," attributed to Simon Willison: data exfiltration becomes catastrophic only when an agent holds three capabilities at once — access to sensitive data, exposure to untrusted content, and external communication. Meta codified it as the Agents Rule of Two: never allow more than two of the three without a human in the loop.

The practical playbook, drawn from the demonstrated attacks:

• Sandbox and restrict capabilities. Run agents in isolated environments (Docker through microVMs), apply least privilege, and where supported, disable skill shell execution entirely — Claude Code admins can set disableSkillShellExecution in managed settings.
• Pin versions to commit SHAs and audit before installing. Orca's demonstrated "Bait-and-Switch" attack relied on blind updates: a benign skill was modified after passing scans, and real users installed it while clean audit results were still displayed.
• Keep humans in the loop at approval points, and monitor installed skills — including skills that arrive silently through cloned repos and nested project directories.
• Scan for dangerous patterns: network-enabled dynamic commands, unrestricted Bash grants, credential discovery patterns like os.environ or .ssh/, and external URL fetches at runtime.

The industry is also building formal standards. OWASP launched the Agentic Skills Top 10, the first comprehensive security framework for agent skills, covering the most critical risks across all major agent platforms with evidence-based mitigations. It's emerging as the reference point for what "safe" skill adoption should look like.

This is the gap SkillzBench is being built to fill: every skill version pinned to a commit SHA, security-scanned and license-checked, benchmarked in a sandbox with published Lift scores, and a free MCP connector where your agent only sees the skills you explicitly toggle on. If you want defense-in-depth from day one, join the waitlist — skills, proven.

Your Action Plan: A Practical Checklist Before You Install Another Skill

Knowing that Snyk's audit found 76 confirmed malicious payloads in a sample of 3,984 skills, you can't afford to install another skill blind. The good news: a disciplined, mostly-automated review catches the majority of documented attacks before they ever reach your agent.

Start by auditing every skill before installation — treat every skill as untrusted code, as Orca Security puts it after demonstrating a real "bait-and-switch" attack where a benign skill was modified after passing scanning. Popularity offers no protection: academic research found over 90% of high-popularity skills failed rigorous auditing.

For technical readers, grep the skill files for the patterns Datadog identified in its analysis of the in-the-wild "Clawsights" skill:

• Network commands in dynamic context — curl, wget, nc — especially one-liners like Orca's proof-of-concept exfiltration payload disguised as an update check.
• Credential discovery patterns: os.environ, process.env, $AWS_, and .ssh/ references.
• Unrestricted tool grants such as allowed-tools:.*Bash\(\*\).
• Skills that fetch and execute remote content at runtime — benign at review, weaponizable later.

Watch for name collisions and blind updates. Orca's research showed marketplaces allow silent skill overrides on name collisions and blind bulk updates with no diffs — so pin every skill to a specific commit SHA and re-read anything that changes. Also check cloned repositories for hidden skills: Claude Code loads skills from project .claude/skills/ directories, so a cloned repo can quietly bring attacker-controlled instructions into a trusted session.

Finally, prefer vetted sources over raw marketplaces. Marketplace install counts are spoofable and scanning is non-deterministic, so look for skills that have been independently security-scanned, version-pinned to commit SHAs, and benchmarked in a sandbox. That's exactly the verification work SkillzBench is building — a free directory where every skill version is pinned to a commit SHA, scanned, and tested in a sandbox, plus a free MCP connector where your agent only sees the skills you explicitly toggle on. The waitlist is live ahead of launch, and everything — directory, scanning, benchmarks, alerts — will be free.

Skills, proven.