Red Teaming Your Agent Loadout

Your Agent Loadout Is Already Under Attack

The skills your agent runs right now were probably installed with less scrutiny than a browser extension — and the numbers back that up. A large-scale academic analysis of 31,132 unique skills found that 26.1% contain at least one vulnerability, spanning prompt injection, data exfiltration, privilege escalation, and supply chain risks. A separate Snyk audit confirmed 76 malicious payloads actively circulating in the wild.

This is not theoretical. In December 2025, a benign-looking "GIF Creator" skill silently downloaded and executed MedusaLocker ransomware. In February 2026, researchers documented the first coordinated malware campaign targeting Claude Code and OpenClaw users — 30+ malicious skills on ClawHub — and 8 confirmed malicious skills were still publicly available when Snyk published its findings. Meanwhile, the state-sponsored GTG-1002 campaign weaponized coding agents against 30+ organizations, with 80%+ of operations running autonomously.

The structural problem underneath all of it is what researchers call the consent gap. Once you approve a skill, it inherits your agent's full permission set — shell access, filesystem read/write, credentials in environment variables, network connections — with no further prompts and no mandatory vetting. Anthropic's own documentation warns that skills "can introduce vulnerabilities" and enable agents "to exfiltrate data and take unintended actions."

What that inherited access actually looks like in practice:

• Shell and command execution, persisting across sessions without re-approval
• Filesystem read/write, including config files that hold credentials
• Network access for data exfiltration and runtime-fetched instructions
• Messaging channels like email, Slack, and WhatsApp

The risk is also unevenly distributed. Skills that bundle executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills — a distinction that shaped how SkillzBench is scoping its launch, focusing on instruction-only SKILL.md packages while listing script-bearing skills as link-outs only. And a skill that's benign at install can turn malicious later: some skills fetch remote instructions at runtime, meaning behavior can change after publication.

Snyk's researchers put it bluntly: the skills ecosystem "has a supply chain security problem that mirrors the early days of npm and PyPI — except with unprecedented access to credentials, file systems, and APIs." The difference is that npm packages never had your shell, your secrets, and your Slack account by default. Your loadout is the new attack surface — and nobody is vetting it for you.

Why Scanning for Malware Isn't Enough: The Full Attack Surface

Your agent passed a malware scan. That proves almost nothing. The most dangerous attacks on coding agents never touch a binary at all — they live in the configuration you approved once and forgot.

Consider what researchers keep finding in "clean" setups. Pillar Security's Rules File Backdoor research showed invisible Unicode characters embedded in Copilot and Cursor rules files steering agents to write backdoored code — no malware on disk, just text the agent reads as instructions. The same principle extends to MCP server configurations and cloned repositories, which attackers routinely poison as import targets. In other words, the same Claude Code binary can be safe or dangerous depending entirely on its settings file.

The empirical numbers back this up. Red-teaming research on tool-invocation hijacking achieved remote code execution on every tested agent-LLM pair across six agents including Cursor, Claude Code, and Copilot. Meanwhile, prompt injection attacks on coding editors succeeded 41–84% of the time across 314 unique payloads. And built-in guardrails? Adversa AI's GuardFall research bypassed command-approval guards in 10 of 11 popular open-source coding agents using decades-old shell tricks — and auto-approve flags reopened the hole in the one agent that initially held.

A malware-only mindset misses the whole loadout. Effective red teaming must cover:

• Skills — including instruction files that fetch remote content at runtime, allowing post-publication behavior modification
• MCP servers — tool descriptions are a documented injection vector, and 5.5% of tested MCP servers showed tool poisoning
• Rules and config files — where invisible Unicode and prompt text can change agent behavior without any executable
• Agent settings — auto-approve flags, pre-approved commands, and hooks that fail open

This is the "consent gap" researchers identify as the core enabler: once approved, loadout components inherit full agent permissions — shell, filesystem, credentials, network — without further prompts. It's why a large academic study of 31,132 skills found 26.1% carry at least one vulnerability, and why Snyk's independent audit confirmed 76 malicious payloads in the wild.

This is precisely the problem SkillzBench is being built to address. Rather than treating skills as install-and-forget, SkillzBench is designing per-agent loadouts where the agent only sees what was explicitly chosen — skills, tools, and settings — with every version pinned, security-scanned, and re-benchmarked as upstream changes land. Loadout minimization isn't paranoia; it's the practical answer to a quantified attack surface.

If you're red-teaming an agent today, scan for malware by all means. But assume the real backdoor is sitting in a rules file, an MCP config, or an auto-approve flag you enabled months ago. The configuration is the attack surface.

The Red-Team Checklist: Five Layers to Audit

Most teams audit their agent skills once, at install, and never look again. The research says that's exactly the wrong posture: a skill that's benign at install can turn malicious later, because some skills fetch instructions at runtime — curl | source patterns allow post-publication behavior modification, meaning yesterday's clean scan tells you nothing about today's code.

Here's a five-layer audit checklist that no single source assembles completely, but that the evidence supports layer by layer.

1. Pin and re-verify skill versions. The GIF Creator ransomware skill looked harmless until it silently downloaded MedusaLocker, and the ClawHub malware campaign left confirmed malicious skills publicly available even after disclosure. Pin every skill to a specific commit SHA and re-scan on upstream changes — one-time vetting is a snapshot, not a control. This is the core principle behind SkillzBench's Verify and Stay-current pillars, which pin each version and re-scan when upstream code moves.

2. Prefer instruction-only skills. Skills bundling executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only ones (p<0.001). If a skill ships scripts, treat it as untrusted code, not configuration.

3. Inventory MCP servers and rules files. Injection text doesn't only live in SKILL.md files. Research on coding editors shows attackers poison rules files (.cursor/rules), READMEs, and MCP tool descriptions — and GTG-1002 weaponized agents specifically through malicious MCP servers. Read every rules file and tool description as if it were an untrusted prompt, because that's what your agent is doing.

4. Strip auto-approve and YOLO flags. Adversa AI's GuardFall research bypassed command-approval guards in 10 of 11 popular coding agents using decades-old shell tricks — and the one agent that held failed once its auto-approve flag was enabled. Risk lives in configuration: the same binary is safe or dangerous depending on what's in settings.json.

5. Minimize permissions per agent. The research consistently identifies the "consent gap" as the core enabler: once approved, skills inherit shell, filesystem, credential, and network access without further prompts. The fix is scope, not vigilance.

• Give each agent only the skills and tools its task requires — nothing more.
• Keep credentials out of env vars and config files agents can read.
• Segment agents so a compromised coding agent can't reach production data.
• Review loadouts quarterly; remove anything unused.

Per-agent loadout control is the practical answer to a documented problem — when the agent only sees what was explicitly chosen, a poisoned skill has far less to work with. That's the design SkillzBench is building toward, and it's the design the attack data demands.

From One-Time Audit to Continuous Verification

A one-time audit is a snapshot of an ecosystem that refuses to stand still. According to Snyk's audit of the skills ecosystem, daily skill submissions jumped from under 50 in mid-January 2026 to over 500 by early February — a 10x increase in a matter of weeks.

Decay doesn't only come from volume. Snyk's same research documents runtime-fetched remote instructions (curl | source patterns) that allow post-publication behavior modification — a skill that was benign at install time can change what it does after you've already trusted it. The GIF Creator ransomware incident made this concrete: a harmless-looking skill silently downloaded and executed MedusaLocker, per the academic study of 31,132 skills.

That's why the red-team checklist above needs to become a loop, not a line item. SkillzBench is being built around exactly this shift, with each building block mapping to a step in the checklist:

• Per-version security scanning — every skill version is pinned to a commit SHA and scanned, so a "safe" verdict attaches to the exact code you reviewed, not to whatever the repo contains next week.
• Sandboxed benchmarking, with and without each skill — the same model runs the same tasks both ways, so you measure what a skill actually changes, not what its README claims.
• Per-agent loadout control — you toggle skills and tools per agent, and the agent only sees what was chosen, closing the consent gap where skills inherit shell, filesystem, and credential access by default.
• Upstream change tracking — when a skill's source changes, it's re-scanned, re-benchmarked, and you're notified, converting your audit from an event into a standing control.

The research makes clear this is the only posture that matches the threat. Unit 42's telemetry shows indirect prompt injection is being actively weaponized in the wild, with 22 distinct attacker techniques observed — not a lab curiosity. And with 26.1% of analyzed skills carrying at least one vulnerability, the odds that your loadout stays clean by luck alone are not good.

SkillzBench is pre-launch and actively under construction. If continuous verification of your agent loadout sounds better than a decaying one-time audit, join the waitlist at skillzbench.com for early access as the pillars come online.