How to Detect a Malicious Skill: Red Flags Checklist

The Skill You Just Installed Might Be the Problem

If you've installed an agent skill in the past month, there's a reasonable chance it's carrying a security flaw — and a smaller but very real chance it's actively working against you. That's not speculation. Three independent audits of the skills ecosystem have converged on the same uncomfortable picture.

The numbers are stark. A large-scale analysis of 31,132 unique skills found that 26.1% contain at least one vulnerability, with 13.3% exhibiting data-exfiltration patterns and 5.2% showing high-severity behavior that strongly suggests malicious intent, according to academic research. A separate Snyk audit of nearly 4,000 skills found 13.4% carried a critical issue. And behavioral verification of 98,380 skills confirmed 157 as actively malicious — not merely risky, but deliberately weaponized.

What makes skills uniquely dangerous is what they inherit. Once you approve a skill, it gains your agent's full privileges — shell access, file system read/write, credentials sitting in environment variables, network connections, even persistent memory across sessions — with no further prompts. Researchers call this the consent gap, and OWASP classifies it as "Identity and Privilege Abuse."

Then there's the detection problem. The most dangerous payloads aren't obfuscated code at all — they're plain-English prose instructions like "don't tell the user" or "no need to flag this to the operator." Your code scanner never reads those lines, because they aren't code. As one practitioner audit put it, the agent doesn't draw a clean line between data and instruction, which makes every sentence in a skill file a potential attack vector.

A few patterns worth internalizing immediately:

• A skill that both reads credentials and makes a network call in the same file — the single most durable red flag researchers have identified
• Description-vs-behavior mismatch: the description never mentions internet access, but the bundled script phones home
• Permission-function mismatch: a writing assistant requesting shell execution, or a calendar tool poking around ~/.ssh/
• Instructions to hide actions from you or disable safety checks

This is exactly the gap SkillzBench is being built to close — pinning every skill to a specific version, running security scans and behavioral verification in a sandbox, and publishing a trust score that money can't influence. Until independent verification becomes standard, the burden of inspection falls on you. And one final caveat before you feel reassured by a clean SKILL.md: sophisticated attacks deliberately avoid explicit malicious instructions and hide payloads in auxiliary scripts. The absence of red flags in the file you read is not evidence of safety.

Why Your Gut Check and the Star Count Will Fail You

If your only defense is a quick skim and a glance at the star count, attackers have already won. The research consistently shows that the two most instinctive checks — "does this look malicious?" and "is it popular?" — are exactly the ones sophisticated skills are designed to defeat.

Start with the gut check. Static scanning of skill content sounds like a solid filter, but when one practitioner audited a legitimate skill, 16 of 20 scanner findings turned out to be the skill simply doing its advertised job — an analysis of static tooling put the false-positive rate at roughly 80%. A wrong "dangerous" flag trains your team to ignore warnings; a wrong "safe" label can ship a credential stealer. Static analysis alone simply cannot distinguish malice from developer mistakes, which is why behavioral verification achieved 99.6% precision — a 150× improvement over static-only approaches, according to large-scale behavioral research.

Star counts fail for a different reason: they can be manufactured. A single actor accounted for 54.1% of confirmed malicious skills by templating brand-impersonation clones, and one account shipped more than 40 near-identical malicious skills. Popularity measures velocity, not safety — and in an ecosystem where submissions grew 10× in under a month, velocity is the threat.

The deeper trap is that the most dangerous skills keep SKILL.md pristine. SkillJect-style attacks inject only a short "inducement prompt" into the instruction file while hiding the actual payload in bundled .py/.sh scripts that look benign in the repository. That matters because skills with bundled executable scripts are 2.12× more likely to contain vulnerabilities than instruction-only skills, per an audit of over 31,000 skills. A clean-looking SKILL.md is not evidence of safety — it can be the decoy.

Then there are the traps that bypass your inspection entirely:

• Dynamic context commands (Claude Code's `!` syntax) run shell commands before the model ever sees the skill, so model-level injection defenses never get a chance to intervene.
• Skills auto-load from nested `.claude/skills/` folders in cloned repos — you don't need to install anything malicious for it to enter a trusted session.
• A previously benign skill can push a malicious update after publication, which is why version history review is essential.
• Model defenses are inconsistent even within the same family: in Datadog's testing, one model version refused a malicious skill while the next ran it without detecting credential theft.

This is why SkillzBench is building verification around version pinning to commit SHAs, security scanning, and sandboxed behavioral testing rather than a single glance at the file. If you've installed a skill in the past month, there's roughly a 13% chance it contains a critical security flaw — a number no star count or gut feeling will protect you from.

The Red Flags Checklist: 20 Signals That Actually Predict Malice

Every malicious skill that researchers have caught shares the same uncomfortable truth: the most dangerous payloads are plain-English prose instructions that no code scanner ever reads. One auditor put the detection philosophy bluntly: "look for these words" ages badly, while "look at the actual behavior, and check it against what was promised" doesn't. That's why this checklist leads with phrasing-independent signals — behaviors that survive rewording, translation, and the next generation of obfuscation.

Tier 1: Instruction-level red flags. Watch for directives that tell the agent to hide actions from the user ("don't tell the user," "no need to flag this") or to suppress safety warnings, a pattern Snyk's ToxicSkills audit flagged across its 3,984-skill review. Also inspect metadata for invisible characters — zero-width spaces and RTL overrides — and for semantic drift where SKILL.md content wanders from the skill's stated purpose. Remember the SkillJect finding: sophisticated attacks deliberately keep explicit instructions clean, so the absence of malicious-looking text is not evidence of safety.

Tier 2: Code-level red flags. The single strongest signal is a script that reads credentials and makes a network call in the same file — a combination behavioral researchers identify as the most durable indicator of malice. Add to that obfuscation (base64, eval $(echo ... | base64 -d), minified code without source maps) and unverifiable curl | bash dependencies that let attackers change behavior at runtime.

Tier 3: Structural red flags. Description-vs-behavior mismatch — no mention of internet access, but the script phones home — plus permission-function mismatch and shadow features are the structural tells. Shadow features are especially telling: they appear in 0% of basic attacks but 100% of advanced ones, according to behavioral verification research covering 98,380 skills.

Tier 4: Author and supply chain red flags. The final tier covers who published it and what changed:

• Typosquatting — skill names closely mimicking popular legitimate ones
• Fresh accounts with no other projects; one actor shipped 40+ near-identical malicious skills and accounted for 54.1% of confirmed cases via templated brand impersonation
• Post-publication changes — a previously benign skill pushing a malicious update, which is why pinning to a commit SHA matters

These four tiers are exactly what SkillzBench is building its Verify and Stay-current pillars around: version-pinned scanning that re-checks skills when upstream code changes. No single flag proves malice — static analysis alone produces roughly 80% false positives on legitimate skills — but stacked signals, checked against actual behavior, are what actually predict it.

From Manual Audit to Systematic Defense

Running the red-flags checklist by hand works — until you count the cost. Practitioner estimates put a careful manual audit at 15–20 minutes for simple skills and 1–2 hours for complex multi-file packages with network connections. Multiply that across a team installing dozens of skills, and manual review collapses under its own weight.

The answer is layered defense, not a single gate. Version-pinning every skill to a specific commit SHA matters because supply-chain compromise happens after publication — a previously benign skill can push a malicious update, and curl | bash dependencies let attackers change behavior at runtime. What you audited yesterday is not what runs tomorrow unless you pin it.

Static scanning alone doesn't work either. On a legitimate skill, static analysis produced roughly 80% false positives — 16 of 20 findings were simply the skill doing its advertised job, as one hands-on audit found. Behavioral verification in a sandbox, by contrast, achieved 99.6% precision — a 150× improvement over static-only approaches, because watching what a skill actually does distinguishes malice from developer mistakes.

A workable layered defense looks like this:

• Pin to commit SHAs so a skill can't change under you after publication
• Combine static scanning with sandboxed behavioral verification rather than trusting either alone
• Monitor upstream continuously — re-scan and re-verify when the source repo changes
• Treat script-bearing skills as elevated risk, since bundled executables are 2.12× more likely to contain vulnerabilities

Here is the uncomfortable truth that ties it together: the absence of red flags is not evidence of safety. SkillJect-style attacks deliberately avoid explicit malicious instructions and hide payloads in auxiliary scripts precisely because naive injections fail when "the malicious intent is too explicit." A clean SKILL.md proves nothing about the files the agent eventually reads.

This is the gap independent verification exists to fill. SkillzBench is building exactly this pipeline — SHA-pinned validation, security scanning, sandboxed benchmarking, and continuous upstream tracking with re-scans on change — because no one else in the skills ecosystem provides it. As Snyk's audit concluded, the ecosystem today resembles early npm and PyPI before security became a first-class concern — except skills inherit the agent's full privileges, making the blast radius larger. The package managers eventually built verification into the platform. Skills need the same, and the research suggests they need it now.

What SkillzBench Is Building to Fix This

Reading a checklist is one thing; running it against 31,000+ skills is another. The audits behind this post converge on the same conclusion: manual vetting doesn't scale, static scanning alone produces roughly 80% false positives on legitimate skills, and the ecosystem needs layered, independent verification. That's the gap SkillzBench is building to close.

To be transparent up front: SkillzBench is pre-launch and actively being built. Nothing below describes shipped results — it describes the pillars under construction, mapped directly to the red flags in this checklist.

• Verify. Every skill version gets pinned to a commit SHA so a silent upstream update can't swap a benign skill for a malicious one — the post-publication compromise pattern researchers flagged. Each pinned version is validated, license-checked, and security-scanned, feeding a trust score from 0–100 built on security, quality, license, maintenance, and community signals.
• Benchmark. Static analysis can't reliably distinguish malice from developer mistakes — behavioral verification hit 99.6% precision, a 150× improvement over static-only approaches. So SkillzBench will test every skill in a sandbox, with the same model, with and without the skill, and publish the Lift in percentage points. You'll see whether a skill actually helps, not just whether it scans clean.
• Equip. Skills auto-load from nested folders and cloned repos, meaning a skill you never chose can enter a trusted session. SkillzBench's per-agent loadouts flip that default: you toggle skills and tools per agent, and the agent only sees what you explicitly enabled.
• Stay current. Upstream changes trigger re-scans and re-benchmarks, with notifications when something shifts — because the skill you vetted last month isn't necessarily the skill running today.

Two framing commitments matter as much as the pillars. First, Lift is always shown separately from ratings, and money never touches trust scores — because a single-integer scanner verdict is, as one audit put it, the least trustworthy thing on the screen. Second, launch scope is instruction-only skills; script-bearing skills carry a quantified 2.12× vulnerability multiplier (OR=2.12, p<0.001), so they're listed link-out only rather than benchmarked alongside.

If that's the control panel you've been missing, join the waitlist at skillzbench.com for early access. Skills, proven.