Question 1

How does SkillzBench measure whether an Agent Skill actually helps?

Accepted Answer

Every skill is tested in a sandbox with the same model, with and without the skill — a paired evaluation. The result is published as Lift in percentage points, always shown separately from ratings so you can see measured performance gain independent of opinion. This matters because seed noise can swamp real capability signal in single-run benchmarks.

Question 2

Are script-bearing skills supported?

Accepted Answer

At launch, SkillzBench covers instruction-only skills — SKILL.md plus text references. Script-bearing skills are listed link-out only, so you can discover them but they aren't fully verified or benchmarked in the sandbox yet.

Question 3

How does SkillzBench handle the security risk in community skills?

Accepted Answer

Published research found ~of skills carry at least one vulnerability, which is why Verify is a core pillar: every version is pinned to a commit SHA, validated, license-checked, security-scanned and quality-judged. The results feed a trust score from 0-100 built on security, quality, license, maintenance and community signals.

Question 4

When can I start using SkillzBench?

Accepted Answer

SkillzBench is pre-launch and actively being built — the waitlist and newsletter are live now, but the product itself is not yet public. Join the waitlist to get notified as benchmarking, scanning, and the MCP server come online, and to help shape launch priorities.

Question 5

How is SkillzBench different from just reading GitHub stars and READMEs?

Accepted Answer

Stars and READMEs tell you popularity, not performance or safety. SkillzBench gives you measured Lift from paired sandbox evaluation, security scans, license checks, and a 0-100 trust score — and money never touches scores, so ratings stay independent. We also track upstream changes and re-benchmark, so your information stays current.

Agent Skill Benchmarking for Freelancers