Question 1

How does SkillzBench keep benchmark results reproducible when skill authors push updates?

Accepted Answer

Every skill version is pinned to a commit SHA, so each benchmark run references the exact artifact tested. SkillzBench also tracks upstream changes, re-scans and re-benchmarks when skills evolve, and notifies you — so your numbers stay tied to a specific, verifiable version rather than a moving target.

Question 2

What exactly does the Lift metric measure, and how is it different from a rating?

Accepted Answer

Lift is the performance difference in percentage points when the same model runs a task in a sandbox with and without the skill. It is always shown separately from the 0-100 Trust score, which combines security, quality, license, maintenance and community signals. Separating measured performance from quality ratings keeps each number honest.

Question 3

Can I trust the security of third-party skills I pull into my research environment?

Accepted Answer

Published research found roughly of skills carry at least one vulnerability, so SkillzBench security-scans and license-checks every skill before it's verified. Script-bearing skills are launch-scoped as link-out only listings; the verified, benchmarked scope is instruction-only skills (SKILL.md plus text references).

Question 4

When can I actually start benchmarking skills — is the product live?

Accepted Answer

SkillzBench is pre-launch and actively being built; the waitlist and newsletter are live now. The five pillars — Discover, Verify, Benchmark, Equip and Stay current — are in development, and joining the waitlist gets you access as benchmarking, scanning and the MCP server come online.

Question 5

How is this different from just reading a GitHub leaderboard or community upvotes for skills?

Accepted Answer

Community signals feed the Trust score, but they're only one of five inputs alongside security, quality, license and maintenance. More importantly, SkillzBench measures Lift empirically in a sandbox with the same model, with and without the skill — and money never touches scores, so there's no pay-to-rank dynamic like typical marketplaces or static leaderboards.

Agent Skill Benchmarking for Researchers