Smithery vs SkillzBench: Which Agent Skills Platform Should You Trust?
Smithery is a live, free-to-start marketplace that connects AI agents to 100K+ MCP tools and skills, with managed OAuth connections and a Hobby (free) plan capped at 3 namespaces — but it's now part of Arcade.dev and focuses on tool distribution rather than proving which skills actually help. SkillzBench, currently in pre-launch, is building a free directory that security-scans, version-pins, and benchmarks agent skills to show which ones measurably improve agent output.
TL;DR
- Smithery is live today and free to start, with a Hobby (free) plan limited to 3 namespaces and paid Pay as you Go (100 namespaces) and Custom (100+) tiers.
- Smithery positions itself as 'the marketplace for AI agents,' connecting agents to 100K+ tools and skills with auth, credentials, and sessions handled for you.
- Arcade.dev acquired Smithery in August 2026, citing the need to shift the MCP ecosystem 'from building fast to building tools you can trust.'
- SkillzBench is pre-launch and building a free, independent directory that pins every skill version to a commit SHA, security-scans, license-checks, and quality-judges each skill.
- SkillzBench's benchmarking tests every skill in a sandbox with the same model, with and without the skill, and publishes the Lift in percentage points.
- Published research found roughly 26% of agent skills carry at least one vulnerability — the core problem SkillzBench is being built to address.
- SkillzBench is entirely free (directory, MCP connector, benchmarks, alerts), funded by blog advertising that never influences scores.
Head to head
| Dimension | Smithery | SkillzBench |
|---|---|---|
| Pricing | Free Hobby plan (3 namespaces); Pay as you Go (100) and Custom (100+) tiers | Everything free — directory, MCP connector, benchmarks, alerts |
| Availability | Live today — sign up free at smithery.ai 🏆 | Pre-launch; waitlist/newsletter live, product not yet public |
| Catalog scope | 100K+ tools and skills; MCP servers, connections, and skills 🏆 | Agent skills (SKILL.md packages) continuously crawled and deduped from GitHub |
| Security vetting | Listing quality scores (e.g., 84/100); no documented per-skill security scanning | Every skill version pinned to a commit SHA, security-scanned, license-checked, quality-judged 🏆 |
| Benchmarking | Not offered at the skill level | Sandbox tests with the same model, with and without each skill; Lift published in percentage points 🏆 |
| Agent connection | Managed MCP connections with OAuth, credential storage, and session handling; Uplink for local servers | One free MCP server plus per-agent loadouts — the agent only sees chosen skills and tools |
| Independence | Part of Arcade.dev since August 2026 | Independent directory from AIQ Labs; ad-funded, money never touches scores 🏆 |
| Setup | npm install -g @smithery/cli, browser auth, then search and connect 🏆 | Free MCP tool connects any agent to chosen skills; per-agent toggles |
The contenders
Smithery is a live public registry and hosting platform for MCP (Model Context Protocol) servers, connections, and skills, self-described as 'the marketplace for AI agents' that connects agents to 100K+ tools and skills instantly. It handles auth, credentials, and sessions for managed MCP connections, including OAuth flows, credential storage, token refresh, and session lifecycle. Developers can publish servers via URL or local bundles, get dedicated server pages with analytics on tool calls and usage, and use a CLI for searching, connecting, and calling tools. Its Uplink feature exposes a local MCP server as a regular Smithery connection via a secure tunnel. In August 2026, Arcade.dev acquired Smithery, with plans to pair Smithery's developer experience with Arcade's enterprise security and governance focus. Popular listings include Brave Search (106.86k uses), Gmail (59.79k uses), and its own CLI skill (1,652 installs).
pricing 0
- + Live and free to start — sign up at smithery.ai and run an MCP server in moments
- + Very large catalog: 100K+ tools and skills, with popular servers showing tens of thousands of uses
- + Handles auth, credentials, and sessions automatically, including OAuth flows and token refresh
- + Strong developer experience: CLI, TypeScript SDK, cURL/API access, and JSONL piped output
- + Uplink lets you test local MCP servers against real agents before publishing
- + Free Hobby plan available for getting started
- − Hobby (free) plan is capped at 3 namespaces; scaling requires Pay as you Go or Custom plans
- − Acquired by Arcade.dev in August 2026, which may shift the roadmap toward enterprise needs
- − Arcade's own ToolBench benchmark found MCP ecosystem growth 'has outpaced quality' — Smithery listings show quality scores (e.g., 84/100) but no documented per-skill security scanning or with/without benchmarking
- − Uplink is a development primitive, not a hosting solution; availability follows the host machine and calls fail fast when the CLI is offline
- − Third-party listings note setup complexity and a learning curve as common reasons users explore alternatives
SkillzBench is AIQ Labs' free, independent directory of AI agent skills plus a free MCP tool that connects any agent to the skills you choose — currently in pre-launch and actively being built. It targets a real gap: agent skills (SKILL.md instruction packages for Claude Code, Codex, Cursor, Gemini CLI and chat-app agents) are everywhere, but published research found roughly 26% carry at least one vulnerability. SkillzBench is building five pillars: Discover (continuously crawl and dedupe skills from GitHub), Verify (pin every version to a commit SHA, validate, license-check, security-scan, quality-judge), Benchmark (test every skill in a sandbox with the same model, with and without the skill, publishing the Lift in percentage points), Equip (one free MCP server plus per-agent loadouts where the agent only sees what was chosen), and Stay current (track upstream changes, re-scan, re-benchmark, notify). A 0-100 Trust score combines security, quality, license, maintenance and community signals; Lift is always shown separately from ratings and money never touches scores. Everything — directory, MCP connector, benchmarks, alerts — is free, funded by advertising in and around blog articles.
pricing 0
- + Completely free — directory, MCP connector, benchmarks, and alerts, with no user charges
- + Purpose-built for trust: commit-SHA version pinning, security scanning, license checks, and quality judging on every skill
- + Benchmarks show measurable Lift (with vs. without the skill) rather than just ratings, and Lift is always shown separately from scores
- + Per-agent loadouts mean an agent only sees the skills and tools explicitly chosen for it
- + Independent and ad-funded — advertising money never influences scores or listings
- + Addresses a documented problem: ~26% of skills carry at least one vulnerability per published research
- − Pre-launch — the waitlist/newsletter is live but the product itself is not yet public
- − No live user counts, skill counts, or published benchmark results yet
- − Focused on skills (SKILL.md packages) rather than hosting or running arbitrary MCP servers
Verdict
Winner: none
Smithery is the mature, available-today option for discovering and running MCP tools; SkillzBench is building a complementary verification and benchmarking layer rather than a direct replacement. Small businesses needing tools now should use Smithery today and watch SkillzBench for trust and Lift data.
| Option | Overall | Breakdown |
|---|---|---|
| Smithery | 7.5 | : 8: 9: 8: 5: 6 |
| SkillzBench | 7 | : 10: 3: 8: 9: 9 |
What each platform actually does
Trust and security compared
Pricing compared
Which should you pick?
- :
- :
- :
- :
- :
FAQ
Is Smithery free to use?
Yes, Smithery offers a Hobby (free) plan that includes up to 3 namespaces, and you can sign up free at smithery.ai. Paid tiers exist — Pay as you Go (100 namespaces) and Custom (100+) — for teams that need more capacity. Arcade.dev, which acquired Smithery in August 2026, still directs new users to sign up for free and find an MCP server.
Is SkillzBench available yet?
SkillzBench is currently pre-launch and actively being built — the waitlist/newsletter is live, but the product itself is not yet public. Its planned offering includes a free skills directory, a free MCP connector, sandboxed Lift benchmarks, and security scanning for agent skills. Join the waitlist at skillzbench.com to be notified at launch.
Can I use both Smithery and SkillzBench together?
Yes, and they complement each other well. Smithery is a marketplace for discovering and running MCP tools and services, while SkillzBench is building an independent verification and benchmarking layer for agent skills. A reasonable workflow is to connect tools through Smithery today, then use SkillzBench's free directory and benchmarks to vet which skills are safe and effective once it launches.
Why does skill security matter for small businesses?
Published research found roughly 26% of agent skills carry at least one vulnerability. Agent skills are instruction packages that tell coding and chat agents what to do, so a vulnerable or low-quality skill can introduce real risk into your business workflows. SkillzBench is being built specifically to surface this: security scans, license checks, and measured Lift for every skill, all free.
What happened with Smithery and Arcade.dev?
Arcade.dev acquired Smithery, announced August 5, 2026, with Smithery co-founder Anirudh Kamath joining the Arcade team. Arcade positions itself as an actions runtime for enterprise AI agents focused on security, reliability, and governance, and said the acquisition pairs Smithery's developer experience with enterprise-grade production controls.
How does SkillzBench make money if everything is free?
SkillzBench earns revenue from advertising shown in and around its blog articles, and never charges users for the directory, MCP connector, benchmarks, or alerts. The platform states that advertising money never influences scores or listings, and that Lift is always shown separately from ratings so monetization can't blur the evidence.