How to pick which AI tool to use?

The Adoption-Without-Evaluation Trap

Businesses are racing to adopt AI tools — but almost no one is checking whether those tools actually work. The result is a widening gap between what companies deploy and what they can prove delivers value.

The scale of adoption is real. Census Bureau data shows overall U.S. business AI use hovering between 17% and 20% over six months, while a U.S. Chamber of Commerce survey puts small-business AI use at nearly 60% — more than double the 2023 rate. That divergence is largely methodological: the Census Bureau revised its question in November 2025 from AI use "in producing goods or services" to "in any business function," and surveys define "use" differently. Whichever figure you trust, the direction is the same: adoption is climbing fast.

What's not climbing is evaluation. According to Liferay's research, 54% of companies are running AI agents in production or actively piloting them — yet only 25% measure AI's impact with clear KPIs, and 13% don't measure at all. Most buyers are choosing tools with no evidence those tools help.

The pattern shows up clearly among small businesses too. A survey of roughly 1,000 small business owners found 76% are using or exploring AI, but 51% remain "AI Explorers" who haven't yet seen enough value to commit. They want proof before they invest — and the market isn't giving it to them.

If you're evaluating AI tools right now, the evidence gap means you need to build your own:

• Define what success looks like before you buy — which task, which metric, which baseline.
• Test the tool against that baseline with and without it, the same way, every time.
• Screen for security and accuracy first: companies rank better accuracy (42%) and stronger security (36%) as top requirements.

Notably, unclear ROI ranks last among barriers companies report — just 12%. Businesses aren't doubting AI pays off; they simply lack the frameworks to evaluate specific tools. As Liferay's report puts it, "moving fast on adoption isn't the same as capturing value from it."

This measurement vacuum is exactly what independent benchmarking is meant to fill. SkillzBench is being built to test every agent skill in a sandbox with the same model, with and without the skill, and publish the resulting Lift in percentage points — the kind of before-and-after evidence most buyers currently go without. With published research finding roughly 26% of agent skills carry at least one vulnerability, evidence-based evaluation isn't a nice-to-have. It's the difference between adopting AI and actually benefiting from it.

Start With the Function, Not the Tool

Most AI tool purchases fail for a predictable reason: teams fall in love with a product before they've identified the problem it's supposed to solve. The research is clear that the strongest signal for choosing well isn't a vendor's feature list — it's matching the tool to a specific business function.

The U.S. Census Bureau recognized this when it redesigned its AI survey in November 2025, measuring AI use across 15 distinct business functions — from finance and HR to customer service, marketing, IT, and R&D. That framework matters because adoption patterns diverge sharply depending on where you look. Information sector firms use AI at 39.7% and finance and insurance at 33.9%, roughly double the national average of 19.8%, while retail trade trails at roughly 14%. A tool that's transformational for a data-heavy analytics team may be irrelevant to a store operations manager.

The same function-first pattern shows up in how companies actually deploy AI. According to Liferay's research on AI agent adoption, the top use cases are data analysis and reporting (35%), customer support (32%), IT service desk (21%), finance operations (19%), and content production (18%). Among companies already using AI, data analysis and writing/editing content each hit 44%. These aren't exotic applications — they're the everyday workloads where pain is most visible and results are most measurable.

So before you open a single pricing page, do this exercise:

• Name the one business function causing the most recurring pain — the one your team complains about weekly.
• Estimate the hours or dollars that function currently consumes, so you have a baseline.
• Check whether that function appears among the proven high-value use cases (data analysis, customer support, content) or requires a more specialized tool.
• Define what success looks like numerically before evaluating any vendor against it.

That last step is where most organizations stumble. Only 25% of companies measure AI's impact with clear KPIs, even as 54% run or pilot AI agents in production — and unclear ROI ranks last among adoption barriers, at just 12%. Companies aren't doubting AI pays off; they lack a framework for evaluating whether a specific tool helps their specific function. As Liferay's co-founder Bryan Cheung puts it, "Building that foundation early gives teams more confidence to use AI in meaningful ways without creating unnecessary risk or complexity."

If your highest-pain function involves agentic workflows — coding, analysis, support automation — the same evidence-first logic applies to the agent skills that extend those tools. SkillzBench is building a free directory where every skill is security-scanned, version-pinned, and benchmarked with and without the skill to measure actual Lift, so you can judge skills by demonstrated function rather than marketing claims. Pick the function first, then demand proof — the tools worth your time will survive that scrutiny.

The Five Criteria That Actually Predict Tool Fit

Most businesses now run AI somewhere in their workflow, yet fewer than half can prove it's paying off. If you can't measure whether a tool helps, you're not choosing — you're guessing. Five criteria, drawn directly from current adoption research, will get you past the guesswork.

1. Demonstrated value. Only 25% of companies measure AI's impact with clear KPIs, according to Liferay's survey of 500 organizations — meaning 75% are flying blind on whether their tools deliver. Notably, unclear ROI ranks last among adoption barriers (just 12%): companies aren't doubting AI pays off, they lack frameworks to evaluate it. Before adopting, run a before/after benchmark on your own tasks. This is exactly the evidence-based approach SkillzBench is building for agent skills — testing every skill in a sandbox with the same model, with and without it, to publish the measurable Lift.

2. Accuracy. Better accuracy is the single most-cited tool requirement at 42%, per the same Liferay research. A tool that's 90% right in a demo may be 60% right on your data. Test on your real workload, not vendor samples.

3. Security. Stronger security controls rank second at 36%, and the stakes are real: published research has found roughly 26% of agent skills carry at least one vulnerability. Screen for security before deployment, not after an incident.

• Demand evidence of value before committing — a before/after benchmark on your tasks, not vendor claims
• Verify accuracy on your own data, since 42% of companies name it their top requirement
• Security-scan and version-pin anything you install — a quarter of published skills carry vulnerabilities
• Check integration with your existing workflows and trusted internal data
• Set governance boundaries before the tool touches production

4. Simplicity and integration. Small business owners consistently report that simpler, more secure tools are critical for broader adoption, per NSBA survey data. Integration with existing systems (24%) and access to trusted internal data (25%) also rank among top requirements — a tool that forces workflow changes rarely survives past the pilot.

5. Governance boundaries. Decide which data a tool can access, what actions it can take, where human approval is required, and how success will be measured — before adding it. The principle of least privilege applies: an agent should only see what you've explicitly chosen for it, which is how SkillzBench's per-agent loadouts are designed to work.

Score every candidate tool against all five criteria. A tool that fails on security or measurability isn't a bargain at any price.

From Assistants to Agents: Why 'Matching a Tool' Has Changed

The question used to be simple: which model gives the best answers? That framing is now obsolete. As AI shifts from assisting people to acting on their behalf, the more useful question is which tool can actually reach your data, your systems, and your permissions.

OpenAI draws the line plainly: assistants help people think through work, while agents help them complete it. Agents take objectives and preferences rather than step-by-step instructions, acting with autonomy across external environments. Harvard Business School researchers go further, describing these systems as digital coworkers rather than tools, which makes human oversight essential for high-stakes tasks.

This changes what "matching a tool" means. Effectiveness now depends on capability matching — access to the right context, tools, and permissions — not on picking the strongest model. OpenAI's enterprise data shows frontier firms generate 8.3× the output tokens per active user of typical firms, up from 2.6× in January, while 21% of their users rely on plugins weekly versus 9% elsewhere. Same models, deeper integration.

Agentic adoption is also spreading well beyond engineering. Weekly active enterprise users of OpenAI's Codex grew fastest in legal (108×), sales (41×), recruiting (41×), and marketing (26×), compared with 5× for engineering.

Before adding agents, define the boundaries:

• Which data can the agent access?
• What actions can it take without asking?
• Where is human approval required?
• How will success be measured?

That last question exposes a real gap. Only 25% of companies measure AI's impact with clear KPIs, even as 54% run agents in production or pilot them. Notably, unclear ROI ranks last among adoption barriers — buyers aren't skeptical that AI pays off; they lack a framework for judging whether a specific tool does.

This is where independent evidence matters, and it's what SkillzBench is building: sandbox benchmarking that tests every agent skill with and without the skill, publishing the Lift separately from ratings. Because capability matching only works when you can verify what a skill actually adds — and what it risks — before you hand it context and permissions.

A Practical Selection Workflow You Can Run This Week

Stop shopping for tools before you define the problem. The most effective selection process starts with a specific business function, not a software catalog.

The U.S. Census Bureau now tracks AI usage across fifteen distinct business functions, highlighting that adoption is highly specific to operational needs. Instead of buying a generic suite, identify the single area where your team faces the clearest pain point, such as data analysis or customer support. This focus ensures your evaluation criteria remain relevant to actual workflow improvements.

Before you evaluate any vendor, write a one-page governance boundary document. This document must explicitly define data access permissions, allowable agent actions, required human approval points, and the specific success metric you will use. As noted in recent industry research, only 25% of companies currently measure AI impact with clear KPIs, leaving most teams without a reliable baseline for value.

With your boundaries set, shortlist tools against five non-negotiable criteria:

• Demonstrated value in your specific use case
• Accuracy rates under real-world conditions
• Security controls and compliance posture
• Simplicity of integration for your team
• Compatibility with existing workflow systems

Do not accept marketing claims at face value; demand before-and-after evidence of value. This is critical because survey data indicates that 51% of small business owners remain "AI Explorers" who have not yet seen enough tangible value to commit to a tool.

For agent skills specifically, you must verify version pinning and security scans. Published research finds that approximately 26% of agent skills carry at least one vulnerability, making security a primary selection criterion. This is where independent verification becomes essential. SkillzBench is building a free directory and MCP connector to help you extend your existing agents safely. The platform is designed to test every skill in a sandbox, publishing the Lift in percentage points so you can see exactly how much a skill improves performance without it.

You do not need to guess which tools work. You need evidence. By defining your pain point, setting governance boundaries, and demanding measurable proof, you move from speculation to strategic adoption. This approach protects your data, respects your team's time, and ensures that every AI investment delivers a clear, quantifiable return.