From Chatbots to Colleagues: What Actually Makes AI "Agentic"

Every vendor now claims their product is "agentic," which makes the term nearly meaningless. MIT Sloan's academic framing cuts through the noise: agentic AI systems are autonomous software that perceives, reasons, and acts in digital environments on behalf of human principals, with capabilities for tool use, economic transactions, and strategic interaction — a definition drawn from research by John Horton and colleagues at MIT Sloan.

That definition separates agentic AI from what most people actually use. A chatbot waits for your prompt and responds. Traditional automation follows rigid, pre-scripted rules. An agent, by contrast, takes a goal, decides which tools to use, executes multi-step work, and adapts when circumstances change.

MIT Sloan's Sinan Aral draws a further distinction: a single AI agent performing a bounded task differs from true multi-agent orchestration, where multiple different agents coordinate to complete a workflow together. Though most people use the terms interchangeably, Aral argues the agentic age is genuinely here — agents are already deployed at scale across the economy.

The real-world deployments back him up:

• JPMorgan Chase runs agents for fraud detection, financial advice, and automating loan approvals and legal-compliance work.
• Walmart uses LLM-powered agents for personal shopping, customer service, and merchandise planning.
• Siemens achieved 90% touchless processing, saving EUR 5 million annually, while Wells Fargo's loan-processing agents cut turnaround to minutes.

These aren't pilots. Per Mordor Intelligence, 61% of CEOs report integrating agents into core operations, and market analysts project roughly 40% compound annual growth for the category through the early 2030s.

The catch is that autonomy changes the security equation. When an agent selects a skill as part of a workflow, that code executes with the agent's permissions — no human click required, as security researchers studying the ClawHub malware campaign warned. Snyk's research found 36% of AI agent skills contain security flaws, and only 2% of enterprises use identity security tools actually built for AI agents, according to market research.

This is why verification tooling matters as agents gain real authority. SkillzBench, an independent testing lab for AI agent skills, is being built around exactly this gap: pinning every skill version to a commit SHA, security-scanning, and benchmarking each skill with and without it installed so teams can see whether it actually helps before an agent runs it autonomously.

The line between a chatbot and a colleague, in other words, isn't intelligence — it's autonomy plus accountability. The agents are already acting. The question is whether anyone verified what they're acting on.

The Pilot Purgatory Problem: Why 79% Are Adopting but Only 7% Are Operationalizing

Nearly four in five US executives say their organizations are adopting AI agents — yet only a tiny fraction have anything to show for it in production. This is the defining paradox of the agentic AI era: enthusiasm is everywhere, operational reality is not.

According to adoption research, 79% of US executives report AI agent adoption, but only 7% of organizations have reached the "Operationalizing" stage — the phase where significant ROI actually appears. The remaining 68% are stuck in Experimenting or Developing phases. The pattern is so common it has earned a name: pilot purgatory.

The stall rate is equally sobering. In the same research, 40% of respondents say more than 40% of their AI pilot projects never make it to production, often because the underlying infrastructure simply isn't ready for autonomous behavior. Capgemini's data tells a similar story: just 14% of organizations have actually deployed agents, while 61% are still exploring or preparing.

Why pilots stall: the productivity mirage

McKinsey's State of AI research exposes a gap that should worry any executive sponsor: 80% of organizations report improved personal productivity from AI, but only 37% report any EBIT contribution — a figure unchanged from the prior year. People feel faster; the P&L doesn't.

As MIT Sloan's Kate Kellogg puts it, reclaiming 20% of someone's time doesn't translate to a 20% labor-cost saving. Productivity gains that never touch workflow redesign, governance, and measurement rarely become profit.

The unglamorous 80%

The hardest truth comes from real deployment experience. A 2025 clinical AI agent deployment documented by MIT Sloan found that roughly 80% of the work wasn't modeling at all — it was:

• Data engineering and building a connected data foundation (78% of adopters report difficulty here)
• Stakeholder alignment across teams and functions
• Governance and access controls for autonomous behavior
• Workflow integration — redesigning processes around agents rather than bolting AI onto existing ones

This is why the gap exists. The demo is easy; the plumbing is brutal. McKinsey's high performers — still just 6% of respondents — stand out precisely because they redesign workflows enabled by AI rather than inserting AI into existing ones.

Closing the gap means treating agent deployment as an engineering discipline, not a prompt-engineering exercise. Tools that reduce the verification and governance burden — like SkillzBench's approach of security-scanning, benchmarking, and version-pinning agent skills before they reach production — exist precisely because the unglamorous work is where agentic AI programs either scale or stall.

The Autonomy-Trust Paradox: The Security Limit Nobody Budgeted For

The defining limit of agentic AI isn't intelligence — it's that agents act without asking. A compromised skill doesn't need a human to click a link or open a file; it only needs an AI agent to select it as part of a workflow, at which point the malicious code executes with the agent's own permissions, according to security researchers. That single sentence inverts decades of security training built around human vigilance.

The supply-chain evidence is already on the table. Koi Security audited all 2,857 skills on ClawHub and found 341 malicious ones, with 335 traced to a single coordinated operation dubbed "ClawHavoc." Snyk's ToxicSkills research went further, finding that 36% of AI agent skills contain security flaws, with roughly 900 skills — about 20% — classified as outright malicious. These attacks are also fast: malicious insertions have lasted as little as 42–90 minutes, exploiting automated dependency resolution before anyone notices.

Meanwhile, the defense side hasn't kept pace. Only 2% of enterprises use identity security tools built for AI agents, while 85% still rely on legacy identity tooling designed for human users, per adoption research. The gap is stark: organizations are handing agents autonomous permissions while securing them with infrastructure that assumes a person is at the keyboard.

The scale of exposure compounds the problem:

• Protect AI examined over 4 million models and flagged roughly 352,000 unsafe or suspicious issues across 51,700 models.
• The March 2026 LiteLLM compromise on PyPI potentially exposed 500,000 credentials, including API keys for Meta, OpenAI and Anthropic.
• Researchers now describe the AI supply chain as "the most attractive target in software security" — trusted repositories, automated consumers, payloads that execute on load.

This is no longer a niche engineering concern. NIST's CAISI has issued a Federal Register notice formally soliciting public input on securing AI agent systems, with the agency warning that unchecked risks "may impact public safety, undermine consumer confidence, and curb adoption of the latest AI innovations," as reported by Cybersecurity Dive. When a federal standards body opens a formal comment period on a technology class, the problem has crossed into national-priority territory.

The practical takeaway for any organization deploying agents is that autonomy requires verification before execution, not after. That's the design premise behind SkillzBench, which is building a control plane for agent skills — security-scanning every skill, pinning versions to commit SHAs, and letting teams define per-agent loadouts so an agent only ever sees the skills deliberately chosen for it. MIT Sloan's Kate Kellogg frames the broader principle well: as agency moves from humans to machines, "there's a real increase in the importance of governance and infrastructure to control and support agentic systems." The autonomy-trust paradox isn't a reason to abandon agents — but it is the line item nobody budgeted for.

How to Operationalize Agentic AI Without the Blind Trust

The gap between piloting agents and proving their value is where most agentic AI programs stall — only 7% of organizations have reached the operationalizing stage, while 68% remain stuck experimenting. MIT Sloan's Kate Kellogg puts the prerequisite plainly: without shared, robust metrics, it's difficult to prove value or know whether agentic systems actually accomplish their intended outcomes. The fix isn't more trust in agents. It's a verification, benchmarking, and equip workflow that treats every skill like unverified code — because that's what it is.

Verify before an agent ever sees a skill. Snyk's research found 36% of AI agent skills contain security flaws, and malicious skills execute with the agent's own permissions — no human click required. Verification should be non-negotiable:

• Pin every skill to a specific commit SHA so the version you tested is the version that runs.
• Security-scan each skill for prompt injection, credential exfiltration, and hidden payloads before deployment.
• License-check so an agent never pulls in code your legal team hasn't cleared.
• Re-verify on every upstream change, since supply-chain insertions can appear in under 90 minutes.

Benchmark with and without the skill. Kellogg's caution — reclaiming 20% of someone's time isn't a 20% labor-cost saving — applies directly to skills. A skill that feels helpful may add nothing measurable, or actively hurt. The only honest test is running the same model on the same task with and without the skill, then publishing the Lift in percentage points. That's the shared metric Kellogg argues governance requires, and it's exactly what SkillzBench is building into its benchmark pillar: sandboxed runs, identical conditions, and results shown separately from ratings so money never touches scores.

Control what each agent can access. Only 2% of enterprises use identity security tools built for AI agents, while 85% still rely on legacy tooling designed for humans. Explicit per-agent loadouts close that gap: an agent sees only the skills and tools someone deliberately chose for it — nothing more. This is the Equip pillar SkillzBench is developing, one MCP server plus per-agent toggles, so a coding agent never loads a skill with an unvetted license or an unresolved vulnerability.

The pattern across all three steps is the same: replace blind trust with evidence. NIST is now formally soliciting input on securing AI agent systems because, in its own words, unchecked risks may curb adoption of AI innovation entirely. Operationalizing agentic AI isn't about waiting for agents to become trustworthy — it's about building the verification and measurement infrastructure that makes trust unnecessary.