How to Make AI Agents?

Why Most AI Agent Projects Fail Before They Start

AI agents are everywhere in the headlines, but the numbers behind the hype tell a more sobering story. According to a survey of 3,466 senior leaders across 24 countries, 52% of executives say their organizations have already deployed AI agents — yet IBM's 2025 C-suite study found that only 25% of AI initiatives deliver the expected ROI, and just 16% have scaled enterprise-wide.

That gap isn't a technology problem. As IBM's consulting team puts it, "the problem isn't AI itself — it's how it's being deployed." Most failing projects skip the groundwork: they pick a use case because it sounds impressive rather than because it's measurable, and they never establish a baseline to prove improvement.

The pattern behind most early failures is predictable:

• "Spray and pray" adoption — launching agents at vague targets instead of repetitive tasks, manual handoffs, or processes with quality data behind them.
• No before-and-after measurement — without process decomposition up front, teams can't demonstrate ROI even when the agent works.
• Security treated as an afterthought — a 2024 OWASP study cited by Obsidian Security found that 67% of deployed LLM applications contained at least one exploitable prompt injection vulnerability.
• Tooling chosen before architecture — jumping to a framework or vendor before defining what the agent must actually do.

Security deserves special emphasis because agents are not chatbots. They make autonomous decisions, handle sensitive data dynamically, and often operate with elevated privileges — which is exactly why traditional perimeter defenses fail against prompt injection and token compromise. When AI-related breaches cost enterprises an average of $4.88 million with recovery times 38% longer than traditional attacks, retrofitting security is far more expensive than designing it in from day one.

There's also a component-level risk most teams overlook: the agent skills and instruction packages that extend what an agent can do. Published research has found that roughly 26% of community-published skills carry at least one vulnerability, which is why we're building SkillzBench — a free directory that security-scans, validates, and benchmarks every skill it lists, with a free MCP connector to equip your agents safely. Join the waitlist to get access as it rolls out.

The good news: the teams that do succeed treat agents as a structured deployment, not a silver bullet. The executives dedicating serious budget to agents — redesigning core processes rather than just automating tasks — report 88% ROI versus 74% on average. Success starts with picking the right use case, baselining it honestly, and making security a design requirement from the very first commit.

The Anatomy of a Functional AI Agent

Strip away the hype, and an AI agent is simply a system that does a job without you holding its hand. IBM's definition captures this well: an agent is a program capable of autonomously performing tasks by designing its own workflow and using available tools — not just chatting, but deciding, planning, and executing (IBM).

That definition surfaces three anatomical parts every functional agent shares. First, a reasoning core — the LLM that interprets instructions and makes contextual decisions. Second, a tool layer that lets the model act on the world: query databases, call APIs, send emails. Third, orchestration: the plumbing that manages control flow, persists state, and polices what tools the agent may touch.

Here the market data tells an interesting story. Agent frameworks appear in only 18% of organizations running LLM applications — up from 9% a year earlier — meaning roughly four in five teams still build without one (Splunk's analysis). And since MCP has commoditized tool integration, your framework choice now hinges on operations, not tool coverage: durable execution, state management, and observability.

Single-agent vs. multi-agent. Single-agent systems dominated 2024 because they demand less expertise and lower investment; multi-agent systems are now the fastest-growing segment, with coordination protocols and orchestration frameworks making them practical (MarketsandMarkets). The rule of thumb: match architecture to task complexity, and remember that multi-agent setups require deliberate loop prevention.

Minimalist SDKs vs. graph-based frameworks. This is the fundamental build trade-off:

• Minimalist SDKs (OpenAI Agents SDK, AWS Strands) get a working agent in roughly 3 lines — ideal for prototyping and validation.
• Graph-based frameworks (LangGraph, Google ADK, Microsoft Agent Framework) offer deeper control, checkpoint-first state management, and replay debugging for production scale.
• Hybrid paths let teams prototype on a lightweight SDK, then graduate the logic to a graph runtime once complexity grows.

The failure mode of skipping this decision is architectural spaghetti — control flow scattered across ad-hoc code that nobody can debug six months later. Graph-based frameworks exist precisely to prevent that, which matters when only 25% of AI initiatives deliver expected ROI and maintainability separates the winners from the shelfware.

One more anatomical layer deserves mention: instructions and skills. Beyond tools, agents increasingly rely on packaged instruction sets — SKILL.md files that teach a coding agent or chat agent how to handle a specific workflow. SkillzBench is building a free directory and MCP connector for exactly this layer, so builders can discover, verify, and benchmark skills rather than pasting unvetted prompts from random repositories.

Get the anatomy right — reasoning core, tool access, orchestration, and trusted instructions — and the rest of the build becomes configuration rather than invention.

Choosing Your Framework and Tools: Operations Over Hype

The Model Context Protocol has quietly rewritten the framework debate. When any agent can connect to any tool through a standard interface, choosing LangGraph over CrewAI over the OpenAI Agents SDK stops being about integrations and starts being about something less glamorous: whether your agent survives a crash.

That is the core insight from Splunk's technical analysis: frameworks are "orchestration runtimes, not reasoning engines." The LLM is the brain; the framework manages control flow, persists state, and acts as a boundary controller for tool invocation and input sanitization. In practice, that means you should evaluate candidates on three operational traits:

• Durable execution — can a half-finished workflow resume after a restart? Splunk's review rated only LangGraph positively on this dimension.
• State management — checkpoint-first persistence, ideally with replay and time-travel debugging for production incidents.
• Observability — full logging of every tool call, input, and output, since you cannot debug what you cannot see.

The architectural trade-off is real, though. Minimalist SDKs like OpenAI Agents SDK or AWS Strands can produce a working agent in roughly three lines — great for prototyping. Graph-based frameworks such as LangGraph or Google ADK demand more upfront but prevent the control-flow spaghetti that multi-step agents tend to accumulate at scale. Notably, frameworks appear in only 18% of organizations running LLM applications, up from 9% a year earlier — four in five teams still build without one.

On the strategy side, IBM Consulting recommends choosing an open orchestration layer over proprietary, vendor-tied agents. The reasoning is straightforward: lock-in limits your ability to pivot in a space moving this fast, and "architectural agility is a competitive advantage." With only 25% of AI initiatives delivering expected ROI, flexibility is not a luxury.

Whichever runtime you pick, capabilities increasingly come from agent skills — SKILL.md instruction packages that extend what Claude Code, Codex, Cursor, and Gemini CLI agents can do. Skills are powerful, but they are also third-party code and instructions entering your workflow. Published research found roughly 26% of skills carry at least one vulnerability, so verification is not optional: pin versions, check licenses, scan for security issues, and test skills in a sandbox before trusting them with real work.

That verification burden is exactly what SkillzBench is building to solve — a free, independent skills directory with security scanning, version pinning, and sandboxed benchmarking, plus a free MCP connector so your agent only ever sees the skills you have deliberately chosen. Treat skills like any other dependency in your orchestration stack: useful, but only after they have earned trust.

Step-by-Step: Building Your First Agent the Right Way

Most agent projects fail before the first line of code is written — not because the technology doesn't work, but because teams deploy without a plan. Only 25% of AI initiatives deliver the ROI leaders expect, and the problem, as IBM puts it, "isn't AI itself — it's how it's being deployed" (IBM's C-suite research). A structured, four-step approach fixes that.

Step 1: Pick the right use case. Resist the urge to "spray and pray" agents across every workflow. IBM's framework targets repetitive tasks, manual handoffs between systems, and processes with quality data — the conditions where autonomy pays off. Cost-reduction use cases are the smartest starting point because they're easier to measure and faster to realize returns.

Step 2: Establish a baseline. Before deploying anything, run a process decomposition exercise to capture a precise "before" picture: how long the task takes, how many people touch it, where errors creep in. Without this baseline, you'll never know whether your agent actually improved anything — you'll only have anecdotes.

Step 3: Build on an open orchestration architecture. Avoid proprietary, vendor-tied agents that lock you in. An open orchestration layer lets you swap models, tools, and frameworks as the landscape shifts — and in a market growing at roughly 46% annually (MarketsandMarkets), it will shift. IBM calls this architectural agility a competitive advantage. When equipping your agent with skills and tools, vet everything first: published research has found roughly 26% of agent skills carry at least one vulnerability, so pin versions, scan for security issues, and benchmark skills before loading them. SkillzBench is building exactly this — a free directory and MCP connector that validates, scans, and benchmarks skills so your agent only sees vetted capabilities.

Step 4: Measure ROI the right way. IBM recommends tracking three dimensions rather than a single vanity metric:

• Speed to outcome — how much faster the task completes end-to-end
• Cost to serve — the true cost per task, including human oversight time
• New capabilities — work the business couldn't do before, only now possible

The payoff is real when you follow the discipline. Google Cloud's survey of 3,466 senior leaders found early adopters reporting higher ROI in customer service (43% vs. 36% average) and marketing (41% vs. 33%) (the study). As IBM summarizes: agents are not a silver bullet nor a science project — they're a powerful tool when used with purpose, structure, and vision.

Securing and Scaling: From Working Prototype to Production

A working prototype proves your agent can do the job. Production proves it can do the job safely, repeatedly, and at scale — and that's where most projects stumble. With 67% of deployed LLM applications containing at least one exploitable prompt-injection vulnerability, security can't be an afterthought bolted on before launch.

The core principle is zero trust: verify explicitly, grant least privilege, and assume breach. Because agents process sensitive data dynamically, make autonomous decisions, and often run with elevated permissions, traditional perimeter defenses simply don't apply. Grant only the minimum permissions each task requires — AI tools frequently request overly broad access during integration, and that excess becomes your attack surface.

In practice, a production-hardened agent needs four defenses working together:

• Input validation against prompt injection on every request, since attackers target the instructions, not just the data
• Output filtering that scrubs sensitive data before it leaves the agent's boundary
• API keys that expire within 90 days, with emergency revocation ready
• Real-time behavioral monitoring — API call patterns, data access sequences, token usage — paired with automated response

One step teams skip at their peril: test your rollback procedures before deployment, not during an incident. AI-related breaches cost enterprises an average of $4.88 million with recovery times 38% longer than traditional attacks, so a rehearsed recovery path is cheap insurance.

Security gets you to production; process redesign gets you ROI. A Google Cloud study of 3,466 senior leaders found that executives dedicating at least half their AI budgets to agents report 88% ROI versus a 74% average. As Google Cloud's Oliver Parker puts it, early adopters aren't just automating tasks — they're redesigning core business processes around what agents do well.

Agents also drift. Models update, upstream dependencies change, and skills that once boosted performance can break or become liabilities. Ongoing verification and re-benchmarking keep agents current — re-test capabilities after every change, and re-validate the third-party skills and tools your agent depends on. SkillzBench is building exactly this loop: a free directory where every skill version is pinned, security-scanned, and benchmarked for measurable Lift, plus a free MCP connector so each agent sees only the verified skills you equip it with. The waitlist is live now, so builders can be first in line when it ships.

The teams that win won't be the ones with the cleverest prototype. They'll be the ones who treat agents as production infrastructure — least-privileged, monitored, rollback-tested, and continuously re-verified.