Is anyone else noticing that people building AI agents are forgetting basic software engineering??

The Evidence: Agent Projects Are Skipping the Fundamentals

If you want to know what's actually hard about building AI agents, don't ask the hype — ask the data. And the data says the hardest problems aren't prompt engineering at all. They're orchestration, retrieval, and contract enforcement: classic software engineering problems that teams keep underestimating.

A large-scale empirical study from Delft University and JetBrains Research analyzed 3,191 Stack Overflow questions and 64,098 GitHub issues across 18 widely used agent repositories. The pattern was striking. Installation and prompting problems — the visible, easy ones — get spotted and fixed quickly. Meanwhile, the hardest and longest-lived issues cluster around RAG engineering, vector stores, and orchestration. These questions take longer to resolve and are far more often left unanswered.

The study's authors put it plainly: configuration and prompting problems are easy to spot, but issues about retrieving information or orchestrating agents are less visible and much harder. Their conclusion is a direct rebuke to complexity-first building — agent tooling and evaluation should prioritize reliability, observability, and contract enforcement above all else.

This isn't an isolated finding. The readiness gap at the top of organizations tells the same story. IBM's Institute for Business Value found that 80% of CIOs and CTOs report CEO-driven AI transformation mandates, yet only 11% say they're fully ready for the scale of agent deployment expected in the next year. Demand is racing ahead of engineering discipline.

The gaps show up in the fundamentals teams skip:

• Testing discipline — errors compound across multi-step agent chains, and similar prompts can produce different tool-call sequences, yet most teams still test like they're shipping deterministic code.
• Observability — agents that work in local testing break in production when the wrong tool gets picked or a conversation loses context, and teams can't reproduce why.
• Dependency and supply-chain hygiene — the skills ecosystem lets anyone publish with just a Markdown file and a week-old GitHub account, with no code signing or security review by default.
• Human review — automated scanning alone catches only a fraction of malicious agent skills, so manual review remains essential for anything touching production.

Even the leading AI labs agree the failure mode is complexity-first thinking. Anthropic, after working with dozens of teams, found the most successful implementations used simple, composable patterns rather than heavy frameworks, which add abstraction layers that make agents harder to debug. OpenAI's guidance echoes this: validate whether you need an agent at all, because a deterministic solution may suffice.

It's worth noting that no single source directly measures how many teams are skipping these practices — the evidence comes from documented failure modes, security audits, and the corrective guidance the industry keeps having to publish. But when Delft researchers, IBM's survey data, and both OpenAI and Anthropic independently converge on the same diagnosis, the pattern is hard to dismiss as anecdote.

This is precisely why we're building SkillzBench: to bring basic engineering rigor — version pinning, security scanning, sandboxed benchmarking — to the agent skills ecosystem, free for everyone who joins the waitlist.

The Skills Supply Chain Is Early npm — Without the Safeguards

If you downloaded an agent skill in the past month, there's a 13% chance it contains a critical security flaw — and a non-zero chance it's quietly exfiltrating your credentials right now. That's not fear-mongering; it's what a recent security audit by Snyk actually found.

The numbers behind that warning are stark. Snyk's researchers examined 3,984 agent skills and discovered that 13.4% carry at least one critical security issue, while 36.82% have at least one flaw at any severity. They found 76 confirmed malicious payloads — credential theft, backdoors, data exfiltration — hiding inside skills that look like ordinary productivity tools.

What makes this ecosystem feel like early npm isn't just the malware. It's how easy publishing is. To get a skill listed, you need only a SKILL.md file and a GitHub account that's one week old. As Snyk put it: no code signing, no security review, no sandbox by default. Meanwhile, daily submissions jumped from under 50 in mid-January 2026 to over 500 by early February — a 10x surge in weeks that has completely outpaced any safety infrastructure.

The threat is already real, not theoretical. A coordinated malware campaign hit ClawHub with 30+ malicious skills targeting Claude Code and OpenClaw users, and eight of those malicious skills remained publicly available at the time of the audit's publication.

Even skills written with good intentions can be dangerous by design. Two patterns deserve special attention from anyone adopting skills:

• Third-party content fetching: when a skill pulls in untrusted web content, an attacker can inject hostile instructions into what the agent reads — the skill author did nothing wrong, yet the agent gets compromised.
• Unverifiable runtime dependencies: a skill that fetches external instructions when it runs looks perfectly safe during review, but its behavior can be changed at any moment on infrastructure the author — or an attacker — controls.
• Inherited permissions: skills typically gain shell access, file system read/write, credential access, messaging ability, and persistent memory across sessions by default — far more power than most tasks require.

For non-technical readers, here's the plain-English version: a skill is a set of instructions your agent follows, and an agent with those permissions can read your files, use your accounts, and send data anywhere. Installing an unvetted skill is closer to running someone else's code with your keys in the door than downloading a helpful template.

The encouraging news is that the fix is classic supply-chain hygiene: know what a skill can access, check its provenance, and prefer skills that have been security-scanned and version-pinned to a specific commit. That's exactly the gap SkillzBench is building to close — a free directory that pins every skill to a commit SHA, runs security scans, and shows a trust score from security, quality, license, and maintenance signals, with a free MCP connector to follow. The waitlist is open now.

Snyk's researchers frame the stakes well: the current state resembles early package managers before security became a first-class concern, and the question is whether the community learns from those hard lessons — or repeats them.

What OpenAI and Anthropic Both Say: Start Simple, Then Engineer

When two competing labs independently reach the same conclusion, it's worth paying attention. Both OpenAI and Anthropic tell agent builders the same thing: start simple, and treat complexity as something you earn — not something you install on day one.

Anthropic's engineering team, after working with dozens of teams across industries, found that "the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns." Their reasoning is refreshingly practical: frameworks add layers of abstraction that can obscure the underlying prompts and responses, making agents harder to debug. If you do use a framework, they warn, make sure you understand what's under the hood — incorrect assumptions there are a common source of customer error.

OpenAI's guidance lands in the same place from a different angle. They caution that "while it's tempting to immediately build a fully autonomous agent with complex architecture, customers typically achieve greater success with an incremental approach." They also push builders to validate the use case first — because otherwise, a deterministic solution may suffice. Sometimes the right answer isn't an agent at all; it's a plain script.

The convergence extends beyond architecture into classic engineering hygiene:

• Test your tools thoroughly. OpenAI describes tools as needing to be "well-documented, thoroughly tested, and reusable" — ordinary software discipline, applied to agent tooling.
• Define explicit exit conditions. Every orchestration loop needs defined stopping points — tool calls, structured output, errors, or max turns — a basic control-flow concept that prevents runaway agents.
• Establish a baseline before optimizing. OpenAI recommends setting up evals to measure performance before touching models, so improvements are measured rather than assumed.
• Keep humans in the loop. Anthropic stresses extensive sandboxed testing plus guardrails, and notes that human review remains crucial for ensuring solutions align with broader system requirements.

LangChain frames the same risk from the tooling side: "abstraction that obscures failure modes costs more in debugging time than it saves in setup time." Their own documentation of common failure modes — agents that work locally then break in production, pick the wrong tool, or burn triple the tokens in an unreproducible loop — reads as a warning that complexity-first building is a documented failure mode, not a stylistic preference.

This matters for skills and tool selection too. An agent is only as reliable as the components it composes, and most teams have no way to verify whether a third-party skill actually helps. That verification gap is exactly what SkillzBench is building: a free directory that benchmarks each agent skill in a sandbox — with and without the skill, on the same model — and publishes the measured lift in percentage points, so builders can compose simple, verified patterns instead of guessing. The waitlist is open now.

The consensus is unambiguous: the labs with the most deployment experience are telling builders to do less, earlier — and to engineer what they do build.

The Fix: Testing, Scanning, and Human Review in the Pipeline

The good news: none of this requires exotic tooling. It requires doing what software teams already know how to do — test, scan, and review — adapted to agents' non-determinism.

IBM recommends a three-tiered testing approach: component tests for individual pieces, trajectory tests for the paths an agent takes, and end-to-end testing with human review. The trajectory layer matters because errors compound in multi-step chains, and similar prompts can produce different tool-call sequences. Success criteria should be defined before building, not after.

On the security side, OWASP prescribes standard software-engineering integration patterns: automated skill scanning wired into GitHub Actions or GitLab CI pipelines, pre-commit hooks, and registry webhooks that reject skills with critical vulnerabilities before publication. Their guidance emphasizes confidence scoring to manage false positives, non-blocking scans that don't break development workflows, and isolated execution environments.

But here's the crucial nuance: automation alone is not enough. Cisco's skill-scanner data shows rules-based scanning alone catches only 7.7% of malicious skills at HIGH severity. Adding an LLM judge improves things substantially — 66.7% of malicious skills reach human review at MEDIUM+ severity — but the project is explicit that no automated tool detects every technique, especially novel or zero-day attacks. "Human review remains essential," and high-risk deployments should pair scanner results with manual code review and threat modeling.

A layered pipeline looks like this:

• Component and trajectory tests run continuously, with success criteria and edge cases defined up front.
• Automated security scans run in CI/CD on every skill and tool before it's installed or published.
• Version pinning ties every dependency to a specific commit so behavior can't silently change upstream.
• Human review gates anything headed to production, especially skills with broad permissions.

No single layer is sufficient — Cisco's 7.7% figure proves scanning alone fails, and human review alone doesn't scale. Layered discipline is the fix. It's the same verification gap we're building SkillzBench to close: pinning every skill to a commit SHA, security-scanning and benchmarking each one in a sandbox, and keeping Lift separate from trust scores — free, with the waitlist open now.

Only 11% of CIOs and CTOs say they're fully ready for the scale of agent deployment expected this year, according to IBM's survey research. The teams who get there won't be the ones with the most sophisticated architecture. They'll be the ones who never stopped doing the fundamentals.

Your Pre-Install Checklist for Any Agent Skill

Installing a skill without inspecting it is the agent-building equivalent of running an unvetted npm package with root access — except the stakes are higher, because skills inherit shell access, file system read/write, and credential access by default. A five-minute pre-install check can mean the difference between a productivity boost and a compromised machine.

Before you adopt any skill, run through this checklist:

• Verify what the skill can actually access. Skills inherit dangerous permissions by default, including shell execution, credential access, and persistent memory across sessions, so read the SKILL.md and ask what the agent gains the moment the skill loads.
• Scan for hardcoded secrets and runtime fetches. A Snyk audit of 3,984 skills found 10.9% contain hardcoded secrets, and 17.7% fetch untrusted third-party content — a setup where "the skill author did nothing wrong... yet the agent is compromised." Treat any skill that pulls external instructions at runtime as unverifiable.
• Check for version pinning. Publishing a skill requires only a Markdown file and a week-old GitHub account — no code signing, no security review, no sandbox. If a skill isn't pinned to a specific commit, what you reviewed today may not be what runs tomorrow.
• Confirm it was security-scanned — then read it yourself. OWASP calls automated scanning essential, but Cisco's data shows rules alone catch only 7.7% of malicious skills at HIGH severity. "Human review remains essential" is not a disclaimer; it's the actual control.
• Benchmark the actual lift. OpenAI's guidance is blunt: set up evals to establish a performance baseline before optimizing anything. A skill that doesn't measurably improve your agent's output is just added attack surface.

That last point is the one most teams skip entirely, and it's where the ecosystem's biggest gap sits. Nobody currently publishes standardized, sandboxed measurements of what a skill actually contributes — tested with and without the skill, on the same model, in an isolated environment. Anthropic's engineering guidance points the same direction: extensive testing in sandboxed environments, with human review to confirm solutions align with broader system requirements.

This is precisely the infrastructure SkillzBench is building: pinning every skill to a commit SHA, security-scanning and quality-judging each one, and benchmarking real lift in a sandbox before publishing the numbers — with a free MCP connector that lets you toggle exactly which skills and tools each agent sees. The directory, benchmarks, and connector will all be free. If you want vetted, version-pinned, benchmarked skills the moment they're available, the waitlist is open now.