Coding With Agents in Rails: Setup, Skills and Pitfalls in 2026

The New Reality: Agents Write Most Code, Humans Review It All

In 2026, the most striking engineering story isn't that AI agents can write code — it's how much of it they now write. OpenAI's team shipped roughly a million lines of code with zero lines written by human hands, merging 1,500 pull requests in five months, as documented in their harness-engineering account. As Ryan Lopopolo put it: "Humans steer. Agents execute."

That sounds like a triumph, and in many ways it is. The OpenAI team estimates the product shipped in about a tenth of the expected time. But the same story contains a quieter warning: the engineers' primary job became enabling and supervising agents, not typing.

The bottleneck has moved from writing to judging. When one Smartsheet engineer produced 7X her team's code, six teammates spent the majority of their time reviewing it — a case study that captures the new dynamic. Reviewers now face two-digit daily PR volumes instead of a handful, and every review demands fresh context-gathering because nobody wrote the original code.

The pressure is psychological as well as logistical. Intuit's Carol Lee described the weight of it: "If I mess up this review, I was the gatekeeper of this code. And if I mess it up, that might be my fault." Decision fatigue is becoming a measurable occupational cost of agent-driven velocity.

And the stakes are higher than reviewer burnout. A DryRun Security study found 87% of agent-generated PRs contained at least one vulnerability, with ten recurring, predictable failure classes — most commonly, security components that were built but never wired in. Agents produce working software fast; security simply isn't part of their default thinking.

So the challenge facing every Rails team in 2026 is velocity without control:

• Agents generate code far faster than humans can meaningfully review it
• 87% of agent PRs may carry at least one vulnerability, often rooted in planning decisions
• High individual output can quietly consume an entire team's review capacity

The teams that win won't be the ones generating the most code — they'll be the ones who make agent output trustworthy before it ever reaches a human reviewer. That means deliberate setup, verified instruction packages, and security review that starts at the design stage. SkillzBench is building toward exactly this problem: a free directory and MCP connector that security-scans and benchmarks agent skills before your team equips them, so the judgment you spend goes further.

Setup Beats Tool Choice: Building a Harness Your Rails App Can Trust

The best Rails teams in 2026 aren't the ones that picked the "right" agent — they're the ones that built an environment any agent can succeed in. According to current tooling analysis, there's no single best agent; developers choose based on leverage, not brand. The real lever is setup.

The most useful mental model is Agent = Model + Harness: the model supplies intelligence, but the harness — your docs, tests, linters, and access to logs and metrics — turns that intelligence into a reliable contributor. OpenAI's harness-engineering experiment proved the point dramatically: roughly one million lines of code with zero lines written by humans, and 1,500 PRs merged in five months.

Three harness patterns drove that result:

• A ~100-line AGENTS.md treated as a "table of contents" pointing to structured docs — monolithic instruction files crowd out task context and rot.
• All critical knowledge kept in-repo. As OpenAI's team put it, anything the agent can't access in-context "effectively doesn't exist" — your Slack threads and tribal conventions are invisible to it.
• Architecture enforced mechanically through custom linters and structural tests, so agents can move fast without drifting.

For Rails teams, these principles translate naturally even though Rails-specific agent guidance is still an emerging practice — no published research yet covers ActiveRecord or migrations in agent contexts. But the general rules apply cleanly: document where ActiveRecord models end and service objects begin, keep schema conventions in-repo rather than in a senior dev's head, and write structural tests that fail when an agent puts business logic in a migration or reaches across module boundaries. "Boring," conventional Rails is exactly what agents model best, because convention removes ambiguity.

The payoff is reliability at speed. OpenAI observed single agent runs working on one task for upwards of six hours — but only because the environment made success the default path. That's also where vetted agent skills fit: instruction packages can encode your team's conventions and checks, though they vary wildly in quality and safety. SkillzBench (skillzbench.com) is building a free directory and free MCP connector that security-scans and benchmarks skills before you load them, so your harness extends with packages you can actually trust. Build the harness well, and the tool choice almost makes itself.

The Security Trap: 87% of Agent PRs Have a Vulnerability

Your agent just shipped a pull request that works, passes its tests, and quietly leaves the door unlocked. That's not a hypothetical — it's the norm. A DryRun Security study found that 87% of agent-generated PRs (26 of 30) contained at least one vulnerability, with 143 security issues surfacing across 38 scans of Claude Code, Codex, and Gemini.

The failures aren't random. DryRun identified ten recurring vulnerability classes — broken access control, OAuth implementation failures, missing WebSocket authentication, weak JWT secret management, session management failures, and more. As DryRun's CEO James Wickett put it, "AI coding agents can produce working software at incredible speed, but security isn't part of their default thinking."

The most common failure mode is almost comical: security components built but never wired in. An agent defines rate-limiting middleware but never mounts it. It scaffolds OAuth flows but omits the state parameter. The code looks secure in review because the security artifacts exist — they're just not connected to anything.

This is precisely why regex-based static analysis keeps missing these bugs. Pattern-matching scanners look for known dangerous strings; they can't tell that a beautifully written Rack::Attack initializer is dead code because nothing references it. Contextual analysis, by contrast, identified 88% of seeded vulnerabilities in DryRun's 2025 SAST Accuracy Report. Many of these flaws originate in design and planning decisions, not syntax — which means security review has to happen at the design stage, not just in the diff.

For Rails teams, that means a deliberate checklist before merging agent PRs:

• Access control: verify `before_action` filters actually run on every controller route the PR touched — not just the ones the agent tested.
• JWT secrets: confirm secrets come from Rails credentials or ENV, never hardcoded fallbacks agents love to add.
• Sessions: check cookie flags (`secure`, `httponly`, `same_site`) and that any new session store is actually configured.
• Middleware: grep for defined-but-unmounted middleware — the single most common agent failure.
• OAuth: require the state parameter on every flow, and verify refresh tokens are revocable.

One more layer worth considering: agents follow the instruction packages you give them, and unvetted skills can import their own risks — published research found roughly 26% of agent skills carry at least one vulnerability. SkillzBench is building a free directory and MCP connector that pins every skill to a commit SHA and security-scans each one before it reaches your agent, so your Rails conventions and security checks come from verified sources rather than unvetted GitHub packages. Working code isn't the bar — wired-in, verified security is.

Working Like a Senior: Heavy Context, Small Changes, Cross-Tool Review

The engineers getting the best results from coding agents aren't the ones generating the most code — they're the ones working like careful seniors. The pattern is simple: load heavy context, make small changes, and never trust a single tool to check its own work.

Research on how experienced developers actually use agents found that seniors "load a lot more in context and then making smaller changes," according to Stack Overflow's reporting on agent adoption. The reasoning is structural: when nobody on the team wrote the original code, review requires more context-gathering than ever, and large diffs simply don't get the scrutiny they need.

The cautionary tale is Smartsheet, where one engineer produced 7X her team's code — while six teammates spent most of their time reviewing it. Volume became a tax on everyone else. In a Rails context, that means giving the agent your conventions, schema context, and test expectations up front, then asking for a narrow, reviewable diff rather than a sprawling refactor.

AI-assisted coding demands AI-assisted reviewing, and practitioner guidance recommends reviewing with a different tool than the one that wrote the code. The security stakes justify it: a DryRun Security study found 87% of agent-generated PRs contained at least one vulnerability, most often security components built but never wired in.

On measurement, drop the vanity stats:

• Track DORA-style change failure rate, not lines of code or "percentage written by AI" — the industry is already regressing on Goodhardt's Law.
• Watch cost-per-attempt: Kilo's benchmark data shows GPT-6 Astra at $107.29/attempt vs. DeepSeek V4.1 Flash at $2.58 — a ~40x gap for ~4 capability points.
• Review security at the design stage, since many agent vulnerabilities originate in planning, not coding.

Agent skills — SKILL.md instruction packages — are one way to encode your conventions and security checks so every attempt starts informed. But with published research finding roughly a quarter of public skills carry vulnerabilities, vetting matters. SkillzBench is building a free directory and MCP connector that pins every skill to a commit SHA, security-scans it, and benchmarks its Lift in a sandbox — so your "heavy context" habit starts from verified material, not unvetted GitHub packages.

Skills, proven. Browse the free skills directory and join the waitlist at skillzbench.com.

Encoding Your Standards: Vetted Agent Skills as the Missing Layer

If your AGENTS.md is the table of contents, agent skills are the chapters your team actually writes. A skill is just a packaged bundle of instructions — a SKILL.md plus any supporting files — that tells Claude Code, Codex, Cursor, Gemini CLI, or chat-app agents how to do one job well. Encoding your conventions as skills turns tribal knowledge into something the agent can load on demand, from "always run brakeman before committing" to "use service objects, not fat controllers."

The trouble is that skills are everywhere and almost none of them are vetted. Published research has found that roughly 26% of published skills carry at least one vulnerability, and the same study showed 87% of agent-generated PRs contain at least one security issue. Most of those issues are structural, not typos: rate limiting defined but never connected, OAuth state parameters missing, JWT secrets handled carelessly. A skill that teaches your agent to scaffold a Rails endpoint is only useful if it also teaches it not to leave the auth door open.

A few rules of thumb keep a private skill library safe:

• Pin every skill to a commit SHA. A floating "main" branch is a supply-chain risk; a hash is auditable.
• Security-scan before adoption, then re-scan on every update. Treat skills like npm packages.
• Benchmark in a sandbox with and without the skill, and publish the Lift in percentage points.
• Keep ratings and Lift separate. Quality scores should never be influenced by who paid for what.

That is the layer SkillzBench is building: a free directory and free MCP connector that crawls skills from GitHub, dedupes them, pins each version to a commit SHA, validates and license-checks, security-scans, quality-judges, then benchmarks Lift in a sandbox so you can see what a skill actually does. Every agent gets its own loadout — toggle skills and tools per agent, and the agent only sees what you chose. The directory, the MCP, the benchmarks, and the alerts are all free; revenue comes from advertising around the blog, and ad money never touches scores.

If you want skills that are proven, not just published, join the SkillzBench waitlist for early access to the directory and MCP.