Can Claude Code connect to local LLMs?

Why Run Claude Code Against a Local LLM at All?

If your codebase contains client data, security research artifacts, or anything you'd rather not ship to a third-party server, the appeal of running Claude Code entirely on your own hardware is obvious. One developer put it bluntly: when you're doing pentesting or reverse engineering, sending disassembled code or extracted strings to a cloud API is "at best, a questionable decision."

Privacy isn't the only driver, though. Cost and usage limits push people over the edge too. One practitioner reported spending over $300 in Anthropic credits since the start of the year, then hitting usage limits that forced 4–5 hour waits before he could work again. Another canceled their $20/month plan entirely and rebuilt their workflow around local models in VS Code.

The motivations cluster into three groups:

• Privacy-sensitive work where code, strings, or client data shouldn't leave your machine.
• Escalating costs — subscription fees plus credit burn that adds up fast for heavy users.
• Usage caps that make cloud Claude Code "almost unusable" during peak work, with multi-hour lockouts.
• Curiosity and control — owning the full stack, from model weights to tooling.

Here's the honest part: local models are capable, but they are not drop-in replacements for cloud Claude. Practitioners who've made the switch consistently report that local models are slower on large projects, make more mistakes, and can't match the depth of the largest cloud models. They shine on well-defined bugs, small features, and privacy-critical tasks — not on sprawling refactors.

There's also a subtler trap: benchmark leaderboards tell you almost nothing about whether a model will work in Claude Code. One tester watched a model score well on public leaderboards yet produce zero tool calls in the actual agentic harness. The same "test it in the real harness" philosophy is what SkillzBench is building around agent skills — sandboxed benchmarks that measure each skill's actual Lift, with and without the skill, rather than trusting paper scores.

And the hardware isn't free. A serious local agentic setup can run €4,250 in hardware, or $3,539 for a single Lenovo ThinkStation PGX box. That's why many developers land on a hybrid: route routine prompts to a local model and save the hard problems for cloud Claude. In one such setup, 7 in 10 prompts ran locally at $0 cost.

Set your expectations now, and the setup guide that follows will make far more sense.

How the Connection Works: ANTHROPIC_BASE_URL and the Native-Endpoint Path

Here's the surprise: Claude Code never actually checks what's answering. As XDA's lead technical editor puts it, it's "at its core, a client that speaks the Anthropic Messages API" and "doesn't actually verify that there's a Claude model on the other end." Any server that responds in the right format gets treated as Claude.

That insight turns connection into a configuration problem rather than a hacking exercise. The core setup uses a handful of environment variables: ANTHROPIC_BASE_URL pointed at your local inference server, ANTHROPIC_AUTH_TOKEN set to a dummy value, and ANTHROPIC_API_KEY cleared so Claude Code doesn't try cloud authentication. You then launch with the --model flag specifying your local model's ID, and Claude Code happily talks to whatever is listening.

The good news is that the translation layer has mostly disappeared. Earlier guides from late 2025 described running a translation proxy as mandatory, but the ecosystem moved fast: Ollama added Anthropic Messages API compatibility in v0.14.0, LM Studio shipped a native /v1/messages endpoint in version 0.4.1, vLLM offers an official Claude Code integration, and llama.cpp's llama-server speaks the format directly. One practitioner guide still cautions that llama.cpp's compatibility is partial — "the speed play, not the compatibility play" — so it's worth checking your backend's current documentation before committing.

• Ollama ≥ v0.14.0: native Anthropic Messages API compatibility, no proxy required
• LM Studio ≥ 0.4.1: built-in /v1/messages endpoint for Claude Code traffic
• llama.cpp: direct Anthropic support in llama-server, though some report partial compatibility
• vLLM: official Claude Code integration for serving larger deployments

When should you still reach for a proxy? Teams with real infrastructure needs — SSO, audit logging, centralized auth — benefit from escalating to LiteLLM or Claude Code Router. But as one practitioner advises, "premature complexity doesn't solve solo developer problems" — start simple, and only add the proxy layer when team features actually demand it.

One caveat worth flagging before you wire anything up: connectivity is only half the battle. Claude Code's attribution header changes on every request and can make local inference dramatically slower by invalidating the KV cache — a fixable issue, but one that bites anyone who assumes a working connection means a fast one. And once your local Claude Code is running well, it becomes a natural home for agent skills; SkillzBench is building a free directory and MCP connector so you can load vetted skills into whatever model sits behind that base URL.

The Two Silent Killers: Context Length and the 90% Slowdown Bug

Here's the frustrating part: your local Claude Code setup can look perfectly healthy while failing. Two silent killers — context length and a cache-busting bug — account for most of the pain, and neither announces itself with an error message.

The first killer is context. Claude Code's system prompt alone consumes 6–10K tokens before you've typed anything. On an 8K context, that means the model never even sees your request properly — and it breaks silently, not loudly. The practical guidance: 32K tokens is the real minimum, 64K is the sweet spot, and anything below 32K is "a chat demo, not Claude Code."

Worse, Ollama defaults to just 4096 tokens of context — far too small for Claude Code's needs. If you launch without adjusting it, you've built a setup that can't work. Here's what to target:

• 8K context: broken — the system prompt alone eats most of it
• 32K context: the practical floor for real agentic work
• 64K context: the sweet spot for most local setups
• 128K and beyond: diminishing returns, heavier memory cost

The second killer is subtler and more expensive. Claude Code sends an attribution header (x-anthropic-billing-header) with every request, and because it changes each time, it invalidates the KV cache — the mechanism that lets local inference servers skip re-processing unchanged context. The result, according to Unsloth's documentation, is local inference running 90% slower than it should.

The fix is two environment variables: CLAUDE_CODE_ATTRIBUTION_HEADER=0 and CLAUDE_CODE_ENABLE_TELEMETRY=0. This is the single highest-impact optimization for any local setup, and it costs nothing. Unsloth also recommends prompt trimming flags like --bare and --exclude-dynamic-system-prompt-sections to improve KV-cache reuse further.

Why does this matter beyond raw speed? Because a 90% slowdown turns a usable local assistant into one that feels broken, and you'll wrongly blame the model. The same lesson applies to agent skills: a skill that looks good on paper can behave very differently in the real harness — which is exactly why SkillzBench is building sandboxed benchmarking that tests every skill with and without it, publishing the measured Lift rather than trusting published claims. If you're extending a local Claude Code setup with skills, its free directory and MCP connector are being built for precisely that kind of verified fit.

Get context length right and the attribution header off, and your local setup goes from frustrating to genuinely productive.

Picking a Model That Actually Works in the Harness

Here's the uncomfortable truth about picking a local model for Claude Code: the benchmark leaderboards won't save you. One practitioner watched a model score well on public benchmarks, then produce zero tool calls inside Claude Code's agentic harness — a total failure that no leaderboard predicted. As Vladimir Ivanov put it, benchmark scores tell you almost nothing about whether a model will follow a long, detailed operating manual.

Then there's the hardware math. A 35B model at FP16 needs roughly 70GB of storage, which simply doesn't fit in 32GB of VRAM. This is where quantization enters — the art of figuring out how much precision you can throw away before the model gets noticeably dumber. A Qwen3.5-35B-A3B with a UD-Q4_K_XL quant and q8_0 KV cache fits into 23GB on an RTX 4090, per Unsloth's documentation, while smaller Qwen2.5 Coder 14B Q4 builds occupy around 9GB.

Quantization is the trade-off you can't escape, but it's manageable. What you should plan around instead:

• Cold-start times of 5–10 seconds for 7B models and 15–30 seconds for 30B models — noticeable when Claude Code restarts sessions frequently.
• Context length as the real gate: anything below 32K tokens is, in one guide's words, "a chat demo, not Claude Code."
• Backend choice matters more than you'd expect — Vulkan was 2x more performant than ROCm/HIP on one Radeon setup.
• Model fit to harness: some models trained specifically for agentic coding workflows behave far better in Claude Code than higher-benchmarked peers.

The proven middle ground is hybrid routing: send routine, low-complexity tasks to a local model and escalate hard problems to cloud Claude. One developer built exactly this and now runs 7 in 10 prompts locally at $0, reserving paid credits for the work that actually needs them.

The same "test in the real harness, not on paper scores" philosophy drives how SkillzBench is building its skill benchmarking — every agent skill gets tested in a sandbox with and without the skill, measuring genuine Lift rather than trusting published claims. Whether you're picking a model or a skill, real behavior beats leaderboard promises every time.

Your Next Step: Test Skills Where They Actually Run

You've pointed Claude Code at a local model, disabled the attribution header, and sized your context window. The next question is the one this entire article keeps circling: does your setup actually work where it runs — not on paper?

That philosophy came straight from the research. Vladimir Ivanov watched a model that scored well on leaderboards produce zero tool calls in the agentic harness — proof that benchmark scores tell you almost nothing about real behavior. The same logic applies to the agent skills layer sitting on top of your local stack.

Skills (the SKILL.md instruction packages that steer Claude Code, Codex, Cursor and other agents) are everywhere on GitHub, but nobody can tell you which ones help, which are dead weight, and which are dangerous — published research found roughly 26% of skills carry at least one vulnerability. Bolting an unvetted skill onto a local model you spent hours tuning is a strange kind of trust.

That's the gap SkillzBench is built to close. It's a free, independent directory of agent skills, plus a free MCP connector that plugs into any agent — including your local Claude Code setup. You browse, toggle skills and tools per agent, and the agent only sees what you chose.

What SkillzBench is building goes a step further, and it mirrors this article's core lesson exactly:

• Benchmark in the harness, not on paper — every skill is tested in a sandbox with the same model, with and without the skill, and the resulting Lift is published in percentage points.
• Every version is pinned to a commit SHA, security-scanned, license-checked and quality-judged, with a 0–100 trust score built from security, quality, license, maintenance and community signals.
• Lift is always shown separately from ratings, and money never touches scores — the platform is entirely free, funded by blog advertising.
• Upstream changes trigger re-scans and re-benchmarks, so a skill that drifts or breaks doesn't silently degrade your agent.

This matters more in local setups, where you're already fighting constraints the research made clear: a 32K-token context floor and models that are, as one practitioner put it, capable alternatives with different characteristics, not drop-in replacements. Every token a bloated or broken skill burns comes out of that budget.

SkillzBench is pre-launch, and the waitlist is open now. Join it, and when the benchmarks go live you'll see exactly which skills earn their keep in the harness you actually run — measured, not claimed.