How to make a free AI calling agent?

The Hidden Cost Problem: Why Commercial Voice Agent Platforms Drain Budgets

A voice agent that costs pennies per call in a demo can quietly become a four-figure monthly line item once it hits production. That gap between the trial and the invoice is where most budgets go to die.

The market is certainly moving in the right direction. According to Grand View Research, AI voice agents are projected to grow from USD 2.5 billion in 2025 to USD 35.2 billion by 2033, a 39.0% CAGR — and the same report notes that demand for affordable, enterprise-grade solutions among small and mid-sized businesses is rising, with buyers preferring deployments measured in days rather than months. Falling model prices reinforce that optimism: a16z's market update notes OpenAI cut GPT-4o realtime pricing in December 2024 by 60% on input and 87.5% on output tokens.

So why do the bills still sting? Because the platform, not the model, is the meter that runs fastest.

One production team building on Vapi reported that 60–70% of total spend was the platform fee itself, with only 30–40% going to actual LLM, STT, and TTS usage, according to a first-hand account from the Dograh team. That inversion is the hidden cost problem in a sentence: you are paying mostly for orchestration and convenience, not intelligence.

The pain compounds across the typical commercial stack:

• Platform fees scale with call volume, so growth makes the margin problem worse, not better.
• Customization hits a ceiling — on hosted platforms you stop where the slider, field, or schema stops, as LiveKit's platform comparison puts it.
• Data control and self-hosting options are limited once privacy or offline deployment enters the picture.
• Iteration is unavoidable anyway — no voice agent works perfectly out of the box, and every build needs tuning in month two.

That combination — high recurring fees plus mandatory ongoing tuning — is precisely why teams start looking at self-hosted, open-source alternatives. The economics of a free build are no longer theoretical, and the tooling to assemble one has matured.

For builders who go that route, the natural next question is what the agent can actually do once it answers. Agent skills extend that reach, and SkillzBench's free skills directory and free MCP connector — currently in development, with the waitlist open — are being built so anyone can discover, verify, and equip skills for their own agent.

Skills, proven. Join the waitlist to be first in line when the free directory, MCP connector, and benchmarks go live.

The Free Blueprint: A Four-Stage Voice Pipeline You Can Assemble for $0

Every "free" voice agent platform quietly charges you twice: once for the AI and once for the privilege of using their dashboard. One production team found that 60–70% of their total spend on a commercial platform was the platform fee itself, with only 30–40% covering actual model usage. The fix is to assemble the pipeline yourself.

The proven blueprint is a four-stage chain: Voice Activity Detection (VAD) → speech-to-text (ASR) → LLM → text-to-speech (TTS). According to a detailed hands-on guide, you stream audio through every stage — the VAD server processes audio in 100ms chunks — because latency is the enemy of natural conversation.

Why orchestrate instead of using an end-to-end voice model? End-to-end models offer lower latency but can't take custom prompts or knowledge per request, and they can't clone your brand's voice. A chained pipeline is fully customizable at every step: swap the LLM, inject your knowledge base, tune the voice.

Here's how each stage maps to free components:

• VAD: a lightweight server that detects when the caller speaks, processing audio in 100ms chunks to keep turn-taking natural.
• ASR: Whisper-large-v3 via a free Groq API key — fast, accurate transcription at zero cost to start.
• LLM: gpt-oss-20b through Groq's free tier for the reasoning layer, or a fully local model like gemma-3-4b-it.
• TTS: ElevenLabs' free tier for natural speech, or Piper for a completely local, zero-cost stack.

If you want zero ongoing cost entirely, LlamaEdge runs the whole thing locally — LLM, Whisper ASR, and Piper TTS — with OpenAI-compatible endpoints, so your agent never depends on an external API bill.

The LLM doesn't just talk, either. It emits JSON tool-call structures your server intercepts and executes, letting the agent send messages, make payments, or control devices. That's where agent skills come in: packages of instructions that extend what your agent can do. SkillzBench is building a free directory and MCP connector to help you discover, verify, and equip those skills — worth watching as your agent matures.

One tradeoff to plan for: bigger models follow instructions better but add latency, while smaller local models respond faster but need more prompt tuning. Budget time for iteration — as one practitioner puts it, every agent inevitably needs tweaking in month two.

Choosing Your Open-Source Framework: Match It to Your Team and Use Case

There is no single "best" framework for a free AI calling agent — there is only the best fit for your team, your use case, and how much you expect the agent to change after launch. The three main open-source options differ sharply in who they serve.

LiveKit Agents is open-source and code-first. It suits teams with engineers who will iterate after launch, because as the LiveKit comparison puts it, customization becomes "a code change to the agent itself in your repo" rather than a slider that stops where the UI stops. Real-world voice agents need that iteration: callers pause mid-sentence and get cut off, tool calls leave three seconds of dead air at p95, voicemails receive full sales pitches, and transfers to humans drop context. No agent ships perfect — LiveKit's own guidance notes that every agent will "inevitably need to be tweaked in month two."

Dograh takes a different path. It is a Pipecat-based, self-hostable alternative to Vapi with a drag-and-drop builder and telephony integrations for Twilio, Vonage, Vobiz, and Cloudonix. Its creators built it after discovering that 60–70% of their total Vapi spend was the platform fee itself, with only 30–40% going to actual LLM, STT, and TTS usage. Their motivation was also control: hosted platforms work for demos, "but once you need data controls, privacy or self/offline deployment, you end up stuck." The honest caveat from the same team: even with solid open-source foundations, they still had to do significant plumbing — variable extraction, tracing, and testing.

EchoKit is a Rust-based orchestration server designed around the standard four-stage pipeline — VAD, ASR, LLM, TTS — and pairs naturally with free-tier APIs like Groq and ElevenLabs, or a fully local LlamaEdge stack.

LiveKit proposes four questions worth asking honestly before choosing:

• Will engineers iterate on the agent after launch?
• What happens when requirements fall outside the platform's defaults?
• Will the agent need channels beyond phone calls?
• Is cost at scale a concern?

There is also a latency-versus-reasoning tradeoff to weigh: bigger models follow instructions better but add latency, while smaller colocated open-source models respond faster but need more prompt tuning. And be candid about the counter-case — as LiveKit itself concedes, for basic scheduling, FAQ, and lead-qualification agents, a hosted platform's defaults are often the right trade. Open source buys control and cost efficiency, but it assumes someone on your team can maintain the plumbing.

Once your framework is chosen and the pipeline is running, agent skills extend what the agent can do — SkillzBench's free skills directory and MCP connector, currently in development with a waitlist open, are being built to help you discover, verify, and equip skills for any agent.

Step-by-Step Build: From Free API Keys to a Working Calling Agent

The good news: you can build a working voice agent today without paying a platform fee, because the entire pipeline is open source and the API keys are free. The core architecture is a four-stage chain — voice activity detection (VAD), speech-to-text, an LLM, and text-to-speech — orchestrated so each stage streams into the next to keep latency down, as detailed in a hands-on freeCodeCamp tutorial.

Start with the free credentials. Groq provides free-tier API keys for Whisper-large-v3 (speech recognition) and gpt-oss-20b (the LLM), while ElevenLabs supplies the text-to-speech voice. If you want zero ongoing cost entirely, the tutorial's LlamaEdge option runs an LLM, Whisper ASR, and Piper TTS locally through OpenAI-compatible endpoints.

Then stand up the three moving parts:

• The VAD server processes incoming audio in 100ms chunks on port 9094, deciding when a caller has actually started or stopped speaking.
• The orchestration server (EchoKit, on port 8000) coordinates the pipeline and keeps a rolling context of 20 messages so the agent remembers the conversation.
• The LLM endpoint handles reasoning — either Groq's free tier in the cloud or a local model like gemma-3-4b-it-Q5_K_M via LlamaEdge on port 8080.

The step that turns a chatbot into an agent is MCP. The LLM emits JSON tool-call structures that the server intercepts and executes, which is how the agent can send messages, trigger payments, or switch devices on and off — as tutorial author Michael Yuan puts it, an AI agent "is not just about chatting. It is about performing actions on specific tasks." If you want to extend that tool layer further, SkillzBench is building a free directory of verified agent skills plus a free MCP connector designed to equip any agent with the skills you choose — worth watching while it's in development.

For actual phone calls, Dograh (a self-hostable, Pipecat-based alternative to commercial platforms) offers a drag-and-drop builder with telephony integrations for Twilio, Vonage, Vobiz, and Cloudonix. Here's the honest gap: PSTN setup and pricing are under-documented across the open-source ecosystem. Dograh's creators list the providers but publish no setup walkthrough or cost breakdown, and no research source covers end-to-end telephony costs for a free build. That's also where the economics argument gets interesting — the Dograh team reported that 60–70% of their spend on a commercial platform was the platform fee itself, with only 30–40% going to actual model usage.

Finally, expect iteration. Production voice agents commonly suffer mid-sentence cutoffs, roughly 3-second tool-call dead air at p95, and full sales pitches left on voicemails. There's no perfect out-of-the-box agent — every build needs tuning in month two, so budget testing time accordingly.

Ship It Safely: Real-World Failure Modes, Security, and Going Further with Agent Skills

Your agent will pass its first demo call and fail its tenth real one. That's normal — as the LiveKit team puts it, there's no such thing as the perfect AI voice agent out of the box, and every agent will need tweaking by month two.

Before you ship, stress-test the failure modes that only appear with real callers:

• Mid-sentence cutoffs — callers who pause to think can trigger the agent to jump in; tune your VAD thresholds and end-of-turn detection.
• Tool-call dead air — tool calls took roughly 3 seconds at p95 in one production comparison, leaving awkward silence; stream filler phrases or pre-fetch data to cover the gap.
• Voicemail disasters — without answering machine detection, your agent will happily deliver a full pitch to a voicemail greeting.
• Context drops on transfer — when escalating to a human, pass the full conversation summary or the caller starts over from scratch.

Security deserves equal attention. OWASP's free, open-source GenAI guidance — backed by a community of over 30,000 members — flags Memory & Context Poisoning as a core agentic risk: "Memory is a Feature. It Is Also an Attack Surface." A caller who injects malicious instructions into your agent's memory can corrupt every future conversation, so treat stored context as untrusted input and validate it like anything else. The OWASP GenAI security resources cost nothing and are the best starting guardrails available.

Once your agent handles calls reliably, the natural next step is giving it more skills — booking, CRM updates, payment follow-ups. Agent skills (SKILL.md instruction packages) are everywhere, but quality varies wildly: published research found roughly 26% of skills carry at least one vulnerability. That's the problem SkillzBench is being built to solve — a free, independent directory that crawls and dedupes skills from GitHub, pins every version, security-scans and benchmarks each one, and publishes a trust score with the performance Lift shown separately. Its free MCP connector (in development, waitlist open) will let you toggle exactly which skills and tools any agent can see.

Voice agents are moving fast — the market is projected to grow from USD 2.5 billion in 2025 to USD 35.2 billion by 2033 — and the teams winning aren't the ones with perfect first builds. They're the ones who test the ugly edge cases, secure the context, and keep equipping their agents with better, verified skills.