Architectures by volume
There is no single best voice architecture; there is a best architecture per call volume, and the honest engineering is knowing where the boundaries sit. This page puts numbers on every option, marks the crossover points where the answer changes, and prices what each shift buys and costs.
The four architectures, priced
Same job, four ways to build it. Marginal cost per talking minute, with the qualities each one keeps or gives up (the mechanics of each are on the architecture page):
| Architecture | ≈ $/min* | Keeps | Gives up |
|---|---|---|---|
| Cached cascade (production today) | 0.025 list · ~0.017 warm | Streaming ~1s first audio, house voice, verbatim scripts, tool-reliable text LLM, full logs | Turn-taking runs on silence timers, engineered down but not eliminated |
| Director (fuse hearing+judgment, hosted) | ~0.015–0.020 target | Everything the cascade keeps, plus audio-native turn-taking; brain wakes for ~20% of turns | Duplex-model maturity risk; the delegation-leak eval gates launch |
| Fused speech-to-speech (demo mode) | ~0.15–0.30 | The most natural turn-taking available today, zero orchestration code | House voice, verbatim delivery, cost tiering, and it costs 5–10× the price |
| Self-hosted open stack | ~0.003–0.006 at utilization | Lowest unit cost, data residency, no per-token bills | Ops burden, capacity planning, model quality drift, $600+/GPU/month whether calls come or not |
*Infrastructure only. Telephony adds ~$0.009/min to every option equally, and is the true floor no architecture removes. A managed voice-agent platform (ConversationRelay, Voice Agent API) layers $0.059–0.075/min on top of these, roughly 2.5× the parts cost for the orchestration this codebase implements itself.
Where the answer flips
Both crossovers are divisions, not opinions. A mid-range cloud GPU runs ≈$600/month whether or not calls arrive; the cached cascade's replaceable spend is ≈$0.0075/min. So the first GPU pays for itself at 600 ÷ 0.0075 = 80,000 min/month (≈16,000 five-minute calls). A production-grade fleet with redundancy, failover, and an ops rotation carries ≈$31,250/month of fixed cost, which the same division puts at ≈4.17M min/month (≈833,000 calls). Below a crossover, the "cheap" option is the expensive one.
Band by band: what to run, and why
| Volume band | Cheapest stack | The reasoning |
|---|---|---|
| Demo / portfolio < 2K min/mo | Cached cascade, hosted | Fixed costs dominate: an always-warm instance ($10–25) plus the number ($1.15) dwarf usage. Elle's own bill lives here. Self-hosting would multiply cost ~10×; the fused demo mode is worth its premium only as a showpiece, and on a spot GPU a demo costs $2–3/month. |
| Side business 2K – 80K min/mo | Cached cascade, hosted | Still strictly cheapest: usage grows but never reaches the first GPU's fixed cost. Adopt the director here for experience, not for savings: turn-taking quality is the argument, hosted economics are unchanged. |
| First crossover 80K – 500K min/mo | Hybrid | Self-host the highest-spend speech component (STT or TTS) on one GPU; keep the LLM on hosted APIs where caching already did the work. One box, no fleet ops, and the division above says it pays from day one of the band. |
| Scale-out 500K – 4.2M min/mo | Mixed fleet + director | Utilization engineering starts mattering more than unit prices. The director's escalation tiering is the big lever now: a cheap fused talker absorbs ~80% of turns, so the metered brain bills on a fifth of the traffic. |
| Call-center > 4.2M min/mo | Self-hosted fleet, full stack | Past the second crossover a fleet wins outright, and per-token pricing becomes the thing you left behind. This is the regime the worked example below lives in. |
The worked example: 1.25M calls a month
The modeled deployment: 1.25M calls/month at 5 minutes average: 6.25M talking minutes, comfortably past the second crossover.
| Line | $/month | Notes |
|---|---|---|
| Human-agent equivalent | ≈ $5.6M | The baseline the whole exercise displaces. |
| Cascade at list prices | ≈ $190K | Naive hosted build, no optimization. |
| Optimized (caching + batching + tiering + fleet) | ≈ $85K | Prompt caching and turn batching cut the LLM line ~85%; escalation tiering and self-hosted speech do the rest. |
The sensitivity that outranks every architecture choice: containment. Every call the AI resolves end-to-end costs ~$0.07 optimized; every call it escalates costs a human ~$4.50. Moving containment a single point at this volume swings ≈$55K/month, more than most infrastructure line items combined. Buy containment (better prompts, better tools, better escalation judgment) before buying cheaper GPUs.
The model, in full
Every number above traces to an editable workbook (download the model: assumptions, LLM token calculator, scenario comparison, and the crossover sheet, with inputs marked for replacement by real quotes and production measurements). These are its assumptions:
Demand and the human baseline
| Input | Value | Note |
|---|---|---|
| Calls per month | 1,250,000 | The modeled call-center demand. |
| Average handle time | 5.0 min | 6.25M conversation minutes per month. |
| Loaded human cost | $0.90 / agent-minute | Wages, benefits, telecom, supervision; replace with your BPO rate. |
| Human cost per call | $4.50 | The number every AI scenario competes against. |
| All-human baseline | ≈ $5.63M / month | 1.25M calls × $4.50. |
Unit prices per minute, by scenario
| Component | List | Negotiated | Hybrid self-host |
|---|---|---|---|
| Telephony | $0.0090 | $0.0045 | $0.0045 |
| Speech-to-text | $0.0077 | $0.0040 | $0.0015 |
| Text-to-speech | $0.0056 | $0.0030 | $0.0010 |
| LLM | from the token calculator below | $0.0030 | |
| Fixed monthly | $2,000 | $2,000 | $25,000 (GPU fleet + ops staffing) |
The LLM line, from token math
The LLM cost is not a guess; it is computed from the conversation shape the
[llm] log lines measure in production:
| Input | Value |
|---|---|
| Turns per call | 13 |
| Cached prefix (persona + knowledge + history) | 7,000 tokens |
| New input per turn (utterance + tool results) | 150 tokens |
| Output per turn (120-token cap in code) | 110 tokens |
| Cache hit rate across turns | 90% |
| Haiku pricing (per MTok) | $1 in · $5 out · $0.10 cache read · $1.25 cache write |
At those inputs the prefix bills at the cache-blended rate on every turn instead of full price, which is the arithmetic behind "caching cuts the LLM line ~85%." Set the hit rate to zero in the workbook to see the uncached cost; the difference is the whole caching argument in one cell.
Containment: the assumption that owns the outcome
| Input | Value | Meaning |
|---|---|---|
| Containment rate | 60% | Calls the AI resolves end to end. Deliberately conservative; mature deployments report higher. |
| AI minutes before transfer | 1.5 | What a non-contained call costs in AI time before handing off. |
| Human minutes after transfer | 4.0 | $3.60 of human time per escalated call, and it dominates that call's cost. |
Run the sensitivity and the structure of the business is visible: a contained call costs pennies, an escalated call costs $3.60 of human time plus its AI minutes, so each point of containment at this volume moves ≈$55K/month. No unit-price negotiation compares. That is why the engineering pages spend so much effort on grounding, tools, and escalation judgment: in this model they are the financial levers.
What each shift buys and costs
| Shift | You gain | You pay | Take it when… |
|---|---|---|---|
| Cascade → + caching & batching | ~85% off the LLM line, faster prefill | Nothing: it's code, already shipped | Always. There is no counter-case. |
| Hosted → managed platform | Orchestration you don't write | ~2.5× per minute, less control at every layer | You lack the engineering, and volume is small enough that 2.5× is cheaper than a hire. |
| Cascade → director | Audio-native turn-taking; brain bills on ~20% of turns | Duplex-model maturity risk; delegation-leak eval must pass first | Experience quality is a differentiator, or volume makes the tiering material. |
| Hosted speech → self-hosted speech | Unit cost ÷ 3–5 above the crossover | Ops, capacity planning, quality ownership | Sustained volume > 80K min/month, never below it. |
| Anything → fused speech-to-speech | Best-in-class naturalness, zero pipeline code | 5–10× price, paraphrased compliance, voice lock-in | Demos and prototypes; revisit as fused pricing falls. |
The principle underneath
Architecture follows volume, not fashion. The system is built so these shifts are configuration, not rewrites: the provider seam makes a new stack a third factory behind the same interface, the runtime switch flips it per call, and the token telemetry tells you, in production numbers, when a crossover is actually approaching rather than theoretically near. Decide from the logs, not the roadmap.