Elle, documented end to end
Elle is an AI assistant that answers a real phone number, texts, and chats on the web on behalf of one person, Lewis W. Campbell. She represents his work, answers questions grounded in his actual resume, and books intro calls onto his calendar. This site documents how every part of her is built, down to the file and the design decision.
Three things make Elle worth documenting rather than just demoing:
- She is a real-time voice agent built from parts, not a platform. Streaming speech-to-text, a text LLM, and streaming text-to-speech are orchestrated by hand: sentence streaming, barge-in, echo suppression, a 20 ms audio pacer, and prompt caching are all code in this repository, not vendor checkboxes.
- Every channel shares one brain. Phone, SMS, and web chat run the same persona, the same knowledge grounding, the same booking tools, and the same memory, so a conversation can start on a call and continue by text.
- The architecture is swappable at runtime. Two complete voice architectures live behind one interface, and the active one can be switched per-call without a deploy. The same seam is where future architectures (a fused duplex talker with inference escalation) plug in.
Why an interaction model is a requirement, not a refinement
The second layer of the model below, interaction, is the one this project treats as load-bearing. That is not a private opinion. Over 2026 three independent labs published the same conclusion from three different directions: turn-taking is not a detail to be tuned around the edges of a language model, it is its own problem, and a conversational agent that does not model it explicitly will fail in real conversation no matter how good its reasoning is.
- Thinking Machines argues that interactivity belongs inside the architecture rather than in an external harness, and calls the prevailing turn-based stack a stopgap: a voice activity detector decides when you have finished speaking without understanding a word you said. Their framing is that today's interfaces make humans "contort themselves" to the machine, batching thoughts and phrasing questions like emails, because the machine can only accept one finished turn at a time.1
- OpenAI reached the same place structurally with GPT-Live: the turn detector is removed from the audio path entirely, the conversation runs full duplex, and deeper reasoning, search, and tool use are escalated asynchronously to frontier models rather than blocking the voice loop.2
- ElevenLabs states the failure mode in production terms: most voice AI breaks in real conversation because it cannot handle interruption, silence, or context that carries across turns. Notably, the architecture they describe building toward true interaction models is an advanced cascade, in-house speech-to-text and text-to-speech with a stack tuned for low latency, which is exactly the shape Elle is.3
- Sean Goedecke, writing independently as an engineer rather than a vendor, reads the same release and reaches the same structural conclusion.4
The distinction that matters for these docs is between having an interaction layer and fusing it into a model. Elle has one, and it is written by hand: barge-in gating with a minimum-character threshold and a grace window, echo suppression, a 20 ms drift-corrected audio pacer, turn-sequence cancellation, sentence-level streaming, and endpointing tuned on real calls. Those are not incidental features; they are layer two, implemented as code because it is not yet fused into a model. Architecture explains where that leaves Elle on the fusion spectrum and why fusing layers 1 and 2, while deliberately keeping 3 and 4 separate, is the roadmap.
- 1Thinking Machines Lab, "Interaction Models: A Scalable Approach to Human-AI Collaboration." thinkingmachines.ai/blog/interaction-models
- 2OpenAI, "Continuous voice interaction with GPT-Live." openai.com/index/continuous-voice-interaction-with-gpt-live
- 3ElevenLabs, "Interaction models: Building natural human-AI dialogue." elevenlabs.io/blog/interaction-models
- 4Sean Goedecke, "Thinking Machines and interaction models." seangoedecke.com/interaction-models
The system at a glance
One Cloud Run service, one brain, many mouths. Every box is a documented module; the code reference maps all of them.
What she can do
Hold a phone conversation
Sub-second first audio, interruptible mid-sentence, an uninterruptible greeting, and echo suppression tuned on real calls. The voice pipeline →
Switch languages mid-call
English and Spanish with automatic detection: the caller just starts speaking Spanish and Elle follows, voice and all. Bilingual mode →
Book real meetings
Tentative holds on a real Google Calendar, confirmed by the owner, with a carrier-compliant SMS consent flow. Booking and compliance →
Pair a browser with a live call
A pairing code spoken on the phone links a browser: live transcripts both ways, and typed messages that Elle answers out loud. Web, SMS and intake →
Run cheaply at any volume
Three layers of prompt caching plus turn batching; documented crossover math for when self-hosting wins. Cost engineering →
Defend herself
Signature-validated webhooks, HMAC-guarded audio sockets, rate limits, and a repeat-caller abuse policy. Security →
How to read these docs
Architecture explains the shape of the system and the theory behind it, including the four-layer model of voice agents and why Elle sits where she does on the fusion spectrum. The pipeline pages (voice, languages, channels, booking) go component by component with real code excerpts. The foundation pages (data, security, cost, operations) cover what keeps it correct, safe, and affordable. Code reference is the file-by-file atlas: every source file, what it owns, and how it connects. And Diagrams collects every sequence diagram and the ERD in one place; each also appears in context on its home page. For lookup rather than reading, the feature index lists every capability in one line each, the glossary defines every term and variable, and the database documents the schema column by column.