The Pipeline: Voice to Action
When you speak to Slime, something complex happens in milliseconds. Your voice travels through multiple systems, each doing specific work, until the right action happens. Understanding this pipeline helps you understand what’s actually possible with voice-first AI.
The pipeline has four main stages: transcription, intent parsing, context retrieval, and action execution.
Stage 1: Transcription
The moment you speak, Slime is converting your voice into text. This happens on your device, locally, which means your voice doesn’t get sent to a server just to transcribe it.
On-device transcription has real advantages. It’s fast. It’s private. And it reduces latency—you get a response quicker because you’re not waiting for a round trip to a server.
But on-device transcription is also limited. The model has to be small enough to run on a phone. So Slime uses a two-stage approach: fast, accurate-enough transcription on-device, and then optional server-side refinement for complex inputs.
The result: most commands get a response in under 500ms, with minimal privacy overhead.
Stage 2: Intent Parsing
Raw text isn’t enough. “Block two hours Thursday for the project” contains implicit intent: create a calendar event, with a duration, on a specific day, for a specific project.
Intent parsing is where Claude comes in. Slime uses Anthropic’s Claude as its AI backbone, because Claude is exceptional at understanding natural language and extracting meaning.
Claude looks at your input and determines: What’s the user asking for? What system should handle this? What parameters are specified? What’s ambiguous and needs clarification?
This is where personalization starts to matter. Claude isn’t just parsing the literal request. It’s parsing the request in context of who you are, how you work, and what you care about.
Stage 3: Context Retrieval
Before Slime acts, it needs context. If you’re asking to schedule a meeting, Slime needs to know: What time do you prefer meetings? Do you have conflicts that day? What’s your travel schedule? Who are you meeting with?
Slime’s memory system stores this context. It isn’t just a database of facts. It’s a semantic knowledge base, where information is stored and retrieved based on meaning, not just keyword matching.
In practice, this means Slime retrieves the right context automatically. You don’t have to specify every parameter. Slime fills in the blanks based on what it knows about you.
This context retrieval is powered by embeddings and vector search. Slime converts your memories into semantic representations, so it can find the most relevant information for the current task.
Stage 4: Action Execution
Once Slime understands the intent and has retrieved the context, it executes the action. This is where Slime differs fundamentally from chatbot assistants.
Slime doesn’t just tell you what to do. It does it. It integrates with your calendar, your email, your task management system, your other tools. When you ask Slime to block time, the calendar event is created. When you ask Slime to draft an email, the email is drafted and ready to send.
Behind the scenes, Slime is using Cloudflare Durable Objects to maintain state. This matters because execution isn’t always instantaneous. Sometimes the action has multiple steps. Sometimes it needs to wait for confirmation. Durable Objects let Slime maintain stateful execution across multiple requests.
The result: actions that would require you to open three different apps and manually coordinate between them happen in one voice command.
The Memory Backend
Persistent memory is the hardest part of the voice AI stack. It isn’t enough to remember facts. You need to understand relationships, update memories over time, and retrieve the right context when needed.
Slime uses a hybrid approach. Structured data (your calendar, your contacts, your goals) is stored in a relational database where it can be queried precisely. Unstructured information (your preferences, your communication style, your ideas) is stored as embeddings in a vector database, so Slime can find relevant information by semantic similarity.
Every conversation updates your memory. Not just explicit facts you state, but patterns Slime learns from how you interact. Over time, Slime becomes a more accurate model of who you are.
Privacy at the Architecture Level
A privacy-respecting voice AI system needs more than promises. It needs architecture that enforces privacy.
Slime processes as much as possible on-device. Transcription happens locally. Intent parsing only sends what’s necessary to Claude. Context retrieval queries your personal database, not a shared one.
Your memories are encrypted and never mixed with other users’ data. The architecture treats your information as fundamentally private and only sends it to services when absolutely necessary.
The Developer Stack
If you’re building a voice AI system, here’s what you need:
Voice input: On-device transcription for privacy and latency. Fallback to cloud transcription for accuracy if needed.
Intent understanding: A capable language model like Claude that can parse natural language and extract meaning.
State management: Durable Objects or similar for maintaining execution state across requests.
Memory system: Hybrid relational + vector database for both structured and unstructured information.
Integrations: APIs to the tools your users actually use—calendar, email, task management.
Agentic execution: The ability to take actions on the user’s behalf, not just answer questions.
Where This Is Heading
The voice AI stack is still new, but it’s getting exponentially better. Voice input is getting more accurate. Language models are getting faster and more reliable. Memory systems are becoming more sophisticated. Integration with third-party tools is becoming seamless.
The next generation of AI assistants won’t be chatbots that you type to. They’ll be voice-first agents that understand you, remember you, and act on your behalf.
Slime is built on this foundation. Every piece of the architecture is designed for a single goal: turn natural voice input into meaningful action, backed by memory that actually works.