In This Article
Introduction
OpenClaw is primarily text-based, but voice interfaces are increasingly important. Voice agents combine speech-to-text (STT), the LLM, and text-to-speech (TTS) to enable spoken interaction. Here's what we're covering: adding voice to OpenClaw and integration patterns.
Voice Architecture
Voice flow: User speaks → STT converts to text → OpenClaw processes → LLM responds → TTS converts to audio → User hears. This can run through a separate voice gateway (e.g., Vapi, Bland, or custom) that connects to OpenClaw's API, or through Skills that handle audio.
Two main patterns. (1) Voice platform (Vapi, Bland) handles STT/TTS, telephony, and streaming. It sends text to OpenClaw's API and receives text back. OpenClaw is the brain; the platform is the interface. (2) Custom Skill: receive audio (e.g., from Telegram voice message), call STT API, pass text to agent, get response, call TTS, return audio. More control, more integration work.
Speech-to-Text: Options & Setup
STT options: Whisper (OpenAI, local or API), Google Speech-to-Text, AssemblyAI, Deepgram. Quality and latency vary. For real-time conversation, low-latency providers (Deepgram, AssemblyAI) matter. Store transcripts in OpenClaw memory for context.
Step-by-step: Adding STT. Choose provider. For OpenAI Whisper API: send audio file, get text. For Deepgram: real-time streaming or batch. Create a Skill that: (1) Receives audio (from webhook, Telegram, etc.), (2) Calls STT API, (3) Passes text to OpenClaw, (4) Returns response. Latency budget: aim for under 500ms STT + 1s LLM + 500ms TTS for natural conversation.
Text-to-Speech: Options & Setup
TTS options: ElevenLabs, Play.ht, OpenAI TTS, Google TTS. Naturalness varies. ElevenLabs and Play.ht offer voice cloning for brand consistency. Stream TTS for lower perceived latency — start playing before full response is generated.
Costs. OpenAI TTS: ~$15/1M chars. ElevenLabs: tiered; higher quality costs more. Google TTS: $4/1M chars. For high volume, compare per-minute costs.
Integration Patterns
Pattern 1: Voice platform (Vapi, Bland) handles STT/TTS and sends text to OpenClaw. OpenClaw is the brain; voice is the interface. Easiest for phone/IVR. Pattern 2: Custom Skill that receives audio, calls STT, passes to agent, gets response, calls TTS. More control, more work. Pattern 3: Telegram/WhatsApp voice messages — OpenClaw can process voice notes via platform APIs and STT. Good for async voice.
Implementation Checklist
- □ Choose pattern: voice platform vs custom Skill
- □ Select STT provider (Whisper, Deepgram, etc.)
- □ Select TTS provider (ElevenLabs, OpenAI, etc.)
- □ Build or integrate voice gateway
- □ Test latency; optimize for real-time
- □ Store transcripts in memory for context
Cost Breakdown for Voice
STT: Whisper API ~$0.006/min. Deepgram ~$0.004/min. TTS: OpenAI ~$15/1M chars (~$0.02/min speech). ElevenLabs varies. For 1000 min/month: ~$30-80 in voice APIs. Add LLM costs. Voice platforms (Vapi) have their own pricing.
Common Pitfalls to Avoid
Pitfall 1: High latency. Users tolerate 1-2s total. Optimize STT (streaming), use faster LLM for voice. Pitfall 2: Wrong language. Ensure STT/TTS support your target languages. Pitfall 3: No fallback. When STT fails (noise, accent), have "I didn't catch that" handling.
Frequently Asked Questions
Can OpenClaw handle phone calls? Via Vapi, Bland, or similar. They handle telephony; OpenClaw handles conversation. What about WhatsApp voice? Process voice notes with STT; respond with text or TTS-to-audio. Local STT/TTS? Whisper runs locally; Coqui TTS for local TTS. No API cost, but need GPU.
Wrapping Up
Voice extends OpenClaw to hands-free and accessibility use cases. OpenClaw Consult helps design and implement voice agent setups.