gnome-jarvis/docs/voice-pipeline.md

Voice pipeline

Voice is input/output around the harness. The daemon owns capture, wake-word processing, VAD, transcription, completion, TTS playback, barge-in policy, and metrics.

flowchart LR
  MIC[PipeWire mic\n16 kHz mono] --> R[Wake ring buffer]
  R --> W[Wake engine]
  W -->|phrase| V[VAD + QVAC ASR]
  V -->|final utterance| H[Harness bridge]
  H -->|streamed reply| T[Sentence buffer]
  T --> Q[QVAC TTS]
  Q --> OUT[PipeWire playback]
  OUT -. anti-feedback gate .-> V

State machine

stateDiagram-v2
  ARMED --> LISTENING: wake phrase / hotkey
  LISTENING --> THINKING: VAD final utterance
  THINKING --> SPEAKING: first reply audio
  THINKING --> LISTENING: tool work / no audio yet
  SPEAKING --> LISTENING: playback + cooldown
  LISTENING --> ARMED: silence timeout
  SPEAKING --> ARMED: cancel / Escape
  ARMED --> SLEEPING: idle timeout
  SLEEPING --> LISTENING: wake phrase

The wake stage is intentionally separate from QVAC ASR. Configure a local openWakeWord or sherpa-onnx bridge through JARVIS_WAKE_COMMAND; no cloud wake service is supported. During TTS, capture submission is gated and a 300–500 ms cooldown prevents playback from becoming a new utterance. Trash transcripts and very short fragments are discarded.

Diagnostics and acceptance

Run npm run voice-doctor, then follow voice acceptance. The daemon metrics cover wake accepts/rejects, feedback drops, utterances, and replies. Raw audio is not persisted by default.