45 lines
1.5 KiB
Markdown
45 lines
1.5 KiB
Markdown
# Voice pipeline
|
||
|
||
Voice is input/output around the harness. The daemon owns capture, wake-word
|
||
processing, VAD, transcription, completion, TTS playback, barge-in policy, and
|
||
metrics.
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
MIC[PipeWire mic\n16 kHz mono] --> R[Wake ring buffer]
|
||
R --> W[Wake engine]
|
||
W -->|phrase| V[VAD + QVAC ASR]
|
||
V -->|final utterance| H[Harness bridge]
|
||
H -->|streamed reply| T[Sentence buffer]
|
||
T --> Q[QVAC TTS]
|
||
Q --> OUT[PipeWire playback]
|
||
OUT -. anti-feedback gate .-> V
|
||
```
|
||
|
||
## State machine
|
||
|
||
```mermaid
|
||
stateDiagram-v2
|
||
ARMED --> LISTENING: wake phrase / hotkey
|
||
LISTENING --> THINKING: VAD final utterance
|
||
THINKING --> SPEAKING: first reply audio
|
||
THINKING --> LISTENING: tool work / no audio yet
|
||
SPEAKING --> LISTENING: playback + cooldown
|
||
LISTENING --> ARMED: silence timeout
|
||
SPEAKING --> ARMED: cancel / Escape
|
||
ARMED --> SLEEPING: idle timeout
|
||
SLEEPING --> LISTENING: wake phrase
|
||
```
|
||
|
||
The wake stage is intentionally separate from QVAC ASR. Configure a local
|
||
openWakeWord or sherpa-onnx bridge through `JARVIS_WAKE_COMMAND`; no cloud wake
|
||
service is supported. During TTS, capture submission is gated and a 300–500 ms
|
||
cooldown prevents playback from becoming a new utterance. Trash transcripts and
|
||
very short fragments are discarded.
|
||
|
||
## Diagnostics and acceptance
|
||
|
||
Run `npm run voice-doctor`, then follow [voice acceptance](voice-acceptance.md).
|
||
The daemon metrics cover wake accepts/rejects, feedback drops, utterances, and
|
||
replies. Raw audio is not persisted by default.
|