# Voice pipeline

Voice is input/output around the harness. The daemon owns capture, wake-word
processing, VAD, transcription, completion, TTS playback, barge-in policy, and
metrics.

```mermaid
flowchart LR
  MIC[PipeWire mic\n16 kHz mono] --> R[Wake ring buffer]
  R --> W[Wake engine]
  W -->|phrase| V[VAD + QVAC ASR]
  V -->|final utterance| H[Harness bridge]
  H -->|streamed reply| T[Sentence buffer]
  T --> Q[QVAC TTS]
  Q --> OUT[PipeWire playback]
  OUT -. anti-feedback gate .-> V
```

## State machine

```mermaid
stateDiagram-v2
  ARMED --> LISTENING: wake phrase / hotkey
  LISTENING --> THINKING: VAD final utterance
  THINKING --> SPEAKING: first reply audio
  THINKING --> LISTENING: tool work / no audio yet
  SPEAKING --> LISTENING: playback + cooldown
  LISTENING --> ARMED: silence timeout
  SPEAKING --> ARMED: cancel / Escape
  ARMED --> SLEEPING: idle timeout
  SLEEPING --> LISTENING: wake phrase
```

The wake stage is intentionally separate from QVAC ASR. A built-in CPU detector
listens for “hey jarvis” (and aliases) without loading Whisper. You can replace
it with a local openWakeWord or sherpa-onnx bridge through `wakeCommand` /
`JARVIS_WAKE_COMMAND`; no cloud wake service is supported. Microphone audio is
still sent to the wake detector while Jarvis is sleeping so the phrase can
restore listening. During TTS, capture submission is gated and a 300–500 ms
cooldown prevents playback from becoming a new utterance. Trash transcripts and
very short fragments are discarded.

## Diagnostics and acceptance

Run `npm run voice-doctor`, then follow [voice acceptance](voice-acceptance.md).
The daemon metrics cover wake accepts/rejects, feedback drops, utterances, and
replies. Raw audio is not persisted by default.

## Release voice readiness

ASR, TTS, and the chat model stay unloaded until wake, Hold Talk, or a typed
question. Idle sleep unloads them again so GPU memory is freed; only PipeWire
capture and the CPU wake detector stay resident. Runtime status includes
separate `asr`, `tts`, `capture`, and `wake` flags plus per-component errors.
The tray shows **WAKE ON** when the detector is alive. Hold Talk (mouse, Space,
or Enter) still works in the GNOME overlay.

The Control Center's wake command and spoken-reply setting are read from
`~/.config/jarvis/config.json` on daemon startup. Restart `jarvisd.service` after
changing them. `JARVIS_WAKE_COMMAND` overrides the saved command. A custom wake
command must consume 16 kHz mono signed PCM and emit detected phrases on stdout.
The default `jarvis-wake-bridge` value selects the built-in CPU detector.

QVAC TTS numeric arrays contain signed 16-bit samples, not bytes. Supertonic
plays at 44.1 kHz. Captured utterances use the non-streaming transcription API
because local VAD already supplies utterance boundaries.

CI installs the Vulkan runtime required by the native addons, pins the native
engines, verifies their registration inside the staged release, and synthesizes a sentence then transcribes it with Whisper. Voice
models download into a temporary CI cache. Both the archive and Debian package
include the validated Bare runtime and native dependencies. The release gate
requires model download access; it fails rather than publishing an untested
voice bundle. Microphone routing and physical speaker output still require a
local PipeWire session and working hardware.

Run the model roundtrip locally with:

```sh
bash packaging/bare-launch.sh packaging/bare-run.js packaging/voice-model-smoke.js
```

PipeWire capture/playback use `--properties node.name=Jarvis` and `-` for
standard I/O; `pw-cat` does not accept `--name`. A local capture check is
`bash packaging/bare-launch.sh packaging/bare-run.js packaging/pipewire-smoke.js`.
Add `--play` to the model roundtrip to exercise speaker playback as well.