100 lines
4.9 KiB
Markdown
100 lines
4.9 KiB
Markdown
# API reference
|
|
|
|
## Session D-Bus
|
|
|
|
Bus name: `io.qvac.Jarvis`
|
|
Object path: `/io/qvac/Jarvis`
|
|
Interface: `io.qvac.Jarvis.Session`
|
|
|
|
The canonical XML contract is [io.qvac.Jarvis.Session.xml](../dbus/io.qvac.Jarvis.Session.xml).
|
|
|
|
| Method | Purpose |
|
|
| --- | --- |
|
|
| `Arm()` / `Sleep()` / `Shutdown()` | Control daemon voice lifecycle |
|
|
| `PushToTalk(bool)` | Begin/end typed or microphone turn |
|
|
| `Say(string)` | Speak text through local TTS |
|
|
| `Ask(string)` | Submit typed text to the harness |
|
|
| `ReloadSettings()` | Re-read `config.json` and return a status string |
|
|
| `PreviewVoice(string)` | Speak preview text without adding a chat turn |
|
|
| `StopSpeech()` | Interrupt TTS playback |
|
|
| `ResetContext()` | Clear the current conversation context |
|
|
| `Confirm(jobId, toolCallId, decision)` | Answer a confirmation gate |
|
|
| `Cancel()` | Cancel active request/job |
|
|
| `SetMode(string)` | Set capability mode (not HUD layout) |
|
|
| `GetState()` | Return serialized daemon state |
|
|
| `EnrollWake(string)` | Configure wake phrase |
|
|
| `IngestPath(string)` | Add a local path to RAG ingestion |
|
|
| `ComputerGrant(bool)` / `ComputerRevoke()` | Start/stop computer-use grant |
|
|
| `ComputerStatus()` | Return computer-use status JSON |
|
|
| `WebcamGrant()` / `WebcamRevoke()` | Start/stop camera grant (portal Access, then timed capture) |
|
|
| `WebcamStatus()` | Return camera grant status JSON |
|
|
| `GetRuntimeStatus()` | Return voice/runtime component status JSON |
|
|
| `AssessModelFit(string)` | Report whether a model fits the GPU profile |
|
|
| `DownloadModel(string)` | Start a model download |
|
|
| `CancelModel(string)` | Cancel a model download |
|
|
| `WipeComputerTraces()` | Wipe computer-use and webcam frames; returns whether wipe ran |
|
|
|
|
| Signal | Purpose |
|
|
| --- | --- |
|
|
| `StateChanged` | Voice/daemon state string |
|
|
| `ContextReset` | Conversation context was cleared |
|
|
| `WakeHeard` | Wake phrase detected |
|
|
| `PartialTranscript` / `FinalTranscript` | ASR text |
|
|
| `Token` / `Reply` | Streamed and final assistant text |
|
|
| `SpeakingLevel` / `ListeningLevel` | Audio RMS. The HUD no-ops these; there is no waveform |
|
|
| `ChipOffered` | Suggestion sidecar |
|
|
| `JobProgress` | Background job progress |
|
|
| `ComputerStep` / `ComputerHighlight` | CU step text and target highlight rect |
|
|
| `ConfirmationRequired` | Tool permission prompt |
|
|
| `Thinking` | Planner thought tokens |
|
|
| `ToolCall` / `ToolResult` | Tool activity JSON |
|
|
| `Error` | Code and message |
|
|
|
|
Payloads stay small; PCM and image bytes do not travel over D-Bus.
|
|
|
|
## Computer-use tools
|
|
|
|
Tools are registered with the harness and exposed through the daemon. The
|
|
observable contract includes `cu.grant`, `cu.revoke`, `cu.status`,
|
|
`cu.observe`, `cu.zoom`, `cu.tree`, `cu.find`, `cu.click`, `cu.drag`,
|
|
`cu.scroll`, `cu.type`, `cu.key`, and `cu.act`. Every action is checked against
|
|
the active grant and safety policy. The HUD shows visible CU chrome (target
|
|
highlight and agent cursor), not a waveform overlay.
|
|
|
|
## Search tools
|
|
|
|
The only public web tool is `browser`. It opens a headed Playwright Chromium
|
|
window owned by `browser-use/helper.js`. The Bare daemon never loads Playwright;
|
|
it talks to that Node sidecar over JSON lines. `web_search`, `google_search`,
|
|
`fetch_page`, `web_fetch`, `wiki_search`, `hn_search`, and `code_search` are
|
|
not registered. There is no HTML/RSS scraper fallback and no SearXNG.
|
|
|
|
| Tool | Behavior |
|
|
| --- | --- |
|
|
| `browser` | The only web tool. Actions: `navigate` (`url`, returns page text), `snapshot` (numbered `refs` plus page text), `click`/`type` (`ref` from the last snapshot), `press` (`key`), `scroll` (`dy`), `wait` (`ms`). Search by navigating to a public search URL, then click a result ref. Cookie walls and leftover challenges use this tool, not computer-use. |
|
|
|
|
`browser` runs JavaScript and waits about 20s on bot-check pages. If a result
|
|
has `challenge: true`, finish the prompt in the Jarvis browser window and call
|
|
the tool again. Private, loopback, and metadata URLs fail before Chromium
|
|
starts. Shell HTTP (`curl`/`wget` over public hosts) remains blocked; use
|
|
`browser` instead. Install the browser once with `npm run browser:install`. That
|
|
also unpacks uBlock Origin Lite and Dark Reader into the Jarvis Chromium
|
|
profile and the helper loads them on the next start. The
|
|
daemon looks up Node via `JARVIS_BROWSER_NODE` because `jarvisd` itself runs
|
|
under Bare.
|
|
|
|
## Webcam
|
|
|
|
`webcam` captures one still after the user presses **Allow now** on
|
|
Settings → Camera. That click is the grant; GNOME Camera portal Access is
|
|
best-effort. Capture still tries the portal, then v4l2. The frame is attached
|
|
for that vision turn. File paths are not spoken. Frames live under
|
|
`/tmp/jarvis-webcam` and are wiped on revoke, lock screen, cancel, or
|
|
`WipeComputerTraces()`.
|
|
|
|
## Local HTTP
|
|
|
|
QVAC's optional OpenAI-compatible server is localhost-only at port `11434`.
|
|
It is a diagnostic surface, not a second Jarvis runtime. Native QVAC-only
|
|
operations use `/qvac/v1/*` where supported by the installed QVAC version.
|