104 lines
5.2 KiB
Markdown
104 lines
5.2 KiB
Markdown
# API reference
|
|
|
|
## Session D-Bus
|
|
|
|
Bus name: `io.qvac.Jarvis`
|
|
Object path: `/io/qvac/Jarvis`
|
|
Interface: `io.qvac.Jarvis.Session`
|
|
|
|
The canonical XML contract is [io.qvac.Jarvis.Session.xml](../dbus/io.qvac.Jarvis.Session.xml).
|
|
|
|
| Method | Purpose |
|
|
| --- | --- |
|
|
| `Arm()` / `Sleep()` / `Shutdown()` | Control daemon voice lifecycle |
|
|
| `PushToTalk(bool)` | Begin/end typed or microphone turn |
|
|
| `Say(string)` | Speak text through local TTS |
|
|
| `Ask(string)` | Submit typed text to the harness |
|
|
| `ReloadSettings()` | Re-read `config.json` and return a status string |
|
|
| `PreviewVoice(string)` | Speak preview text without adding a chat turn |
|
|
| `StopSpeech()` | Interrupt TTS playback |
|
|
| `ResetContext()` | Clear the current conversation context |
|
|
| `Confirm(jobId, toolCallId, decision)` | Answer a confirmation gate |
|
|
| `Cancel()` | Cancel active request/job |
|
|
| `SetMode(string)` | Set capability mode (not HUD layout) |
|
|
| `GetState()` | Return serialized daemon state |
|
|
| `EnrollWake(string)` | Configure wake phrase |
|
|
| `IngestPath(string)` | Add a local path to RAG ingestion |
|
|
| `ComputerGrant(bool)` / `ComputerRevoke()` | Start/stop computer-use grant |
|
|
| `ComputerStatus()` | Return computer-use status JSON |
|
|
| `WebcamGrant()` / `WebcamRevoke()` | Start/stop camera grant (portal Access, then timed capture) |
|
|
| `WebcamStatus()` | Return camera grant status JSON |
|
|
| `GetRuntimeStatus()` | Return voice/runtime component status JSON |
|
|
| `AssessModelFit(string)` | Report whether a model fits the GPU profile |
|
|
| `DownloadModel(string)` | Start a model download |
|
|
| `CancelModel(string)` | Cancel a model download |
|
|
| `WipeComputerTraces()` | Wipe computer-use and webcam frames; returns whether wipe ran |
|
|
|
|
| Signal | Purpose |
|
|
| --- | --- |
|
|
| `StateChanged` | Voice/daemon state string |
|
|
| `ContextReset` | Conversation context was cleared |
|
|
| `WakeHeard` | Wake phrase detected |
|
|
| `PartialTranscript` / `FinalTranscript` | ASR text |
|
|
| `Token` / `Reply` | Streamed and final assistant text |
|
|
| `SpeakingLevel` / `ListeningLevel` | Audio RMS. The HUD no-ops these; there is no waveform |
|
|
| `ChipOffered` | Suggestion sidecar |
|
|
| `JobProgress` | Background job progress |
|
|
| `ComputerStep` / `ComputerHighlight` | CU step text and target highlight rect |
|
|
| `ConfirmationRequired` | Tool permission prompt |
|
|
| `Thinking` | Planner thought tokens |
|
|
| `ToolCall` / `ToolResult` | Tool activity JSON |
|
|
| `Error` | Code and message |
|
|
|
|
Payloads stay small; PCM and image bytes do not travel over D-Bus.
|
|
|
|
## Computer-use tools
|
|
|
|
Tools are registered with the harness and exposed through the daemon. The
|
|
observable contract includes `cu.grant`, `cu.revoke`, `cu.status`,
|
|
`cu.observe`, `cu.zoom`, `cu.tree`, `cu.find`, `cu.click`, `cu.drag`,
|
|
`cu.scroll`, `cu.type`, `cu.key`, and `cu.act`. Every action is checked against
|
|
the active grant and safety policy. The HUD shows visible CU chrome (target
|
|
highlight and agent cursor), not a waveform overlay.
|
|
|
|
## Search tools
|
|
|
|
Search and fetch open a headed Playwright Chromium window owned by
|
|
`browser-use/helper.js`. The Bare daemon never loads Playwright; it talks to
|
|
that Node sidecar over JSON lines. Builtin tool names stay the same. There is
|
|
no HTML/RSS scraper fallback and no SearXNG.
|
|
|
|
| Tool | Behavior |
|
|
| --- | --- |
|
|
| `web_search` | Public search in the Jarvis Chromium window. Default `engine=auto` tries DuckDuckGo, then Google. Pin `engine` to retry one backend. |
|
|
| `google_search` | Google first, then DuckDuckGo |
|
|
| `fetch_page` | Open a public http(s) URL in Chromium and return readable text, links, headings, and metadata; `offset`, `max_chars`, and `find` continue reading |
|
|
| `web_fetch` | Same as `fetch_page`, including IP lookup pages such as ifconfig.me |
|
|
| `wiki_search` | Wikipedia in Chromium |
|
|
| `hn_search` | Hacker News in Chromium |
|
|
| `code_search` | GitHub, npm, and MDN in Chromium |
|
|
| `browser` | Drive the same Chromium window. Actions: `navigate` (`url`), `snapshot` (numbered `refs`), `click`/`type` (`ref` from the last snapshot), `press` (`key`), `scroll` (`dy`), `wait` (`ms`). Snapshot before every click. Cookie walls and leftover challenges use this tool, not computer-use. |
|
|
|
|
Those tools run JavaScript and wait about 20s on bot-check pages. If a result
|
|
has `challenge: true`, finish the prompt in the Jarvis browser window and call
|
|
the tool again. Private, loopback, and metadata URLs fail before Chromium
|
|
starts. Shell HTTP (`curl`/`wget` over public hosts) remains blocked; use these
|
|
tools instead. Install the browser once with `npm run browser:install`. The
|
|
daemon looks up Node via `JARVIS_BROWSER_NODE` because `jarvisd` itself runs
|
|
under Bare.
|
|
|
|
## Webcam
|
|
|
|
`webcam` captures one still after the user presses **Allow now** on
|
|
Settings → Camera. That click is the grant; GNOME Camera portal Access is
|
|
best-effort. Capture still tries the portal, then v4l2. The frame is attached
|
|
for that vision turn. File paths are not spoken. Frames live under
|
|
`/tmp/jarvis-webcam` and are wiped on revoke, lock screen, cancel, or
|
|
`WipeComputerTraces()`.
|
|
|
|
## Local HTTP
|
|
|
|
QVAC's optional OpenAI-compatible server is localhost-only at port `11434`.
|
|
It is a diagnostic surface, not a second Jarvis runtime. Native QVAC-only
|
|
operations use `/qvac/v1/*` where supported by the installed QVAC version.
|