gnome-jarvis/docs/harness-integration.md

Agent harness integration

The copied harness at vendor/agent-harness is the cognitive core. It is kept as a repository copy so releases are reproducible and do not depend on the machine at ~/dev/agent-harness after checkout. The original source remains the upstream reference described in agent-harness-map.

flowchart LR
  P[Prompt] --> B[harness-bridge.js]
  B --> S[Agent session]
  S --> L[agent/loop.js]
  L --> T{Tool call?}
  T -->|yes| R[Jarvis JSON-schema tool]
  R --> X[master / desktop / CU / files]
  X --> L
  T -->|no| O[Reply + HUD sidecar]
  S --> MEM[Local sessions + memory]

Harness responsibilities

• Multi-turn session state and compaction.
• Planner/tool-call loop and permission mode.
• Short- and long-term local memory.
• Tool schema validation and execution events.
• Streaming model messages adapted into daemon events.

Jarvis responsibilities

Jarvis registers domain tools from skills/, supplies the system prompt, maps tool permissions to confirmation gates, and translates harness events into the D-Bus protocol. The harness cwd is the per-user agent workspace ($XDG_DATA_HOME/jarvis/workspace, default ~/.local/share/jarvis/workspace), seeded from vendor/agent-harness/agent-workspace (SOUL.md, IDENTITY.md, AGENTS.md, USER.md, MEMORY.md, BOOTSTRAP.md, PERSONA.md). Voice turns inline workspace files into the system prompt. Settings How they should act notes are injected as a ## Acting block in that system prompt only; they are not also inlined from PERSONA.md, and they override SOUL.md / IDENTITY.md for name and character. A first-run BOOTSTRAP.md ritual stays open until the agent overwrites it with # completed. Computer-use tools are custom harness tools backed by computer-use/; the computer-use layer never owns planning.

Adding a tool

1. Put the implementation in the appropriate skill module. Public web search and page fetch already live in vendor/agent-harness/agent/web-search.js (web_search, google_search, fetch_page, web_fetch, wiki_search, hn_search, code_search), which calls the Playwright sidecar in browser-use/. Extra clicks use the custom browser gateway from skills/browser-tools.js. Webcam stills use skills/webcam-tools.js after Settings → Camera → Allow now.
2. Export a JSON-schema tool with a stable name, description, timeout, and permission level.
3. Route QVAC work through the master facade.
4. Add a focused fixture test with a fake master or backend.
5. Update API, the capability matrix, and the roadmap if the tool is user-visible.

Destructive tools require explicit confirmation. Computer-use tools additionally require an active grant, visible CU chrome / target highlight, a live step budget, and a lock-screen check. The webcam tool needs Camera enabled in Settings plus a separate Allow now grant.