81 lines
3.1 KiB
Markdown
81 lines
3.1 KiB
Markdown
# System architecture
|
|
|
|
JARVIS-QVAC is split into a thin desktop surface, a user daemon, a cognitive
|
|
harness, and isolated capability adapters. This keeps GNOME Shell responsive,
|
|
prevents multiple model runtimes from competing for GPU memory, and makes
|
|
computer use revocable as a session capability.
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
subgraph Desktop[GNOME desktop session]
|
|
EXT[Shell extension\nARC HUD + panel state]
|
|
CC[Control Center\nGTK4/libadwaita]
|
|
PORTAL[XDG portals\nScreenCast + RemoteDesktop]
|
|
ATSPI[AT-SPI2\naccessible tree]
|
|
PW[PipeWire\nmicrophone + playback]
|
|
end
|
|
subgraph Service[User services]
|
|
D[ jarvisd\nNode 22 ESM ]
|
|
CU[Computer-use session\nobserve + act + audit]
|
|
end
|
|
subgraph Cognitive[Cognition]
|
|
H[agent-harness\nsessions + planner + memory]
|
|
S[Skills\nQVAC + desktop + files]
|
|
end
|
|
subgraph Inference[One inference authority]
|
|
M[QVAC master\nload / schedule / cancel]
|
|
Q[QVAC SDK 0.19.x]
|
|
GPU[(GPU backend)]
|
|
end
|
|
EXT <--> |session D-Bus| D
|
|
CC <--> |settings + diagnostics| D
|
|
D --> H --> S
|
|
D --> M --> Q --> GPU
|
|
D <--> CU
|
|
CU <--> PORTAL
|
|
CU <--> ATSPI
|
|
D <--> PW
|
|
```
|
|
|
|
## Process responsibilities
|
|
|
|
| Process | Owns | Must not own |
|
|
| --- | --- | --- |
|
|
| GNOME Shell extension | panel, ARC overlay, highlights, Shell facts, D-Bus client | model loading, microphone capture, input injection, blocking work |
|
|
| `jarvisd` | voice state, QVAC master, harness bridge, jobs, D-Bus service | arbitrary Shell evaluation or a second planner |
|
|
| Control Center | settings, model fit, permissions, first-run controls | inference loop or hidden data deletion |
|
|
| Computer-use helpers | portal sessions, frames, AT-SPI, EIS/legacy backend | model planning or lock-screen bypass |
|
|
| Agent harness | session history, planning, memory, tool loop, permissions | direct QVAC ownership or direct desktop hacks |
|
|
| QVAC master | SDK worker, GPU admission, lifecycle, serialization, cancellation | cloud APIs or independent model instances |
|
|
|
|
## Data and control boundaries
|
|
|
|
Control messages and small state changes use session D-Bus. Audio streams and
|
|
large screenshots use PipeWire, temporary files, or the local IPC channel. The
|
|
daemon never sends screen frames or microphone data to a remote service.
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Shell as GNOME Shell
|
|
participant Daemon as jarvisd
|
|
participant Harness as Harness
|
|
participant Master as QVAC master
|
|
participant GPU as GPU
|
|
Shell->>Daemon: Ask(text) / wake / PTT
|
|
Daemon->>Master: admit(interactive lane)
|
|
Daemon->>Harness: prompt(text, tools)
|
|
Harness->>Master: completion(request)
|
|
Master->>GPU: one serialized inference
|
|
GPU-->>Master: streamed tokens
|
|
Master-->>Harness: tokens + tool calls
|
|
Harness-->>Daemon: reply + sidecar
|
|
Daemon-->>Shell: Token / Reply / StateChanged
|
|
```
|
|
|
|
## Failure containment
|
|
|
|
QVAC failures, unavailable models, or failed background jobs are reported as
|
|
capability errors and do not silently switch to CPU inference. A daemon restart
|
|
releases the master and restores the user service according to systemd policy.
|
|
Computer-use revoke is immediate and independent of the language model.
|