snxraven/gnome-jarvis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

P7

Browse Source
This commit is contained in:
snxraven
2026-09-11 14:26:44 -04:00
parent 78cb03e6cb
commit b264c31b86
16 changed files with 181 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ROADMAP.md
+17 -11
View File
@@ -214,8 +214,8 @@ process and reflects daemon state without jank.
- [x] Keep frames local in `/tmp/jarvis-cu`, downscale to WebP before
perception, and return explicit unavailable reasons when portal, Shell, or
AT-SPI providers cannot connect.
- [ ] Implement `cu.act` and semantic `cu.click`.
- [ ] Implement grant/revoke, expiry, step budget, audit hashes, and no-frame
- [x] Implement `cu.act` and semantic `cu.click`.
- [x] Implement grant/revoke, expiry, step budget, audit hashes, and no-frame
retention by default.
Exit gate: “what’s on my screen?” returns grounded local observations without
@@ -223,16 +223,22 @@ actuation.
## Phase 7 — computer-use portal actuation
- [ ] Implement RemoteDesktop portal consent and restore tokens.
- [ ] Implement libei/EIS pointer, keyboard, scroll, drag, hover, and key input.
- [ ] Implement typed Unicode and submit behavior.
- [ ] Add target preview, agent cursor, and step ticker.
- [ ] Prefer domain tools, app D-Bus, AT-SPI, Shell helper, then vision
- [x] Implement RemoteDesktop portal consent and restore tokens.
- [x] Implement libei/EIS pointer, keyboard, scroll, drag, hover, and key input.
- [x] Implement typed Unicode and submit behavior.
- [x] Add target preview, agent cursor, and step ticker.
- [x] Prefer domain tools, app D-Bus, AT-SPI, Shell helper, then vision
coordinates in that order.
- [ ] Refuse password/PAM roles and lock-screen/greeter actions.
- [ ] Require confirmation for destructive or high-impact actions.
- [ ] Keep ydotool and X11 tools disabled unless explicitly enabled.
- [ ] Add state-change self-checks, animation waits, and no-progress aborts.
- [x] Refuse password/PAM roles and lock-screen/greeter actions.
- [x] Require confirmation for destructive or high-impact actions.
- [x] Keep ydotool and X11 tools disabled unless explicitly enabled.
- [x] Add state-change self-checks, animation waits, and no-progress aborts.
Actuation is implemented in `computer-use/portal-input.js`, `computer-use/actuator.js`,
`computer-use/safety.js`, `computer-use/audit.js`, `skills/computer-act.js`, and
the local portal/AT-SPI helpers. The input backend stays unavailable until the
GNOME RemoteDesktop/EIS helper completes consent; no hidden uinput or CPU/QVAC
fallback is used.
Exit gate: Night Light, Text Editor save, Firefox URL entry, hands-off abort,
and lock-screen kill pass `docs/cu-acceptance.md`.