@@ -39,9 +39,11 @@ sequenceDiagram
|
||||
5. Vision coordinates for broken accessibility.
|
||||
6. Opt-in ydotool/X11 fallback.
|
||||
|
||||
The primary Wayland input path is XDG RemoteDesktop plus EIS/libei. The
|
||||
portal helper refuses to report input readiness without a configured local EIS
|
||||
bridge (`JARVIS_LIBEI_BRIDGE`). Screenshot frames are downscaled for vision,
|
||||
The primary Wayland input path is XDG RemoteDesktop plus EIS/libei. After
|
||||
portal consent, Jarvis opens `ConnectToEIS` and injects pointer/keyboard
|
||||
events through a built-in libei sender. `JARVIS_LIBEI_BRIDGE` is an optional
|
||||
override. If EIS is unavailable, the helper falls back to portal Notify
|
||||
methods instead of refusing readiness. Screenshot frames are downscaled for vision,
|
||||
kept in temporary storage, and removed on revoke unless trace retention is
|
||||
enabled.
|
||||
|
||||
|
||||
@@ -3,12 +3,13 @@
|
||||
Run these on Ubuntu GNOME after `npm run cu-doctor` reports its required portal,
|
||||
PipeWire, AT-SPI, and libei checks as passed. The ydotool result is optional.
|
||||
The explicit computer-use grant is required, screenshots stay in
|
||||
`/tmp/jarvis-cu`, and actuation is enabled only after the local EIS injector
|
||||
`/tmp/jarvis-cu`, and actuation is enabled only after the portal helper
|
||||
reports ready.
|
||||
|
||||
For live pointer and keyboard injection, configure `JARVIS_LIBEI_BRIDGE` with
|
||||
the local EIS injector command. The portal helper refuses readiness without it;
|
||||
this prevents a consent-only session from being mistaken for working input.
|
||||
For live pointer and keyboard injection, Jarvis uses xdg-desktop-portal
|
||||
`ConnectToEIS` plus a built-in libei sender. Set `JARVIS_LIBEI_BRIDGE` only to
|
||||
replace that sender. The helper no longer refuses readiness when the env var is
|
||||
unset.
|
||||
|
||||
1. Observe Settings and toggle Night Light.
|
||||
2. Type a sentence into Text Editor and save it.
|
||||
|
||||
@@ -18,10 +18,11 @@ remains useful for typed or manually triggered testing.
|
||||
|
||||
## Portal consent succeeds but input is unavailable
|
||||
|
||||
ScreenCast consent does not prove that RemoteDesktop/EIS injection is ready.
|
||||
Configure `JARVIS_LIBEI_BRIDGE`; the helper intentionally fails closed without
|
||||
it. Use observe-only mode until the injector reports ready. `ydotool` is an
|
||||
optional, explicitly enabled fallback.
|
||||
Grant desktop starts RemoteDesktop, then Jarvis connects to EIS with the
|
||||
bundled libei sender. An external `JARVIS_LIBEI_BRIDGE` is optional. If input
|
||||
still fails, run `npm run cu-doctor` and confirm `libei` is installed. Observation
|
||||
does not need the injector: after a grant, `cu_observe` should return a screenshot
|
||||
and accessibility tree even when pointer injection is down.
|
||||
|
||||
## The extension is missing
|
||||
|
||||
|
||||
Reference in New Issue
Block a user