# Computer use

Computer use is a revocable mode that combines observation, semantic actions,
portal input, and a visible ARC overlay. It is never implicitly enabled by a
normal chat turn.

```mermaid
sequenceDiagram
  participant User
  participant D as jarvisd
  participant H as Harness
  participant CU as CU session
  participant P as Portal/EIS
  participant A as AT-SPI
  User->>D: “take the wheel”
  D->>CU: grant(persist, monitors)
  CU->>P: ScreenCast + RemoteDesktop consent
  User-->>P: approve
  loop step budget
    CU->>P: capture frame
    CU->>A: tree snapshot
    CU->>H: observe bundle
    H-->>CU: one action
    CU->>CU: safety + target validation
    CU->>P: EIS input or
    CU->>A: named action
  end
  User->>D: “hands off” / Escape
  D->>CU: revoke
  CU->>P: close sessions immediately
```

## Actuator priority

1. Domain skill or app API.
2. App D-Bus/GIO action.
3. AT-SPI named action or stable reference.
4. GNOME Shell window/focus helper.
5. Vision coordinates for broken accessibility.
6. Opt-in ydotool/X11 fallback.

The primary Wayland input path is XDG RemoteDesktop plus EIS/libei. The
portal helper refuses to report input readiness without a configured local EIS
bridge (`JARVIS_LIBEI_BRIDGE`). Screenshot frames are downscaled for vision,
kept in temporary storage, and removed on revoke unless trace retention is
enabled.

## Safety rules

- Grant is off by default and dies on lock screen.
- Password/PAM roles are refused.
- Dangerous keys and destructive targets need confirmation.
- Default budget is 20 actions; hard cap is 100.
- Each action gets a visible target highlight and audit record.
- Three state-changing steps with no observed change abort the run.

See [computer-use acceptance](cu-acceptance.md) for the live GNOME scenarios.