@@ -4,20 +4,20 @@
|
|||||||
|
|
||||||
Local voice for your desktop.
|
Local voice for your desktop.
|
||||||
|
|
||||||
JARVIS-QVAC is a local-first voice assistant and computer-use layer for Ubuntu
|
Jarvis is a local-first voice assistant and computer-use layer for Ubuntu
|
||||||
GNOME. It combines a wake-word loop, QVAC speech and language models, the
|
GNOME. It combines a wake-word loop, QVAC speech and language models, the
|
||||||
copied agent-harness planner, GNOME-native UI, local memory, and an explicit
|
copied agent-harness planner, GNOME-native UI, local memory, and an explicit
|
||||||
Wayland computer-use grant.
|
Wayland computer-use grant.
|
||||||
|
|
||||||
The implementation covers Phases 0–11: daemon lifecycle, GPU-gated QVAC
|
The implementation covers Phases 0–11: daemon lifecycle, GPU-gated QVAC
|
||||||
ownership, voice, ARC UI, computer use, control center, capability adapters,
|
ownership, voice, tray HUD, computer use, control center, capability adapters,
|
||||||
privacy hardening, packaging, and Gitea release automation. Full production
|
privacy hardening, packaging, and Gitea release automation. Full production
|
||||||
acceptance remains host-dependent on GPU visibility, GNOME portals, the local
|
acceptance remains host-dependent on GPU visibility, GNOME portals, the local
|
||||||
wake bridge, and EIS input.
|
wake bridge, and EIS input.
|
||||||
|
|
||||||
## Runtime contract
|
## Runtime contract
|
||||||
|
|
||||||
JARVIS-QVAC has one cognitive loop and one QVAC authority.
|
Jarvis has one cognitive loop and one QVAC authority.
|
||||||
|
|
||||||
- jarvisd owns the QVAC SDK worker, model lifecycle, GPU policy, scheduler,
|
- jarvisd owns the QVAC SDK worker, model lifecycle, GPU policy, scheduler,
|
||||||
cancellation, voice loop, jobs, and D-Bus service.
|
cancellation, voice loop, jobs, and D-Bus service.
|
||||||
@@ -33,7 +33,7 @@ JARVIS-QVAC has one cognitive loop and one QVAC authority.
|
|||||||
|
|
||||||
~~~mermaid
|
~~~mermaid
|
||||||
flowchart LR
|
flowchart LR
|
||||||
U[User] --> G[GNOME Shell / ARC]
|
U[User] --> G[GNOME Shell HUD]
|
||||||
G <--> D[jarvisd / packaged Bare]
|
G <--> D[jarvisd / packaged Bare]
|
||||||
D --> H[agent-harness]
|
D --> H[agent-harness]
|
||||||
D --> M[Single QVAC master]
|
D --> M[Single QVAC master]
|
||||||
@@ -54,8 +54,9 @@ flowchart LR
|
|||||||
VLA, and ABot-World lab slots.
|
VLA, and ABot-World lab slots.
|
||||||
- GNOME app launch/focus/window actions, clipboard, notifications, media,
|
- GNOME app launch/focus/window actions, clipboard, notifications, media,
|
||||||
screenshot observation, AT-SPI actions, portal input, and vision fallback.
|
screenshot observation, AT-SPI actions, portal input, and vision fallback.
|
||||||
- ARC panel indicator and overlay, live waveform, transcripts, suggestions,
|
- Tray panel mark, compact conversation menu, Chat/Thinking tabs, hold-to-talk,
|
||||||
job progress, target highlights, and visible agent cursor.
|
optional expanded session, Settings/Control Center, job progress, target
|
||||||
|
highlights, and a visible agent cursor.
|
||||||
|
|
||||||
Unavailable capabilities remain visible with an honest reason such as missing
|
Unavailable capabilities remain visible with an honest reason such as missing
|
||||||
GPU, insufficient VRAM, missing model, unavailable hardware, or missing
|
GPU, insufficient VRAM, missing model, unavailable hardware, or missing
|
||||||
@@ -166,7 +167,7 @@ inference owner beside jarvisd.
|
|||||||
## Computer use
|
## Computer use
|
||||||
|
|
||||||
Computer use is a mode, not default chat behavior. Say “take the wheel” or
|
Computer use is a mode, not default chat behavior. Say “take the wheel” or
|
||||||
select Computer in ARC to request a grant. Jarvis then prefers domain actions,
|
open Settings and grant desktop access. Jarvis then prefers domain actions,
|
||||||
app D-Bus/GIO, AT-SPI references, GNOME Shell helpers, and finally vision
|
app D-Bus/GIO, AT-SPI references, GNOME Shell helpers, and finally vision
|
||||||
coordinates. RemoteDesktop plus EIS/libei is the primary Wayland path.
|
coordinates. RemoteDesktop plus EIS/libei is the primary Wayland path.
|
||||||
Jarvis ships a built-in libei injector; `JARVIS_LIBEI_BRIDGE` is optional.
|
Jarvis ships a built-in libei injector; `JARVIS_LIBEI_BRIDGE` is optional.
|
||||||
@@ -244,9 +245,11 @@ installation.
|
|||||||
The [documentation index](docs/README.md) is the authoritative guide.
|
The [documentation index](docs/README.md) is the authoritative guide.
|
||||||
|
|
||||||
- [Architecture](docs/architecture.md)
|
- [Architecture](docs/architecture.md)
|
||||||
|
- [Brand](docs/brand.md)
|
||||||
- [QVAC runtime and GPU authority](docs/qvac-runtime.md)
|
- [QVAC runtime and GPU authority](docs/qvac-runtime.md)
|
||||||
- [Agent harness integration](docs/harness-integration.md)
|
- [Agent harness integration](docs/harness-integration.md)
|
||||||
- [Voice pipeline](docs/voice-pipeline.md)
|
- [Voice pipeline](docs/voice-pipeline.md)
|
||||||
|
- [Settings](docs/settings.md)
|
||||||
- [Computer use](docs/computer-use.md)
|
- [Computer use](docs/computer-use.md)
|
||||||
- [API reference](docs/api.md)
|
- [API reference](docs/api.md)
|
||||||
- [Operations](docs/operations.md)
|
- [Operations](docs/operations.md)
|
||||||
|
|||||||
@@ -1,14 +1,16 @@
|
|||||||
# Jarvis settings
|
# Jarvis settings
|
||||||
|
|
||||||
Run `python3 apps/control-center/main.py` (or `gjs -m apps/control-center/main.js`).
|
Run `python3 apps/control-center/main.py` (or `gjs -m apps/control-center/main.js`).
|
||||||
The Control Center and the GNOME extension's Settings button now open the same
|
After `packaging/install.sh`, the **Jarvis** launcher
|
||||||
Libadwaita settings pages and use the same configuration catalog. Requires GJS,
|
(`io.qvac.Jarvis.Control`) opens the same window. The Control Center and the
|
||||||
GTK 4, and Libadwaita 1.4 or newer. The window uses the Jarvis mark, Gold accent,
|
GNOME extension's Settings button share the Libadwaita pages and configuration
|
||||||
and tagline “Local voice for your desktop.”
|
catalog. Requires GJS, GTK 4, and Libadwaita 1.4 or newer. The window uses the
|
||||||
|
Jarvis mark, Gold accent, and tagline “Local voice for your desktop.” Copyright
|
||||||
|
HoneyPeer, LLC; license AGPLv3 or later.
|
||||||
|
|
||||||
Voice, listening, and desktop settings are saved to
|
Voice, listening, and desktop settings are saved to
|
||||||
`$XDG_CONFIG_HOME/jarvis/config.json` (default `~/.config/jarvis/config.json`).
|
`$XDG_CONFIG_HOME/jarvis/config.json` (default `~/.config/jarvis/config.json`).
|
||||||
Use **Apply** or **Apply & Preview**. Appearance and shortcuts use GSettings and
|
Use **Apply** or **Apply & Preview**. Appearance and shortcuts use GSettings and
|
||||||
apply immediately. Chat model and agent limits require restarting `jarvisd`.
|
apply immediately. Chat model and agent limits require restarting `jarvisd`.
|
||||||
|
|
||||||
See [settings reference](../../docs/settings.md) for details.
|
See [settings reference](../../docs/settings.md) and [brand](../../docs/brand.md).
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import Gio from 'gi://Gio';
|
|||||||
import GLib from 'gi://GLib';
|
import GLib from 'gi://GLib';
|
||||||
import St from 'gi://St';
|
import St from 'gi://St';
|
||||||
import Clutter from 'gi://Clutter';
|
import Clutter from 'gi://Clutter';
|
||||||
|
import Pango from 'gi://Pango';
|
||||||
import { installShellService } from './shell-dbus.js';
|
import { installShellService } from './shell-dbus.js';
|
||||||
import { ConversationView, JarvisOsd, ComputerUseChrome, SessionPanel, coerceText, safeText, shortError } from './ui.js';
|
import { ConversationView, JarvisOsd, ComputerUseChrome, SessionPanel, coerceText, safeText, shortError } from './ui.js';
|
||||||
|
|
||||||
@@ -71,10 +72,12 @@ export default class JarvisExtension extends Extension {
|
|||||||
const file = Gio.File.new_for_path(`${this.path}/brand/icons/jarvis-mark.svg`);
|
const file = Gio.File.new_for_path(`${this.path}/brand/icons/jarvis-mark.svg`);
|
||||||
if (file.query_exists(null)) this._mark.gicon = new Gio.FileIcon({ file });
|
if (file.query_exists(null)) this._mark.gicon = new Gio.FileIcon({ file });
|
||||||
} catch (error) { log(`Jarvis mark unavailable: ${error.message}`); }
|
} catch (error) { log(`Jarvis mark unavailable: ${error.message}`); }
|
||||||
|
this._mark.set_icon_size(16);
|
||||||
this._glyph = new St.Label({ text: 'Jarvis', style_class: 'jarvis-panel-glyph', y_align: Clutter.ActorAlign.CENTER }); this._glyph.accessible_name = 'Jarvis idle';
|
this._glyph = new St.Label({ text: 'Jarvis', style_class: 'jarvis-panel-glyph', y_align: Clutter.ActorAlign.CENTER }); this._glyph.accessible_name = 'Jarvis idle';
|
||||||
|
if (this._glyph.clutter_text) this._glyph.clutter_text.ellipsize = Pango.EllipsizeMode.NONE;
|
||||||
this._panelBox.add_child(this._mark);
|
this._panelBox.add_child(this._mark);
|
||||||
this._panelBox.add_child(this._glyph);
|
this._panelBox.add_child(this._glyph);
|
||||||
this.osd.attach(this._panelBox);
|
this.osd.attach(this._panelBox, this._glyph);
|
||||||
this._indicator.add_child(this._panelBox);
|
this._indicator.add_child(this._panelBox);
|
||||||
Main.panel.addToStatusArea('jarvis-qvac', this._indicator, 0, 'right');
|
Main.panel.addToStatusArea('jarvis-qvac', this._indicator, 0, 'right');
|
||||||
this._mountPopup();
|
this._mountPopup();
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
"name": "Jarvis",
|
"name": "Jarvis",
|
||||||
"description": "Local voice for your desktop. Jarvis QVAC lives in the GNOME top bar.",
|
"description": "Local voice for your desktop. Jarvis QVAC lives in the GNOME top bar.",
|
||||||
"shell-version": ["46", "47", "48", "49", "50"],
|
"shell-version": ["46", "47", "48", "49", "50"],
|
||||||
"version": 2,
|
"version": 3,
|
||||||
"session-modes": ["user"],
|
"session-modes": ["user"],
|
||||||
"settings-schema": "org.gnome.shell.extensions.jarvis"
|
"settings-schema": "org.gnome.shell.extensions.jarvis"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -46,7 +46,8 @@ function accentRow(settings) {
|
|||||||
button.connect('clicked', () => settings.set_string('accent-color', accent.value));
|
button.connect('clicked', () => settings.set_string('accent-color', accent.value));
|
||||||
box.append(button);
|
box.append(button);
|
||||||
}
|
}
|
||||||
const entry = new Gtk.Entry({ text: current, width_chars: 8, maxlength: 7, valign: Gtk.Align.CENTER });
|
const entry = new Gtk.Entry({ text: current, width_chars: 9, valign: Gtk.Align.CENTER });
|
||||||
|
try { entry.get_buffer().set_max_length(7); } catch {}
|
||||||
entry.connect('changed', () => {
|
entry.connect('changed', () => {
|
||||||
const value = normalizeAccent(entry.get_text(), '');
|
const value = normalizeAccent(entry.get_text(), '');
|
||||||
if (value) settings.set_string('accent-color', value);
|
if (value) settings.set_string('accent-color', value);
|
||||||
|
|||||||
@@ -1,91 +1,57 @@
|
|||||||
/* Jarvis brand tokens. Gold is presence; cyan is thought. Accent is overridden live via --jarvis-accent. */
|
.jarvis-panel-box { spacing: 6px; }
|
||||||
.jarvis-panel-box {
|
.jarvis-panel-mark { color: var(--jarvis-accent, #F4B942); width: 16px; height: 16px; }
|
||||||
spacing: 7px;
|
.jarvis-panel-glyph { color: var(--jarvis-accent, #F4B942); font-size: 13px; font-weight: bold; }
|
||||||
--jarvis-signal: #4FD2FF;
|
.jarvis-panel-state { padding: 3px 10px; border-radius: 999px; background-color: rgba(244, 185, 66, .22); }
|
||||||
--jarvis-ink: #0B0E14;
|
.jarvis-panel-state-label { color: var(--jarvis-accent, #F4B942); font-size: 12px; font-weight: bold; }
|
||||||
--jarvis-slate: #16191F;
|
.jarvis-panel-state.jarvis-state-listening, .jarvis-panel-state.jarvis-state-speaking { background-color: rgba(244, 185, 66, .28); }
|
||||||
--jarvis-paper: #F6F7FB;
|
.jarvis-panel-state.jarvis-state-thinking { background-color: rgba(79, 210, 255, .22); }
|
||||||
--jarvis-mute: #AEB6C8;
|
|
||||||
--jarvis-dim: #9AA8C0;
|
|
||||||
}
|
|
||||||
.jarvis-panel-mark { color: var(--jarvis-accent, #F4B942); icon-size: 16px; }
|
|
||||||
.jarvis-panel-glyph {
|
|
||||||
color: var(--jarvis-accent, #F4B942);
|
|
||||||
font-size: 13px;
|
|
||||||
font-weight: 700;
|
|
||||||
letter-spacing: 0.8px;
|
|
||||||
}
|
|
||||||
.jarvis-panel-state { padding: 2px 9px; border-radius: 999px; background-color: alpha(var(--jarvis-accent, #F4B942), .22); }
|
|
||||||
.jarvis-panel-state-label { color: var(--jarvis-accent, #F4B942); font-size: 11px; font-weight: 700; letter-spacing: 0.3px; }
|
|
||||||
.jarvis-panel-box.jarvis-state-thinking .jarvis-panel-state,
|
|
||||||
.jarvis-panel-state.jarvis-state-thinking { background-color: alpha(#4FD2FF, .22); }
|
|
||||||
.jarvis-panel-box.jarvis-state-thinking .jarvis-panel-state-label,
|
|
||||||
.jarvis-panel-state.jarvis-state-thinking .jarvis-panel-state-label { color: #4FD2FF; }
|
.jarvis-panel-state.jarvis-state-thinking .jarvis-panel-state-label { color: #4FD2FF; }
|
||||||
.jarvis-panel-box.jarvis-state-sleeping .jarvis-panel-state,
|
|
||||||
.jarvis-panel-state.jarvis-state-sleeping { background-color: rgba(174, 182, 200, .18); }
|
.jarvis-panel-state.jarvis-state-sleeping { background-color: rgba(174, 182, 200, .18); }
|
||||||
.jarvis-panel-box.jarvis-state-sleeping .jarvis-panel-state-label,
|
|
||||||
.jarvis-panel-state.jarvis-state-sleeping .jarvis-panel-state-label { color: #AEB6C8; }
|
.jarvis-panel-state.jarvis-state-sleeping .jarvis-panel-state-label { color: #AEB6C8; }
|
||||||
.jarvis-menu { max-width: 360px; }
|
.jarvis-menu { max-width: 360px; }
|
||||||
.jarvis-menu-item { padding: 0; }
|
.jarvis-menu-item { padding: 0; }
|
||||||
.jarvis-popup {
|
.jarvis-popup, .jarvis-session { color: #F6F7FB; }
|
||||||
width: 320px;
|
.jarvis-popup { width: 320px; padding: 12px 14px; spacing: 8px; background-color: #0B0E14; border-radius: 16px; }
|
||||||
padding: 12px 14px;
|
.jarvis-session { width: 420px; padding: 14px 16px; spacing: 8px; border-radius: 16px; background-color: #0B0E14; border: 1px solid rgba(244, 185, 66, .42); box-shadow: 0 12px 40px rgba(0, 0, 0, .45); }
|
||||||
spacing: 8px;
|
|
||||||
color: #f6f7fb;
|
|
||||||
--jarvis-signal: #4FD2FF;
|
|
||||||
}
|
|
||||||
.jarvis-session {
|
|
||||||
width: 420px;
|
|
||||||
padding: 14px 16px;
|
|
||||||
spacing: 8px;
|
|
||||||
border-radius: 16px;
|
|
||||||
background-color: rgba(11, 14, 20, .94);
|
|
||||||
border: 1px solid alpha(var(--jarvis-accent, #F4B942), .42);
|
|
||||||
color: #f6f7fb;
|
|
||||||
box-shadow: 0 12px 40px rgba(0, 0, 0, .45);
|
|
||||||
--jarvis-signal: #4FD2FF;
|
|
||||||
}
|
|
||||||
.jarvis-popup-header { spacing: 8px; }
|
.jarvis-popup-header { spacing: 8px; }
|
||||||
.jarvis-mark { icon-size: 18px; color: var(--jarvis-accent, #F4B942); }
|
.jarvis-mark { color: var(--jarvis-accent, #F4B942); }
|
||||||
.jarvis-title { font-weight: 700; letter-spacing: 0.6px; font-size: 15px; color: #f6f7fb; }
|
.jarvis-title { font-weight: bold; font-size: 15px; color: #F6F7FB; }
|
||||||
.jarvis-local { color: var(--jarvis-accent, #F4B942); font-size: 11px; font-weight: 700; letter-spacing: 0.4px; }
|
.jarvis-local { color: #D7DDE8; font-size: 11px; font-weight: bold; }
|
||||||
.jarvis-status-line { color: #aeb6c8; font-size: 12px; }
|
.jarvis-status-line { color: #D7DDE8; font-size: 12px; }
|
||||||
.jarvis-tabs { spacing: 6px; }
|
.jarvis-tabs { spacing: 6px; }
|
||||||
.jarvis-tab { padding: 4px 10px; border-radius: 999px; background-color: transparent; color: #9aa8c0; font-size: 12px; }
|
.jarvis-tab { padding: 4px 10px; border-radius: 999px; background-color: transparent; color: #D7DDE8; font-size: 12px; }
|
||||||
.jarvis-tab:hover, .jarvis-tab:focus { color: #f6f7fb; background-color: rgba(255, 255, 255, .08); }
|
.jarvis-tab:hover, .jarvis-tab:focus { color: #F6F7FB; background-color: rgba(255, 255, 255, .10); }
|
||||||
.jarvis-tab-active { background-color: var(--jarvis-accent, #F4B942); color: #16191f; font-weight: 700; }
|
.jarvis-tab-active { background-color: #F4B942; color: #16191F; font-weight: bold; }
|
||||||
.jarvis-thinking { color: #aeb6c8; font-size: 12px; padding: 8px 10px; border-left: 2px solid #4FD2FF; background-color: rgba(79, 210, 255, .06); border-radius: 8px; }
|
.jarvis-thinking { color: #D7DDE8; font-size: 12px; padding: 8px 10px; border-left: 2px solid #4FD2FF; background-color: rgba(79, 210, 255, .08); border-radius: 8px; }
|
||||||
.jarvis-notice { color: var(--jarvis-accent, #F4B942); font-size: 12px; }
|
.jarvis-notice { color: #F4B942; font-size: 12px; }
|
||||||
.jarvis-confirm { spacing: 8px; padding: 4px 0; }
|
.jarvis-confirm { spacing: 8px; padding: 4px 0; }
|
||||||
.jarvis-confirm-actions { spacing: 6px; }
|
.jarvis-confirm-actions { spacing: 6px; }
|
||||||
.jarvis-confirm-label { color: #f6f7fb; font-size: 13px; }
|
.jarvis-confirm-label { color: #F6F7FB; font-size: 13px; }
|
||||||
.jarvis-popup-scroll { height: 180px; }
|
.jarvis-popup-scroll { height: 180px; }
|
||||||
.jarvis-thinking-scroll { height: 160px; }
|
.jarvis-thinking-scroll { height: 160px; }
|
||||||
.jarvis-thinking-box { spacing: 0; }
|
.jarvis-thinking-box { spacing: 0; }
|
||||||
.jarvis-session-scroll { height: 220px; }
|
.jarvis-session-scroll { height: 220px; }
|
||||||
.jarvis-session-thinking { height: 200px; }
|
.jarvis-session-thinking { height: 200px; }
|
||||||
.jarvis-transcript { spacing: 6px; }
|
.jarvis-transcript { spacing: 6px; }
|
||||||
.jarvis-row { padding: 6px 10px; border-radius: 8px; font-size: 13px; }
|
.jarvis-row { padding: 6px 10px; border-radius: 8px; font-size: 13px; color: #F6F7FB; }
|
||||||
.jarvis-row-user { border-left: 2px solid var(--jarvis-accent, #F4B942); background-color: alpha(var(--jarvis-accent, #F4B942), .08); }
|
.jarvis-row-user { border-left: 2px solid #F4B942; background-color: rgba(244, 185, 66, .10); }
|
||||||
.jarvis-row-jarvis { border-right: 2px solid #4FD2FF; background-color: rgba(79, 210, 255, .05); }
|
.jarvis-row-jarvis { border-right: 2px solid #4FD2FF; background-color: rgba(79, 210, 255, .08); }
|
||||||
.jarvis-row-tool { background-color: transparent; color: #9aa8c0; font-size: 12px; padding: 2px 6px; }
|
.jarvis-row-tool { background-color: transparent; color: #D7DDE8; font-size: 12px; padding: 2px 6px; }
|
||||||
.jarvis-chip-scroll { height: 32px; }
|
.jarvis-chip-scroll { height: 32px; }
|
||||||
.jarvis-chips { spacing: 6px; }
|
.jarvis-chips { spacing: 6px; }
|
||||||
.jarvis-chip { padding: 5px 9px; border-radius: 999px; background-color: rgba(255, 255, 255, .08); }
|
.jarvis-chip { padding: 5px 9px; border-radius: 999px; background-color: rgba(255, 255, 255, .10); color: #F6F7FB; }
|
||||||
.jarvis-chip:hover, .jarvis-chip:focus { background-color: alpha(var(--jarvis-accent, #F4B942), .25); }
|
.jarvis-chip:hover, .jarvis-chip:focus { background-color: rgba(244, 185, 66, .28); color: #F6F7FB; }
|
||||||
.jarvis-chip-quiet { background-color: transparent; color: #aeb6c8; padding: 5px 8px; }
|
.jarvis-chip-quiet { background-color: transparent; color: #F6F7FB; padding: 5px 8px; }
|
||||||
.jarvis-chip-quiet:hover, .jarvis-chip-quiet:focus { color: #f6f7fb; background-color: rgba(255, 255, 255, .08); }
|
.jarvis-chip-quiet:hover, .jarvis-chip-quiet:focus { color: #FFFFFF; background-color: rgba(255, 255, 255, .10); }
|
||||||
.jarvis-talk { background-color: var(--jarvis-accent, #F4B942); color: #16191f; font-weight: 700; padding: 8px 14px; }
|
.jarvis-talk { background-color: #F4B942; color: #16191F; font-weight: bold; padding: 8px 14px; }
|
||||||
.jarvis-talk:hover, .jarvis-talk:focus { background-color: #ffe09a; }
|
.jarvis-talk:hover, .jarvis-talk:focus { background-color: #FFE09A; color: #16191F; }
|
||||||
.jarvis-talk:active { background-color: #ffe09a; }
|
.jarvis-talk:active { background-color: #FFE09A; color: #16191F; }
|
||||||
.jarvis-controls { spacing: 6px; }
|
.jarvis-controls { spacing: 6px; }
|
||||||
.jarvis-settings { font-size: 12px; padding: 4px 8px; }
|
.jarvis-settings { font-size: 12px; padding: 4px 8px; color: #F6F7FB; }
|
||||||
.jarvis-popup StEntry, .jarvis-session StEntry { border-radius: 10px; padding: 8px 10px; background-color: rgba(255, 255, 255, .06); color: #f6f7fb; border: 1px solid rgba(255, 255, 255, .15); }
|
.jarvis-popup StEntry, .jarvis-session StEntry { border-radius: 10px; padding: 8px 10px; background-color: rgba(255, 255, 255, .08); color: #F6F7FB; border: 1px solid rgba(255, 255, 255, .22); }
|
||||||
.jarvis-popup StEntry:focus, .jarvis-session StEntry:focus { border-color: var(--jarvis-accent, #F4B942); }
|
.jarvis-popup StEntry:focus, .jarvis-session StEntry:focus { border-color: #F4B942; color: #F6F7FB; }
|
||||||
.jarvis-cu { padding: 8px 12px; spacing: 4px; border-radius: 12px; background-color: rgba(11, 14, 20, .82); border: 1px solid rgba(79, 210, 255, .4); }
|
.jarvis-cu { padding: 8px 12px; spacing: 4px; border-radius: 12px; background-color: rgba(11, 14, 20, .90); border: 1px solid rgba(79, 210, 255, .4); }
|
||||||
.jarvis-job, .jarvis-target, .jarvis-cu-step { color: #4FD2FF; font-size: 12px; }
|
.jarvis-job, .jarvis-target, .jarvis-cu-step { color: #4FD2FF; font-size: 12px; }
|
||||||
.jarvis-agent-cursor { color: var(--jarvis-accent, #F4B942); font-size: 22px; }
|
.jarvis-agent-cursor { color: #F4B942; font-size: 22px; }
|
||||||
.jarvis-high-contrast { border-width: 2px; background-color: #000; }
|
.jarvis-high-contrast { border-width: 2px; background-color: #000; }
|
||||||
.jarvis-high-contrast .jarvis-title,
|
.jarvis-high-contrast .jarvis-title, .jarvis-high-contrast .jarvis-panel-glyph, .jarvis-high-contrast .jarvis-status-line, .jarvis-high-contrast .jarvis-local { color: #fff; }
|
||||||
.jarvis-high-contrast .jarvis-panel-glyph,
|
|
||||||
.jarvis-high-contrast .jarvis-status-line { color: #fff; }
|
|
||||||
|
|||||||
@@ -82,7 +82,8 @@ export class ConversationView {
|
|||||||
this.mark = brandMark(compact ? 16 : 20, brandDir);
|
this.mark = brandMark(compact ? 16 : 20, brandDir);
|
||||||
this.title = new St.Label({ text: 'Jarvis', style_class: 'jarvis-title', x_align: Clutter.ActorAlign.START });
|
this.title = new St.Label({ text: 'Jarvis', style_class: 'jarvis-title', x_align: Clutter.ActorAlign.START });
|
||||||
this.title.accessible_name = 'Jarvis status';
|
this.title.accessible_name = 'Jarvis status';
|
||||||
this.status = new St.Label({ text: 'LOCAL', style_class: 'jarvis-local' });
|
if (this.title.clutter_text) this.title.clutter_text.ellipsize = Pango.EllipsizeMode.NONE;
|
||||||
|
this.status = new St.Label({ text: 'Local', style_class: 'jarvis-local' });
|
||||||
this.status.accessible_name = 'Local model status';
|
this.status.accessible_name = 'Local model status';
|
||||||
if (this.status.clutter_text) this.status.clutter_text.ellipsize = Pango.EllipsizeMode.END;
|
if (this.status.clutter_text) this.status.clutter_text.ellipsize = Pango.EllipsizeMode.END;
|
||||||
if (this.mark) this.header.add_child(this.mark);
|
if (this.mark) this.header.add_child(this.mark);
|
||||||
@@ -342,16 +343,21 @@ export class ConversationView {
|
|||||||
this.replyFinalized = true;
|
this.replyFinalized = true;
|
||||||
}
|
}
|
||||||
setConnectionStatus(kind) {
|
setConnectionStatus(kind) {
|
||||||
const labels = { local: 'LOCAL', offline: 'LOCAL · daemon unavailable', 'voice-unavailable': 'LOCAL · voice unavailable' };
|
this._connection = kind;
|
||||||
|
const labels = { local: 'Local', offline: 'Offline', 'voice-unavailable': 'Voice off' };
|
||||||
this.status.text = labels[kind] || labels.local;
|
this.status.text = labels[kind] || labels.local;
|
||||||
this.status.accessible_name = this.status.text;
|
this.status.accessible_name = this.status.text;
|
||||||
|
this.status.style_class = 'jarvis-local';
|
||||||
}
|
}
|
||||||
setVoiceStatus({ tts, input, wake } = {}) {
|
setVoiceStatus({ tts, input, wake } = {}) {
|
||||||
this.status.text = `${tts ? 'SPEECH ON' : 'SPEECH OFF'} · ${input ? (wake ? 'WAKE ON' : 'HOLD TALK') : 'MIC UNAVAILABLE'}`;
|
this._voice = { tts: Boolean(tts), input: Boolean(input), wake: Boolean(wake) };
|
||||||
|
this.status.text = input ? 'Local' : 'Mic off';
|
||||||
this.status.accessible_name = this.status.text;
|
this.status.accessible_name = this.status.text;
|
||||||
|
this.status.style_class = 'jarvis-local';
|
||||||
this.talk.reactive = Boolean(input);
|
this.talk.reactive = Boolean(input);
|
||||||
this.talk.can_focus = Boolean(input);
|
this.talk.can_focus = Boolean(input);
|
||||||
this.talk.label = input ? 'Hold to talk' : 'Mic unavailable';
|
this.talk.label = input ? 'Hold to talk' : 'Mic unavailable';
|
||||||
|
this._refreshStatusLine();
|
||||||
}
|
}
|
||||||
setState(state) {
|
setState(state) {
|
||||||
const value = STATES.has(state) ? state : 'ARMED';
|
const value = STATES.has(state) ? state : 'ARMED';
|
||||||
@@ -368,15 +374,22 @@ export class ConversationView {
|
|||||||
}
|
}
|
||||||
this._refreshStatusLine();
|
this._refreshStatusLine();
|
||||||
this.title.text = value === 'SLEEPING' ? 'Jarvis · privacy' : 'Jarvis';
|
this.title.text = value === 'SLEEPING' ? 'Jarvis · privacy' : 'Jarvis';
|
||||||
this.status.style_class = `jarvis-local jarvis-state-${value.toLowerCase()}`;
|
|
||||||
}
|
}
|
||||||
_refreshStatusLine() {
|
_refreshStatusLine() {
|
||||||
const value = this._state;
|
const value = this._state;
|
||||||
if (value === 'LISTENING') this.statusLine.text = 'Listening';
|
const voice = this._voice || {};
|
||||||
else if (value === 'SPEAKING') this.statusLine.text = 'Speaking';
|
const bits = [];
|
||||||
else if (value === 'SLEEPING') this.statusLine.text = 'privacy · microphone off';
|
if (voice.tts) bits.push('speech on');
|
||||||
else if (value === 'THINKING') this.statusLine.text = this._activity || 'Thinking';
|
else if (voice.tts === false) bits.push('speech off');
|
||||||
else this.statusLine.text = 'armed · Hold Talk to speak';
|
if (voice.input && voice.wake) bits.push('wake on');
|
||||||
|
else if (voice.input) bits.push('hold talk');
|
||||||
|
else if (voice.input === false) bits.push('mic off');
|
||||||
|
const extra = bits.length ? ` · ${bits.join(' · ')}` : '';
|
||||||
|
if (value === 'LISTENING') this.statusLine.text = `Listening${extra}`;
|
||||||
|
else if (value === 'SPEAKING') this.statusLine.text = `Speaking${extra}`;
|
||||||
|
else if (value === 'SLEEPING') this.statusLine.text = `privacy · microphone off${extra}`;
|
||||||
|
else if (value === 'THINKING') this.statusLine.text = `${this._activity || 'Thinking'}${extra}`;
|
||||||
|
else this.statusLine.text = `armed · Hold Talk to speak${extra}`;
|
||||||
}
|
}
|
||||||
addChip(id, label, payload) {
|
addChip(id, label, payload) {
|
||||||
const chip = new St.Button({ label: safeText(label), style_class: 'jarvis-chip jarvis-chip-suggested', reactive: true, can_focus: true });
|
const chip = new St.Button({ label: safeText(label), style_class: 'jarvis-chip jarvis-chip-suggested', reactive: true, can_focus: true });
|
||||||
@@ -408,13 +421,19 @@ export class JarvisOsd {
|
|||||||
constructor() {
|
constructor() {
|
||||||
this.root = new St.BoxLayout({ style_class: 'jarvis-panel-state', visible: false, y_align: Clutter.ActorAlign.CENTER });
|
this.root = new St.BoxLayout({ style_class: 'jarvis-panel-state', visible: false, y_align: Clutter.ActorAlign.CENTER });
|
||||||
this.label = new St.Label({ text: '', style_class: 'jarvis-panel-state-label', y_align: Clutter.ActorAlign.CENTER });
|
this.label = new St.Label({ text: '', style_class: 'jarvis-panel-state-label', y_align: Clutter.ActorAlign.CENTER });
|
||||||
|
if (this.label.clutter_text) this.label.clutter_text.ellipsize = Pango.EllipsizeMode.NONE;
|
||||||
this.root.add_child(this.label);
|
this.root.add_child(this.label);
|
||||||
this.root.accessible_name = 'Jarvis status';
|
this.root.accessible_name = 'Jarvis status';
|
||||||
|
this._name = null;
|
||||||
this._state = 'ARMED';
|
this._state = 'ARMED';
|
||||||
this._wake = false;
|
this._wake = false;
|
||||||
this._timeout = 0;
|
this._timeout = 0;
|
||||||
}
|
}
|
||||||
attach(parent) { parent?.add_child?.(this.root); this.hide(); }
|
attach(parent, nameLabel) {
|
||||||
|
this._name = nameLabel || null;
|
||||||
|
parent?.add_child?.(this.root);
|
||||||
|
this.hide();
|
||||||
|
}
|
||||||
setState(state) {
|
setState(state) {
|
||||||
this._state = STATES.has(state) ? state : 'ARMED';
|
this._state = STATES.has(state) ? state : 'ARMED';
|
||||||
this._wake = false;
|
this._wake = false;
|
||||||
@@ -436,6 +455,7 @@ export class JarvisOsd {
|
|||||||
const text = this._wake ? 'Wake' : (PANEL_STATES[this._state] || '');
|
const text = this._wake ? 'Wake' : (PANEL_STATES[this._state] || '');
|
||||||
this.label.text = text;
|
this.label.text = text;
|
||||||
this.root.visible = Boolean(text);
|
this.root.visible = Boolean(text);
|
||||||
|
if (this._name) this._name.visible = !this.root.visible;
|
||||||
this.root.accessible_name = text ? `Jarvis ${text}` : 'Jarvis status';
|
this.root.accessible_name = text ? `Jarvis ${text}` : 'Jarvis status';
|
||||||
for (const name of ['armed', 'listening', 'speaking', 'thinking', 'sleeping']) {
|
for (const name of ['armed', 'listening', 'speaking', 'thinking', 'sleeping']) {
|
||||||
this.root.remove_style_class_name(`jarvis-state-${name}`);
|
this.root.remove_style_class_name(`jarvis-state-${name}`);
|
||||||
@@ -443,7 +463,12 @@ export class JarvisOsd {
|
|||||||
if (this._state) this.root.add_style_class_name(`jarvis-state-${this._state.toLowerCase()}`);
|
if (this._state) this.root.add_style_class_name(`jarvis-state-${this._state.toLowerCase()}`);
|
||||||
}
|
}
|
||||||
_clearTimer() { if (this._timeout) { GLib.Source.remove(this._timeout); this._timeout = 0; } }
|
_clearTimer() { if (this._timeout) { GLib.Source.remove(this._timeout); this._timeout = 0; } }
|
||||||
hide() { this._wake = false; this._clearTimer(); this.root.visible = false; }
|
hide() {
|
||||||
|
this._wake = false;
|
||||||
|
this._clearTimer();
|
||||||
|
this.root.visible = false;
|
||||||
|
if (this._name) this._name.visible = true;
|
||||||
|
}
|
||||||
destroy() { this._clearTimer(); this.root.destroy(); }
|
destroy() { this._clearTimer(); this.root.destroy(); }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+4
-2
@@ -14,8 +14,10 @@ data under `~/.config/jarvis`, `~/.local/share/jarvis`, and the model cache are
|
|||||||
preserved. If the service was running, the installer restarts it after the
|
preserved. If the service was running, the installer restarts it after the
|
||||||
replacement; `--enable` also enables the unit. The GNOME HUD files are always
|
replacement; `--enable` also enables the unit. The GNOME HUD files are always
|
||||||
copied into `~/.local/share/gnome-shell/extensions/[email protected]`, then the
|
copied into `~/.local/share/gnome-shell/extensions/[email protected]`, then the
|
||||||
installer asks the running Shell to reload and enable the extension so the
|
installer asks Shell to enable the extension. That request does not always
|
||||||
panel and overlay come back without a logout. Each of those steps is appended
|
hot-reload a stylesheet or JS error on GNOME 50 (`ReloadExtension` is not
|
||||||
|
implemented). Disable then enable `[email protected]` on `org.gnome.Shell`, or
|
||||||
|
start a new session, if the panel HUD does not come back. Each of those steps is appended
|
||||||
to `~/.local/state/jarvis/install.log` (or `$XDG_STATE_HOME/jarvis/install.log`)
|
to `~/.local/state/jarvis/install.log` (or `$XDG_STATE_HOME/jarvis/install.log`)
|
||||||
and to the system log as tag `jarvis-install`. Run
|
and to the system log as tag `jarvis-install`. Run
|
||||||
`systemctl --user daemon-reload` only when you install the service file by
|
`systemctl --user daemon-reload` only when you install the service file by
|
||||||
|
|||||||
+16
-6
@@ -1,7 +1,9 @@
|
|||||||
# JARVIS-QVAC documentation
|
# Jarvis documentation
|
||||||
|
|
||||||
JARVIS-QVAC is a local-first voice and computer-use assistant for Ubuntu GNOME.
|
Jarvis is a local-first voice and computer-use assistant for Ubuntu GNOME,
|
||||||
This directory is the maintained technical reference for the repository.
|
released by HoneyPeer, LLC under AGPLv3 or later. This directory is the
|
||||||
|
maintained technical reference for the repository. The Debian/npm package id
|
||||||
|
remains `jarvis-qvac`.
|
||||||
|
|
||||||
## Start here
|
## Start here
|
||||||
|
|
||||||
@@ -10,6 +12,8 @@ This directory is the maintained technical reference for the repository.
|
|||||||
| Understand the system | [Architecture](architecture.md) |
|
| Understand the system | [Architecture](architecture.md) |
|
||||||
| Run it locally | [Operations](operations.md) and [Local demo](local-demo.md) |
|
| Run it locally | [Operations](operations.md) and [Local demo](local-demo.md) |
|
||||||
| Understand the voice loop | [Voice pipeline](voice-pipeline.md) |
|
| Understand the voice loop | [Voice pipeline](voice-pipeline.md) |
|
||||||
|
| Change voice, listening, or appearance | [Settings](settings.md) |
|
||||||
|
| Brand tokens and HUD styling | [Brand](brand.md) |
|
||||||
| Build desktop control | [Computer use](computer-use.md) |
|
| Build desktop control | [Computer use](computer-use.md) |
|
||||||
| Integrate QVAC | [QVAC runtime](qvac-runtime.md) |
|
| Integrate QVAC | [QVAC runtime](qvac-runtime.md) |
|
||||||
| Run the packaged runtime | [Bare runtime](bare-runtime.md) |
|
| Run the packaged runtime | [Bare runtime](bare-runtime.md) |
|
||||||
@@ -20,12 +24,20 @@ This directory is the maintained technical reference for the repository.
|
|||||||
| Test changes | [Testing](testing.md) |
|
| Test changes | [Testing](testing.md) |
|
||||||
| Diagnose failures | [Troubleshooting](troubleshooting.md) |
|
| Diagnose failures | [Troubleshooting](troubleshooting.md) |
|
||||||
| Track implementation | [Roadmap](ROADMAP.md) |
|
| Track implementation | [Roadmap](ROADMAP.md) |
|
||||||
|
| Upgrade an install | [Migration notes](MIGRATIONS.md) |
|
||||||
|
| Release a build | [Release checklist](RELEASE.md) |
|
||||||
|
| Hardware and GPU fit | [Hardware compatibility](hardware-compatibility.md) |
|
||||||
|
| Capability matrix | [Capabilities](capabilities.md) |
|
||||||
|
| Development workflow | [Development](development.md) |
|
||||||
|
| Vendored harness layout | [Agent harness map](agent-harness-map.md) |
|
||||||
|
| Voice acceptance | [Voice acceptance](voice-acceptance.md) |
|
||||||
|
| Computer-use acceptance | [CU acceptance](cu-acceptance.md) |
|
||||||
|
|
||||||
## Runtime promise
|
## Runtime promise
|
||||||
|
|
||||||
The project has one cognition loop and one QVAC authority. `jarvisd` owns the
|
The project has one cognition loop and one QVAC authority. `jarvisd` owns the
|
||||||
QVAC SDK worker, model lifecycle, GPU policy, scheduler, and cancellation. The
|
QVAC SDK worker, model lifecycle, GPU policy, scheduler, and cancellation. The
|
||||||
vendored harness supplies planning, sessions, memory, and tool-loop behavior.
|
vendored harness supplies planning, sessions, memory, and the tool-loop behavior.
|
||||||
The GNOME Shell extension only renders state and forwards desktop facts. It
|
The GNOME Shell extension only renders state and forwards desktop facts. It
|
||||||
does not load models or perform inference.
|
does not load models or perform inference.
|
||||||
|
|
||||||
@@ -58,5 +70,3 @@ must be verified on the target GNOME session.
|
|||||||
|
|
||||||
Jarvis is released by HoneyPeer, LLC under the GNU Affero General Public
|
Jarvis is released by HoneyPeer, LLC under the GNU Affero General Public
|
||||||
License v3 or later. See [LICENSE](../LICENSE) and [COPYRIGHT](../COPYRIGHT).
|
License v3 or later. See [LICENSE](../LICENSE) and [COPYRIGHT](../COPYRIGHT).
|
||||||
|
|
||||||
- [Settings and voice customization](settings.md)
|
|
||||||
|
|||||||
+4
-3
@@ -2,9 +2,10 @@
|
|||||||
|
|
||||||
## Local verification
|
## Local verification
|
||||||
|
|
||||||
1. Confirm Node 22, GNOME dependencies, PipeWire, portal access, and a QVAC
|
1. Confirm Node 22 (test and package build only), GNOME dependencies, PipeWire,
|
||||||
GPU backend with `npm run gpu-doctor`, `npm run cu-doctor`, and
|
portal access, and a QVAC GPU backend with `npm run gpu-doctor`,
|
||||||
`npm run voice-doctor`.
|
`npm run cu-doctor`, and `npm run voice-doctor`. Production `jarvisd` is
|
||||||
|
packaged Bare, not Node.
|
||||||
2. Run `npm ci`, `npm test`, and `npm run package:test`.
|
2. Run `npm ci`, `npm test`, and `npm run package:test`.
|
||||||
3. Review `dist/SHA256SUMS` and test the user installer in a disposable user
|
3. Review `dist/SHA256SUMS` and test the user installer in a disposable user
|
||||||
session with `packaging/install.sh --enable`.
|
session with `packaging/install.sh --enable`.
|
||||||
|
|||||||
+11
-8
@@ -1,4 +1,4 @@
|
|||||||
# JARVIS-QVAC roadmap
|
# Jarvis roadmap
|
||||||
|
|
||||||
This is the implementation tracker for the local Ubuntu GNOME assistant. The
|
This is the implementation tracker for the local Ubuntu GNOME assistant. The
|
||||||
roadmap is ordered by dependency: runtime authority first, then cognition,
|
roadmap is ordered by dependency: runtime authority first, then cognition,
|
||||||
@@ -183,22 +183,25 @@ Jarvis” starts a local GPU-backed turn, speaks a response, and does not
|
|||||||
self-trigger from its own TTS. `docs/voice-acceptance.md` contains the live
|
self-trigger from its own TTS. `docs/voice-acceptance.md` contains the live
|
||||||
hardware/session acceptance procedure.
|
hardware/session acceptance procedure.
|
||||||
|
|
||||||
## Phase 5 — GNOME ARC surface
|
## Phase 5 — GNOME HUD surface
|
||||||
|
|
||||||
- [x] Add GNOME 45+ ESM extension metadata, panel indicator, and settings keys.
|
- [x] Add GNOME 45+ ESM extension metadata, panel indicator, and settings keys.
|
||||||
- [x] Add async D-Bus client and panel state glyphs.
|
- [x] Add async D-Bus client and panel state glyphs.
|
||||||
- [x] Add ARC overlay, waveform, transcript rows, chips, local/model status,
|
- [x] Add tray HUD: compact conversation menu, Chat/Thinking tabs, hold-to-talk,
|
||||||
|
optional expanded session, transcript rows, chips, local/model status,
|
||||||
reduced motion, high contrast, keyboard navigation, and screen-reader labels.
|
reduced motion, high contrast, keyboard navigation, and screen-reader labels.
|
||||||
- [x] Add listening halo, privacy slash, job theater, target highlights, and
|
- [x] Add listening/privacy chips, job theater, target highlights, and
|
||||||
visible computer-use cursor.
|
visible computer-use cursor.
|
||||||
- [x] Keep all extension work nonblocking and free of QVAC/native inference.
|
- [x] Keep all extension work nonblocking and free of QVAC/native inference.
|
||||||
|
- [x] Waveform HUD removed. `SpeakingLevel` / `ListeningLevel` are still
|
||||||
|
emitted on D-Bus; the extension no-ops them. There is no 48-bar overlay.
|
||||||
|
|
||||||
The extension implementation is in `apps/gnome-extension/[email protected]/`.
|
The extension implementation is in `apps/gnome-extension/[email protected]/`.
|
||||||
It owns only Shell UI and an asynchronous session-bus client; all model work
|
It owns only Shell UI and an asynchronous session-bus client; all model work
|
||||||
stays in `jarvisd`. ARC renders 48 waveform bars, transcript rows, mode and
|
stays in `jarvisd`. The HUD is a panel mark plus compact popup or expanded
|
||||||
suggestion chips, state glyphs, computer-use steps/highlights, and local/error
|
session. Computer-use chrome is a target highlight, not a waveform overlay.
|
||||||
status. The extension has keyboard focus, accessible names, configurable
|
The extension has keyboard focus, accessible names, configurable accent color,
|
||||||
accent color, and a reduced-motion-safe layout.
|
and a reduced-motion-safe layout.
|
||||||
|
|
||||||
Exit gate: enabling/disabling the extension never starts inference in the Shell
|
Exit gate: enabling/disabling the extension never starts inference in the Shell
|
||||||
process and reflects daemon state without jank.
|
process and reflects daemon state without jank.
|
||||||
|
|||||||
@@ -7,10 +7,12 @@ that ships with this repository.
|
|||||||
|
|
||||||
## Layout and runtime
|
## Layout and runtime
|
||||||
|
|
||||||
The harness is a standalone Node.js CommonJS package. Its public entrypoint is
|
The harness is a standalone CommonJS package. Its public entrypoint is
|
||||||
`index.js`, with the CLI at `bin/cli.js`. The README says Node 20 is enough for
|
`index.js`, with the CLI at `bin/cli.js`. The harness README says Node 20 is
|
||||||
the harness itself, while its QVAC inference dependency requires Node 22.17+;
|
enough for the package itself; QVAC inference on Node needs 22.17+. That
|
||||||
Jarvis therefore requires Node 22.17+.
|
Node requirement applies to tests and `node vendor/agent-harness/bin/cli.js`,
|
||||||
|
not to production `jarvisd`. Jarvis runs the copied harness inside packaged
|
||||||
|
Bare.
|
||||||
|
|
||||||
Important modules:
|
Important modules:
|
||||||
|
|
||||||
|
|||||||
+47
-6
@@ -14,18 +14,40 @@ The canonical XML contract is [io.qvac.Jarvis.Session.xml](../dbus/io.qvac.Jarvi
|
|||||||
| `PushToTalk(bool)` | Begin/end typed or microphone turn |
|
| `PushToTalk(bool)` | Begin/end typed or microphone turn |
|
||||||
| `Say(string)` | Speak text through local TTS |
|
| `Say(string)` | Speak text through local TTS |
|
||||||
| `Ask(string)` | Submit typed text to the harness |
|
| `Ask(string)` | Submit typed text to the harness |
|
||||||
|
| `ReloadSettings()` | Re-read `config.json` and return a status string |
|
||||||
|
| `PreviewVoice(string)` | Speak preview text without adding a chat turn |
|
||||||
|
| `StopSpeech()` | Interrupt TTS playback |
|
||||||
|
| `ResetContext()` | Clear the current conversation context |
|
||||||
|
| `Confirm(jobId, toolCallId, decision)` | Answer a confirmation gate |
|
||||||
| `Cancel()` | Cancel active request/job |
|
| `Cancel()` | Cancel active request/job |
|
||||||
| `SetMode(string)` | Set ARC capability mode |
|
| `SetMode(string)` | Set capability mode (not HUD layout) |
|
||||||
| `GetState()` | Return serialized daemon state |
|
| `GetState()` | Return serialized daemon state |
|
||||||
| `EnrollWake(string)` | Configure wake phrase |
|
| `EnrollWake(string)` | Configure wake phrase |
|
||||||
| `IngestPath(string)` | Add a local path to RAG ingestion |
|
| `IngestPath(string)` | Add a local path to RAG ingestion |
|
||||||
| `ComputerGrant(bool)` / `ComputerRevoke()` | Start/stop computer-use grant |
|
| `ComputerGrant(bool)` / `ComputerRevoke()` | Start/stop computer-use grant |
|
||||||
| `ComputerStatus()` | Return computer-use status JSON |
|
| `ComputerStatus()` | Return computer-use status JSON |
|
||||||
| `ShellListWindows()` / `ShellFocusWindow(uint64)` | Query/focus Shell windows |
|
| `GetRuntimeStatus()` | Return voice/runtime component status JSON |
|
||||||
|
| `AssessModelFit(string)` | Report whether a model fits the GPU profile |
|
||||||
|
| `DownloadModel(string)` | Start a model download |
|
||||||
|
| `CancelModel(string)` | Cancel a model download |
|
||||||
|
| `WipeComputerTraces()` | Wipe computer-use frames; returns whether wipe ran |
|
||||||
|
|
||||||
|
| Signal | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| `StateChanged` | Voice/daemon state string |
|
||||||
|
| `ContextReset` | Conversation context was cleared |
|
||||||
|
| `WakeHeard` | Wake phrase detected |
|
||||||
|
| `PartialTranscript` / `FinalTranscript` | ASR text |
|
||||||
|
| `Token` / `Reply` | Streamed and final assistant text |
|
||||||
|
| `SpeakingLevel` / `ListeningLevel` | Audio RMS. The HUD no-ops these; there is no waveform |
|
||||||
|
| `ChipOffered` | Suggestion sidecar |
|
||||||
|
| `JobProgress` | Background job progress |
|
||||||
|
| `ComputerStep` / `ComputerHighlight` | CU step text and target highlight rect |
|
||||||
|
| `ConfirmationRequired` | Tool permission prompt |
|
||||||
|
| `Thinking` | Planner thought tokens |
|
||||||
|
| `ToolCall` / `ToolResult` | Tool activity JSON |
|
||||||
|
| `Error` | Code and message |
|
||||||
|
|
||||||
Signals include `StateChanged`, `WakeHeard`, `PartialTranscript`,
|
|
||||||
`FinalTranscript`, `Token`, `Reply`, `SpeakingLevel`, `ListeningLevel`,
|
|
||||||
`ChipOffered`, `JobProgress`, `ComputerStep`, `ComputerHighlight`, and `Error`.
|
|
||||||
Payloads stay small; PCM and image bytes do not travel over D-Bus.
|
Payloads stay small; PCM and image bytes do not travel over D-Bus.
|
||||||
|
|
||||||
## Computer-use tools
|
## Computer-use tools
|
||||||
@@ -34,7 +56,26 @@ Tools are registered with the harness and exposed through the daemon. The
|
|||||||
observable contract includes `cu.grant`, `cu.revoke`, `cu.status`,
|
observable contract includes `cu.grant`, `cu.revoke`, `cu.status`,
|
||||||
`cu.observe`, `cu.zoom`, `cu.tree`, `cu.find`, `cu.click`, `cu.drag`,
|
`cu.observe`, `cu.zoom`, `cu.tree`, `cu.find`, `cu.click`, `cu.drag`,
|
||||||
`cu.scroll`, `cu.type`, `cu.key`, and `cu.act`. Every action is checked against
|
`cu.scroll`, `cu.type`, `cu.key`, and `cu.act`. Every action is checked against
|
||||||
the active grant and safety policy.
|
the active grant and safety policy. The HUD shows visible CU chrome (target
|
||||||
|
highlight and agent cursor), not a waveform overlay.
|
||||||
|
|
||||||
|
## Search tools
|
||||||
|
|
||||||
|
Implemented in `vendor/agent-harness/agent/web-search.js` for Bare (fetch plus
|
||||||
|
regex/JSON, no extra npm, no SearXNG):
|
||||||
|
|
||||||
|
| Tool | Behavior |
|
||||||
|
| --- | --- |
|
||||||
|
| `web_search` | Public search. Default `engine=auto` walks duckduckgo → ddg_lite → jina → bing → bing_rss → google → wikipedia → ddg_instant. Pin `engine` to retry one backend. |
|
||||||
|
| `google_search` | Google HTML first, then the auto chain |
|
||||||
|
| `fetch_page` | Readable text via Jina Reader, then HTML strip |
|
||||||
|
| `web_fetch` | Raw stripped text, including IP lookup pages (do not send those through Jina) |
|
||||||
|
| `wiki_search` | Wikipedia MediaWiki JSON |
|
||||||
|
| `hn_search` | Hacker News via Algolia |
|
||||||
|
| `code_search` | GitHub, npm, and MDN in parallel |
|
||||||
|
|
||||||
|
These are public reads and do not require extra API keys. Shell HTTP
|
||||||
|
(`curl`/`wget` over public hosts) remains blocked; use these tools instead.
|
||||||
|
|
||||||
## Local HTTP
|
## Local HTTP
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# System architecture
|
# System architecture
|
||||||
|
|
||||||
JARVIS-QVAC is split into a thin desktop surface, a user daemon, a cognitive
|
Jarvis is split into a thin desktop surface, a user daemon, a cognitive
|
||||||
harness, and isolated capability adapters. This keeps GNOME Shell responsive,
|
harness, and isolated capability adapters. This keeps GNOME Shell responsive,
|
||||||
prevents multiple model runtimes from competing for GPU memory, and makes
|
prevents multiple model runtimes from competing for GPU memory, and makes
|
||||||
computer use revocable as a session capability.
|
computer use revocable as a session capability.
|
||||||
@@ -8,14 +8,14 @@ computer use revocable as a session capability.
|
|||||||
```mermaid
|
```mermaid
|
||||||
flowchart TB
|
flowchart TB
|
||||||
subgraph Desktop[GNOME desktop session]
|
subgraph Desktop[GNOME desktop session]
|
||||||
EXT[Shell extension\nARC HUD + panel state]
|
EXT[Shell extension\ntray HUD + CU chrome]
|
||||||
CC[Control Center\nGTK4/libadwaita]
|
CC[Control Center\nGTK4/libadwaita]
|
||||||
PORTAL[XDG portals\nScreenCast + RemoteDesktop]
|
PORTAL[XDG portals\nScreenCast + RemoteDesktop]
|
||||||
ATSPI[AT-SPI2\naccessible tree]
|
ATSPI[AT-SPI2\naccessible tree]
|
||||||
PW[PipeWire\nmicrophone + playback]
|
PW[PipeWire\nmicrophone + playback]
|
||||||
end
|
end
|
||||||
subgraph Service[User services]
|
subgraph Service[User services]
|
||||||
D[ jarvisd\nNode 22 ESM ]
|
D[ jarvisd\npackaged Bare ]
|
||||||
CU[Computer-use session\nobserve + act + audit]
|
CU[Computer-use session\nobserve + act + audit]
|
||||||
end
|
end
|
||||||
subgraph Cognitive[Cognition]
|
subgraph Cognitive[Cognition]
|
||||||
@@ -41,13 +41,17 @@ flowchart TB
|
|||||||
|
|
||||||
| Process | Owns | Must not own |
|
| Process | Owns | Must not own |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| GNOME Shell extension | panel, ARC overlay, highlights, Shell facts, D-Bus client | model loading, microphone capture, input injection, blocking work |
|
| GNOME Shell extension | panel mark, tray popup, optional expanded session, CU target highlight, Shell facts, D-Bus client | model loading, microphone capture, input injection, blocking work |
|
||||||
| `jarvisd` | voice state, QVAC master, harness bridge, jobs, D-Bus service | arbitrary Shell evaluation or a second planner |
|
| `jarvisd` | voice state, QVAC master, harness bridge, jobs, D-Bus service | arbitrary Shell evaluation or a second planner |
|
||||||
| Control Center | settings, model fit, permissions | inference loop or hidden data deletion |
|
| Control Center | settings, model fit, permissions | inference loop or hidden data deletion |
|
||||||
| Computer-use helpers | portal sessions, frames, AT-SPI, EIS/legacy backend | model planning or lock-screen bypass |
|
| Computer-use helpers | portal sessions, frames, AT-SPI, EIS/legacy backend | model planning or lock-screen bypass |
|
||||||
| Agent harness | session history, planning, memory, tool loop, permissions | direct QVAC ownership or direct desktop hacks |
|
| Agent harness | session history, planning, memory, tool loop, permissions | direct QVAC ownership or direct desktop hacks |
|
||||||
| QVAC master | SDK worker, GPU admission, lifecycle, serialization, cancellation | cloud APIs or independent model instances |
|
| QVAC master | SDK worker, GPU admission, lifecycle, serialization, cancellation | cloud APIs or independent model instances |
|
||||||
|
|
||||||
|
Production `jarvisd` is the packaged Bare process (`daemon/bare-entry.js` via
|
||||||
|
`packaging/jarvisd.service`). Node 22 is for tests, packaging, and the harness
|
||||||
|
CLI, not the live daemon.
|
||||||
|
|
||||||
## Data and control boundaries
|
## Data and control boundaries
|
||||||
|
|
||||||
Control messages and small state changes use session D-Bus. Audio streams and
|
Control messages and small state changes use session D-Bus. Audio streams and
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
# Brand
|
||||||
|
|
||||||
|
Jarvis is the product name. Jarvis QVAC is the product line. HoneyPeer, LLC
|
||||||
|
holds copyright. Gold is presence; cyan is thought.
|
||||||
|
|
||||||
|
## Tokens
|
||||||
|
|
||||||
|
Source of truth: `apps/gnome-extension/[email protected]/brand/tokens.json` and
|
||||||
|
`apps/gnome-extension/[email protected]/brand.js`.
|
||||||
|
|
||||||
|
| Token | Hex | Use |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Gold | `#F4B942` | Default accent, talk button, user rows, idle panel name |
|
||||||
|
| Gold bright | `#FFE09A` | Talk hover |
|
||||||
|
| Signal | `#4FD2FF` | Thinking, Jarvis reply rows, computer-use chrome |
|
||||||
|
| Ink | `#0B0E14` | HUD background |
|
||||||
|
| Paper | `#F6F7FB` | Primary HUD text |
|
||||||
|
|
||||||
|
Accent presets in Settings: Gold, Signal `#4FD2FF`, Copper `#E07A3D`, Jade
|
||||||
|
`#3DDC97`, Rose `#F071A8`. A custom `#RRGGBB` value is accepted. Invalid hex
|
||||||
|
keeps Gold.
|
||||||
|
|
||||||
|
## Assets
|
||||||
|
|
||||||
|
All live under `apps/gnome-extension/[email protected]/brand/`:
|
||||||
|
|
||||||
|
- `icons/jarvis-mark.svg` — visor-J mark (gold stroke, cyan core)
|
||||||
|
- `icons/jarvis-mark-symbolic.svg` — `currentColor` symbolic
|
||||||
|
- `icons/jarvis-wordmark.svg` — mark + JARVIS letters
|
||||||
|
- `icons/io.qvac.Jarvis.Control.svg` — app icon
|
||||||
|
- `gtk.css` — Control Center / preferences CSS
|
||||||
|
|
||||||
|
Install copies the app icon to
|
||||||
|
`~/.local/share/icons/hicolor/scalable/apps/io.qvac.Jarvis.Control.svg` and
|
||||||
|
writes `~/.local/share/applications/io.qvac.Jarvis.Control.desktop`.
|
||||||
|
|
||||||
|
## HUD rules
|
||||||
|
|
||||||
|
The GNOME Shell stylesheet is St CSS, not GTK4 CSS.
|
||||||
|
|
||||||
|
- Do not put `--custom-properties` inside `/* */` comments.
|
||||||
|
- Do not declare extra `--var: value` blocks in the stylesheet; `--jarvis-accent`
|
||||||
|
is set from GSettings in JS.
|
||||||
|
- Do not use `alpha(var(--jarvis-accent), …)` or `icon-size`.
|
||||||
|
- Panel: mark plus **Jarvis** when idle; mark plus a state chip
|
||||||
|
(Listening / Thinking / Speaking / Privacy) when busy. Never both labels.
|
||||||
|
- Popup status uses `jarvis-local` only. Do not put `jarvis-state-*` on HUD
|
||||||
|
labels; those classes are for `.jarvis-panel-state`.
|
||||||
@@ -15,16 +15,18 @@ that is not present.
|
|||||||
| Media | `imagine`, `edit-image`, `make-video`, `compose` | Job runner + QVAC master | write |
|
| Media | `imagine`, `edit-image`, `make-video`, `compose` | Job runner + QVAC master | write |
|
||||||
| Training | `teach-me` | Isolated job + QVAC master | dangerous |
|
| Training | `teach-me` | Isolated job + QVAC master | dangerous |
|
||||||
| Lab | `bci`, `robot`, `world` | Explicit lab adapters | read/write/dangerous |
|
| Lab | `bci`, `robot`, `world` | Explicit lab adapters | read/write/dangerous |
|
||||||
|
| Search | `web_search`, `google_search`, `fetch_page`, `web_fetch`, `wiki_search`, `hn_search`, `code_search` | Bare `web-search.js` | read (no extra keys) |
|
||||||
| Desktop | launch, focus, window, clipboard, media, settings | GNOME/GIO/D-Bus | read/write |
|
| Desktop | launch, focus, window, clipboard, media, settings | GNOME/GIO/D-Bus | read/write |
|
||||||
| Computer use | `cu.observe`, `cu.find`, `cu.act`, `cu.click`, `cu.type`, `cu.key` | CU session | computer-use |
|
| Computer use | `cu.observe`, `cu.find`, `cu.act`, `cu.click`, `cu.type`, `cu.key` | CU session | computer-use |
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart TB
|
flowchart TB
|
||||||
J((JARVIS-QVAC))
|
J((Jarvis))
|
||||||
J --> C[Conversation\nchat · plan · code · summarize]
|
J --> C[Conversation\nchat · plan · code · summarize]
|
||||||
J --> M[Memory\nembeddings · RAG · batch]
|
J --> M[Memory\nembeddings · RAG · batch]
|
||||||
J --> P[Perception\nscreenshot · OCR · classification]
|
J --> P[Perception\nscreenshot · OCR · classification]
|
||||||
J --> V[Voice\nwake · ASR · TTS · translation]
|
J --> V[Voice\nwake · ASR · TTS · translation]
|
||||||
|
J --> W[Search\nweb_search · fetch_page · wiki]
|
||||||
J --> G[Media\nimage · video · music]
|
J --> G[Media\nimage · video · music]
|
||||||
J --> D[Desktop\nGNOME · AT-SPI · computer use]
|
J --> D[Desktop\nGNOME · AT-SPI · computer use]
|
||||||
J --> L[Lab\nLoRA · BCI · VLA · ABot-World]
|
J --> L[Lab\nLoRA · BCI · VLA · ABot-World]
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
# Computer use
|
# Computer use
|
||||||
|
|
||||||
Computer use is a revocable mode that combines observation, semantic actions,
|
Computer use is a revocable mode that combines observation, semantic actions,
|
||||||
portal input, and a visible ARC overlay. It is never implicitly enabled by a
|
portal input, and visible CU chrome (target highlight and agent cursor). It is never implicitly enabled by a
|
||||||
normal chat turn.
|
normal chat turn.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
|
|||||||
+7
-3
@@ -3,19 +3,23 @@
|
|||||||
## Repository map
|
## Repository map
|
||||||
|
|
||||||
```text
|
```text
|
||||||
daemon/ Node daemon, QVAC master, voice, D-Bus
|
daemon/ Bare daemon, QVAC master, voice, D-Bus
|
||||||
computer-use/ observe/act session and host helpers
|
computer-use/ observe/act session and host helpers
|
||||||
skills/ harness tool adapters
|
skills/ harness tool adapters
|
||||||
apps/gnome-extension/ GNOME Shell ESM UI and Shell bridge
|
apps/gnome-extension/ GNOME Shell ESM UI, HUD, and brand/
|
||||||
apps/control-center/ GTK4/libadwaita settings application
|
apps/control-center/ GTK4/libadwaita settings application
|
||||||
vendor/agent-harness/ copied cognitive core
|
vendor/agent-harness/ copied cognitive core
|
||||||
dbus/ introspection XML
|
dbus/ introspection XML
|
||||||
packaging/ installer, service, artifacts, release API
|
packaging/ installer, Bare launcher, artifacts
|
||||||
systemd/user/ checkout development service
|
systemd/user/ checkout development service
|
||||||
docs/ maintained technical documentation
|
docs/ maintained technical documentation
|
||||||
test/ Node acceptance/unit fixtures
|
test/ Node acceptance/unit fixtures
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Brand tokens and SVG marks live under
|
||||||
|
`apps/gnome-extension/[email protected]/brand/`. `test/brand.test.js` and
|
||||||
|
`test/license.test.js` cover palette, AGPL/HoneyPeer, and St-safe HUD CSS.
|
||||||
|
|
||||||
## Change workflow
|
## Change workflow
|
||||||
|
|
||||||
1. Read the relevant architecture and safety contract.
|
1. Read the relevant architecture and safety contract.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Hardware compatibility
|
# Hardware compatibility
|
||||||
|
|
||||||
JARVIS-QVAC has one inference owner: `jarvisd` owns the QVAC master and every
|
Jarvis has one inference owner: `jarvisd` owns the QVAC master and every
|
||||||
model call goes through `daemon/qvac-master.js`. The harness, media jobs, and
|
model call goes through `daemon/qvac-master.js`. The harness, media jobs, and
|
||||||
computer-use tools do not start competing QVAC servers. The GPU gate is strict:
|
computer-use tools do not start competing QVAC servers. The GPU gate is strict:
|
||||||
when QVAC cannot report a usable GPU backend, inference fails with a truthful
|
when QVAC cannot report a usable GPU backend, inference fails with a truthful
|
||||||
|
|||||||
@@ -35,7 +35,10 @@ D-Bus protocol. Computer-use tools are custom harness tools backed by
|
|||||||
|
|
||||||
## Adding a tool
|
## Adding a tool
|
||||||
|
|
||||||
1. Put the implementation in the appropriate skill module.
|
1. Put the implementation in the appropriate skill module. Public web search
|
||||||
|
and page fetch already live in
|
||||||
|
`vendor/agent-harness/agent/web-search.js` (`web_search`, `google_search`,
|
||||||
|
`fetch_page`, `web_fetch`, `wiki_search`, `hn_search`, `code_search`).
|
||||||
2. Export a JSON-schema tool with a stable name, description, timeout, and
|
2. Export a JSON-schema tool with a stable name, description, timeout, and
|
||||||
permission level.
|
permission level.
|
||||||
3. Route QVAC work through the master facade.
|
3. Route QVAC work through the master facade.
|
||||||
@@ -44,5 +47,5 @@ D-Bus protocol. Computer-use tools are custom harness tools backed by
|
|||||||
user-visible.
|
user-visible.
|
||||||
|
|
||||||
Destructive tools require explicit confirmation. Computer-use tools additionally
|
Destructive tools require explicit confirmation. Computer-use tools additionally
|
||||||
require an active grant, a visible overlay, a live step budget, and a lock-screen
|
require an active grant, visible CU chrome / target highlight, a live step
|
||||||
check.
|
budget, and a lock-screen check.
|
||||||
|
|||||||
@@ -18,6 +18,11 @@ systemctl --user status jarvisd.service
|
|||||||
gnome-extensions enable [email protected]
|
gnome-extensions enable [email protected]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Open the tray HUD (panel mark). Type a prompt or hold Talk. Chat and Thinking
|
||||||
|
are tabs in the compact menu; expand the session from the header if you want a
|
||||||
|
larger conversation. Open **Settings** from the HUD header (or the **Jarvis**
|
||||||
|
launcher). There is no waveform overlay.
|
||||||
|
|
||||||
Complete the typed prompt from the tray, then say “Hey Jarvis, what’s on my
|
Complete the typed prompt from the tray, then say “Hey Jarvis, what’s on my
|
||||||
screen?” and approve a computer-use grant only for the acceptance actions in
|
screen?” and approve a computer-use grant only for the acceptance actions in
|
||||||
`docs/cu-acceptance.md`. If the doctor reports no GPU, stop at diagnostics and
|
`docs/cu-acceptance.md`. If the doctor reports no GPU, stop at diagnostics and
|
||||||
|
|||||||
+42
-6
@@ -30,9 +30,13 @@ npm test
|
|||||||
npm run gpu-doctor
|
npm run gpu-doctor
|
||||||
npm run cu-doctor
|
npm run cu-doctor
|
||||||
npm run voice-doctor
|
npm run voice-doctor
|
||||||
bare daemon/bare-entry.js
|
bash packaging/bare-launch.sh daemon/bare-entry.js
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`packaging/bare-launch.sh` selects the vendored Bare binary and is the same
|
||||||
|
wrapper the packaged service uses. Do not reload the daemon with
|
||||||
|
`node --watch`; Bare is the production loop.
|
||||||
|
|
||||||
Use `npm run qvac:serve:diagnostic` only when a separate local client needs the
|
Use `npm run qvac:serve:diagnostic` only when a separate local client needs the
|
||||||
OpenAI-compatible endpoint. Keep its host at `127.0.0.1`.
|
OpenAI-compatible endpoint. Keep its host at `127.0.0.1`.
|
||||||
|
|
||||||
@@ -45,9 +49,28 @@ systemctl --user status jarvisd.service
|
|||||||
journalctl --user -u jarvisd.service -f
|
journalctl --user -u jarvisd.service -f
|
||||||
```
|
```
|
||||||
|
|
||||||
The packaged service uses `~/.local/share/jarvis-qvac` and the generated config
|
The unit at `packaging/jarvisd.service` (installed to
|
||||||
at `~/.config/jarvis/qvac.config.json`. The repository development unit uses
|
`~/.config/systemd/user/jarvisd.service`) starts Bare on
|
||||||
the checkout path; use the packaged unit when testing installation behavior.
|
`daemon/bare-entry.js` with working directory `~/.local/share/jarvis-qvac` and
|
||||||
|
`QVAC_CONFIG_PATH=~/.config/jarvis/qvac.config.json`. The repository
|
||||||
|
development unit uses the checkout path; use the packaged unit when testing
|
||||||
|
installation behavior.
|
||||||
|
|
||||||
|
Restart after daemon code changes:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl --user restart jarvisd.service
|
||||||
|
```
|
||||||
|
|
||||||
|
## Desktop launcher
|
||||||
|
|
||||||
|
`packaging/install.sh` writes:
|
||||||
|
|
||||||
|
- `~/.local/share/applications/io.qvac.Jarvis.Control.desktop`
|
||||||
|
- `~/.local/share/icons/hicolor/scalable/apps/io.qvac.Jarvis.Control.svg`
|
||||||
|
|
||||||
|
The **Jarvis** launcher opens Control Center (`gjs -m …/apps/control-center/main.js`).
|
||||||
|
See [brand](brand.md) and [settings](settings.md).
|
||||||
|
|
||||||
## Extension development
|
## Extension development
|
||||||
|
|
||||||
@@ -56,8 +79,21 @@ gnome-extensions enable [email protected]
|
|||||||
dbus-run-session -- gnome-shell --nested --wayland
|
dbus-run-session -- gnome-shell --nested --wayland
|
||||||
```
|
```
|
||||||
|
|
||||||
Keep inference and long-running work in `jarvisd`. Reloading the Shell is only
|
Keep inference and long-running work in `jarvisd`. Live HUD files are
|
||||||
needed for extension changes; daemon changes can use `node --watch`.
|
`~/.local/share/gnome-shell/extensions/[email protected]`. Copy the checkout
|
||||||
|
tree there, then reload the extension.
|
||||||
|
|
||||||
|
GNOME 50 does not implement `ReloadExtension`. Calling
|
||||||
|
`org.gnome.Shell.Extensions.ReloadExtension` on the standalone Extensions
|
||||||
|
helper can also hit a stale cache. After copying files:
|
||||||
|
|
||||||
|
1. Disable then enable on the real Shell (`org.gnome.Shell` at
|
||||||
|
`/org/gnome/Shell`), or
|
||||||
|
2. Log out and back in.
|
||||||
|
|
||||||
|
A stylesheet or JS parse error (for example St CSS `errcode:15`) will not
|
||||||
|
hot-reload until that disable/enable or a new session. See
|
||||||
|
[troubleshooting](troubleshooting.md).
|
||||||
|
|
||||||
## State and storage
|
## State and storage
|
||||||
|
|
||||||
|
|||||||
@@ -2,9 +2,11 @@
|
|||||||
|
|
||||||
The default posture is local inference, explicit writes, and fail-closed
|
The default posture is local inference, explicit writes, and fail-closed
|
||||||
computer use. No telemetry or cloud inference is required. Model endpoints
|
computer use. No telemetry or cloud inference is required. Model endpoints
|
||||||
stay on localhost. Public `web_search` and `web_fetch` are allowed by default
|
stay on localhost. Public `web_search`, `google_search`, `fetch_page`,
|
||||||
without a confirmation prompt. Shell commands that open public HTTP (curl,
|
`web_fetch`, `wiki_search`, `hn_search`, and `code_search` are allowed by
|
||||||
wget) remain blocked by the runtime; use the web tools instead.
|
default without a confirmation prompt and without extra API keys. There is no
|
||||||
|
SearXNG dependency. Shell commands that open public HTTP (curl, wget) remain
|
||||||
|
blocked by the runtime; use the web tools instead.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart TD
|
flowchart TD
|
||||||
@@ -15,7 +17,7 @@ flowchart TD
|
|||||||
C -->|no| B[Block and explain]
|
C -->|no| B[Block and explain]
|
||||||
X --> A[Audit metadata]
|
X --> A[Audit metadata]
|
||||||
X --> D[Return result]
|
X --> D[Return result]
|
||||||
L[Lock screen] --> K[Revoke CU + mute overlay]
|
L[Lock screen] --> K[Revoke CU + mute HUD]
|
||||||
```
|
```
|
||||||
|
|
||||||
## Protected assets
|
## Protected assets
|
||||||
|
|||||||
+9
-4
@@ -1,8 +1,11 @@
|
|||||||
# Settings and voice customization
|
# Settings and voice customization
|
||||||
|
|
||||||
Open **Settings** in the Jarvis tray or conversation header. The standalone
|
Open **Settings** in the Jarvis tray or conversation header. The window banner
|
||||||
Control Center (`python3 apps/control-center/main.py`) uses the same pages.
|
shows the Jarvis mark, **Jarvis QVAC**, and **Local voice for your desktop ·
|
||||||
Search is available in the window header.
|
HoneyPeer, LLC**. The standalone Control Center
|
||||||
|
(`python3 apps/control-center/main.py`, `gjs -m apps/control-center/main.js`,
|
||||||
|
or the installed **Jarvis** launcher) uses the same pages. Search is available
|
||||||
|
in the window header.
|
||||||
|
|
||||||
## Choosing a voice
|
## Choosing a voice
|
||||||
|
|
||||||
@@ -158,7 +161,9 @@ and WebP quality tune observation detail and processing cost.
|
|||||||
The Desktop page also exposes the GNOME hotkey, accent color, and tray versus
|
The Desktop page also exposes the GNOME hotkey, accent color, and tray versus
|
||||||
expanded layout. These retain their GSettings keys: `hotkey`, `accent-color`,
|
expanded layout. These retain their GSettings keys: `hotkey`, `accent-color`,
|
||||||
and `overlay-style`. Gold `#F4B942` is the brand default. Signal, Copper, Jade,
|
and `overlay-style`. Gold `#F4B942` is the brand default. Signal, Copper, Jade,
|
||||||
and Rose are presets; a custom hex value is also accepted.
|
and Rose are circular swatches; a custom hex field is also accepted. The hex
|
||||||
|
field uses the GTK 4 entry buffer (`set_max_length`), not a `maxLength`
|
||||||
|
constructor property. See [brand](brand.md) for tokens and HUD rules.
|
||||||
|
|
||||||
Previously displayed controls for chimes, legacy input fallback, enrollment,
|
Previously displayed controls for chimes, legacy input fallback, enrollment,
|
||||||
startup, memory ingestion, Lab toggles, transcript retention, and disabling
|
startup, memory ingestion, Lab toggles, transcript retention, and disabling
|
||||||
|
|||||||
+6
-1
@@ -12,7 +12,12 @@ git diff --check
|
|||||||
|
|
||||||
The Node suite covers daemon infrastructure, QVAC ownership, network policy,
|
The Node suite covers daemon infrastructure, QVAC ownership, network policy,
|
||||||
observer behavior, computer-use safety, runtime tools, and voice behavior. The
|
observer behavior, computer-use safety, runtime tools, and voice behavior. The
|
||||||
vendored harness test is included in the root test command.
|
vendored harness test is included in the root test command. `test/brand.test.js`
|
||||||
|
checks tokens, SVG marks, HoneyPeer/AGPL fields, and St-safe HUD CSS (no
|
||||||
|
`/* */`, no `alpha(`, no `icon-size`). `test/license.test.js` checks the AGPL
|
||||||
|
license files. `test/gnome-extension.test.js` covers tray HUD behavior
|
||||||
|
(Chat/Thinking tabs, hold-to-talk, expanded session, panel name versus state
|
||||||
|
chip).
|
||||||
|
|
||||||
## Host acceptance
|
## Host acceptance
|
||||||
|
|
||||||
|
|||||||
+32
-10
@@ -24,12 +24,33 @@ still fails, run `npm run cu-doctor` and confirm `libei` is installed. Observati
|
|||||||
does not need the injector: after a grant, `cu_observe` should return a screenshot
|
does not need the injector: after a grant, `cu_observe` should return a screenshot
|
||||||
and accessibility tree even when pointer injection is down.
|
and accessibility tree even when pointer injection is down.
|
||||||
|
|
||||||
|
## Stylesheet parse `errcode:15`
|
||||||
|
|
||||||
|
GNOME Shell's St CSS parser rejects comments that contain `--`, `alpha(var())`,
|
||||||
|
and `icon-size`. If `journalctl` shows a stylesheet parse error on
|
||||||
|
`[email protected]`, copy the current `stylesheet.css` into the live extension
|
||||||
|
directory and disable/enable the extension (GNOME 50 cannot `ReloadExtension`).
|
||||||
|
See [brand](brand.md).
|
||||||
|
|
||||||
|
## Settings window crashes on accent hex
|
||||||
|
|
||||||
|
GTK 4 `Gtk.Entry` has no `maxLength` constructor property. The hex field uses
|
||||||
|
`entry.get_buffer().set_max_length(7)`. If an older checkout still constructs
|
||||||
|
`maxlength`, Settings will abort when opening the Desktop page.
|
||||||
|
|
||||||
|
## Panel label shows “Jarv…”
|
||||||
|
|
||||||
|
The panel used to show the product name and a state chip at the same time,
|
||||||
|
which GNOME truncated. Idle is mark plus **Jarvis**; busy is mark plus one
|
||||||
|
state chip (Listening / Thinking / Speaking / Privacy). Never both.
|
||||||
|
|
||||||
## The extension is missing
|
## The extension is missing
|
||||||
|
|
||||||
Preferences can open while the panel HUD is gone if the extension is installed
|
Preferences can open while the panel HUD is gone if the extension is installed
|
||||||
but not enabled in the running GNOME Shell. Re-run the installer so it copies
|
but not enabled in the running GNOME Shell. Re-run the installer so it copies
|
||||||
the HUD files and asks Shell to reload/enable `[email protected]`. The
|
the HUD files and asks Shell to enable `[email protected]`. On GNOME 50 that
|
||||||
installer also keeps a copy at
|
does not hot-reload; disable/enable the extension or log out if the panel is
|
||||||
|
still gone. The installer also keeps a copy at
|
||||||
`~/.local/share/gnome-shell/extensions/[email protected]`:
|
`~/.local/share/gnome-shell/extensions/[email protected]`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -46,22 +67,23 @@ journalctl --user -b | grep -i jarvis
|
|||||||
If the extension still does not appear, log out and back in once so GNOME Shell
|
If the extension still does not appear, log out and back in once so GNOME Shell
|
||||||
reloads its extension search path.
|
reloads its extension search path.
|
||||||
|
|
||||||
## The service reports `203/EXEC` for `/usr/bin/node`
|
## The service reports `203/EXEC`
|
||||||
|
|
||||||
Older installers wrote a fixed `/usr/bin/node` path. Re-run the current web
|
Current `jarvisd.service` starts packaged Bare, not `/usr/bin/node`. If
|
||||||
installer; it records the Node executable found in the installing user's PATH
|
`ExecStart` still points at a missing `node` or Bare binary, re-run
|
||||||
when it writes the user service. Check the result with:
|
`packaging/install.sh --enable` or the web installer. Check the result with:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
systemctl --user cat jarvisd.service | grep ExecStart
|
systemctl --user cat jarvisd.service | grep ExecStart
|
||||||
command -v node
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## The daemon restarts
|
## The daemon restarts
|
||||||
|
|
||||||
Inspect `journalctl --user -u jarvisd.service`. Common causes are Node version,
|
Inspect `journalctl --user -u jarvisd.service`. Common causes are a missing
|
||||||
invalid QVAC config, missing session D-Bus, missing GPU visibility, or a stale
|
Bare binary, invalid QVAC config, missing session D-Bus, missing GPU
|
||||||
portal grant. Run the three doctor commands before changing model settings.
|
visibility, or a stale portal grant. Run the three doctor commands before
|
||||||
|
changing model settings. Restarting `jarvisd` also clears an active desktop
|
||||||
|
grant.
|
||||||
|
|
||||||
## Rolling release did not publish
|
## Rolling release did not publish
|
||||||
|
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ question. Idle sleep unloads them again so GPU memory is freed; only PipeWire
|
|||||||
capture and the CPU wake detector stay resident. Runtime status includes
|
capture and the CPU wake detector stay resident. Runtime status includes
|
||||||
separate `asr`, `tts`, `capture`, and `wake` flags plus per-component errors.
|
separate `asr`, `tts`, `capture`, and `wake` flags plus per-component errors.
|
||||||
The tray shows **WAKE ON** when the detector is alive. Hold Talk (mouse, Space,
|
The tray shows **WAKE ON** when the detector is alive. Hold Talk (mouse, Space,
|
||||||
or Enter) still works in the GNOME overlay.
|
or Enter) still works in the tray HUD.
|
||||||
|
|
||||||
The Control Center's wake command and spoken-reply setting are read from
|
The Control Center's wake command and spoken-reply setting are read from
|
||||||
`~/.config/jarvis/config.json` on daemon startup. Restart `jarvisd.service` after
|
`~/.config/jarvis/config.json` on daemon startup. Restart `jarvisd.service` after
|
||||||
|
|||||||
+5
-2
@@ -59,9 +59,11 @@ test('brand icons ship as SVG marks with gold visor and cyan core', () => {
|
|||||||
|
|
||||||
test('HUD, settings, and launcher consume the brand package', () => {
|
test('HUD, settings, and launcher consume the brand package', () => {
|
||||||
assert.match(stylesheet, /--jarvis-accent/);
|
assert.match(stylesheet, /--jarvis-accent/);
|
||||||
assert.match(stylesheet, /--jarvis-signal: #4FD2FF/);
|
assert.doesNotMatch(stylesheet, /alpha\(/);
|
||||||
|
assert.doesNotMatch(stylesheet, /icon-size:/);
|
||||||
|
assert.doesNotMatch(stylesheet, /\/\*/);
|
||||||
assert.match(stylesheet, /\.jarvis-panel-mark/);
|
assert.match(stylesheet, /\.jarvis-panel-mark/);
|
||||||
assert.match(stylesheet, /\.jarvis-talk \{ background-color: var\(--jarvis-accent/);
|
assert.match(stylesheet, /\.jarvis-talk \{ background-color: #F4B942/);
|
||||||
assert.match(gtkCss, /@define-color jarvis_gold #F4B942/);
|
assert.match(gtkCss, /@define-color jarvis_gold #F4B942/);
|
||||||
assert.match(gtkCss, /button\.jarvis-swatch-gold/);
|
assert.match(gtkCss, /button\.jarvis-swatch-gold/);
|
||||||
assert.equal(metadata.name, 'Jarvis');
|
assert.equal(metadata.name, 'Jarvis');
|
||||||
@@ -70,6 +72,7 @@ test('HUD, settings, and launcher consume the brand package', () => {
|
|||||||
assert.match(settingsWindow, /brandBanner/);
|
assert.match(settingsWindow, /brandBanner/);
|
||||||
assert.match(settingsWindow, /BRAND\.publisher/);
|
assert.match(settingsWindow, /BRAND\.publisher/);
|
||||||
assert.match(settingsWindow, /accentRow/);
|
assert.match(settingsWindow, /accentRow/);
|
||||||
|
assert.doesNotMatch(settingsWindow, /maxlength/);
|
||||||
assert.match(extensionSource, /brand\/icons\/jarvis-mark\.svg/);
|
assert.match(extensionSource, /brand\/icons\/jarvis-mark\.svg/);
|
||||||
assert.match(extensionSource, /text: 'Jarvis'/);
|
assert.match(extensionSource, /text: 'Jarvis'/);
|
||||||
assert.match(install, /io\.qvac\.Jarvis\.Control\.desktop/);
|
assert.match(install, /io\.qvac\.Jarvis\.Control\.desktop/);
|
||||||
|
|||||||
@@ -233,7 +233,9 @@ test('GNOME GI imports expose default namespaces and panel has a real menu', ()
|
|||||||
assert.match(extensionSource, /_openPopup/);
|
assert.match(extensionSource, /_openPopup/);
|
||||||
assert.match(uiSource, /Open conversation/);
|
assert.match(uiSource, /Open conversation/);
|
||||||
assert.match(extensionSource, /PopupMenuSection/);
|
assert.match(extensionSource, /PopupMenuSection/);
|
||||||
assert.match(extensionSource, /osd\.attach\(this\._panelBox\)/);
|
assert.match(extensionSource, /import Pango from 'gi:\/\/Pango'/);
|
||||||
|
assert.match(extensionSource, /ellipsize = Pango\.EllipsizeMode\.NONE/);
|
||||||
|
assert.match(extensionSource, /osd\.attach\(this\._panelBox, this\._glyph\)/);
|
||||||
assert.match(extensionSource, /jarvis-panel-mark/);
|
assert.match(extensionSource, /jarvis-panel-mark/);
|
||||||
assert.match(extensionSource, /brand\/icons\/jarvis-mark\.svg/);
|
assert.match(extensionSource, /brand\/icons\/jarvis-mark\.svg/);
|
||||||
assert.match(uiSource, /jarvis-panel-state/);
|
assert.match(uiSource, /jarvis-panel-state/);
|
||||||
@@ -274,12 +276,14 @@ test('lazy ASR still shows the microphone as available when capture is up', () =
|
|||||||
const { ConversationView } = harness();
|
const { ConversationView } = harness();
|
||||||
const popup = new ConversationView({ compact: true });
|
const popup = new ConversationView({ compact: true });
|
||||||
popup.setVoiceStatus({ tts: true, input: true, wake: true });
|
popup.setVoiceStatus({ tts: true, input: true, wake: true });
|
||||||
assert.match(popup.status.text, /SPEECH ON/);
|
assert.equal(popup.status.text, 'Local');
|
||||||
assert.match(popup.status.text, /WAKE ON/);
|
assert.match(popup.statusLine.text, /speech on/);
|
||||||
assert.doesNotMatch(popup.status.text, /MIC UNAVAILABLE/);
|
assert.match(popup.statusLine.text, /wake on/);
|
||||||
|
assert.doesNotMatch(popup.status.style_class, /jarvis-state-/);
|
||||||
assert.equal(popup.talk.label, 'Hold to talk');
|
assert.equal(popup.talk.label, 'Hold to talk');
|
||||||
popup.setVoiceStatus({ tts: false, input: false, wake: false });
|
popup.setVoiceStatus({ tts: false, input: false, wake: false });
|
||||||
assert.match(popup.status.text, /MIC UNAVAILABLE/);
|
assert.match(popup.status.text, /Mic off/);
|
||||||
|
assert.match(popup.statusLine.text, /mic off/);
|
||||||
assert.equal(popup.talk.label, 'Mic unavailable');
|
assert.equal(popup.talk.label, 'Mic unavailable');
|
||||||
assert.match(extensionSource, /Boolean\(voice\.capture\)/);
|
assert.match(extensionSource, /Boolean\(voice\.capture\)/);
|
||||||
assert.doesNotMatch(extensionSource, /voice\.asr && voice\.capture/);
|
assert.doesNotMatch(extensionSource, /voice\.asr && voice\.capture/);
|
||||||
@@ -289,22 +293,27 @@ test('voice status lives in the panel chip, not a floating overlay', () => {
|
|||||||
const { JarvisOsd, SessionPanel, timers, chrome } = harness();
|
const { JarvisOsd, SessionPanel, timers, chrome } = harness();
|
||||||
const osd = new JarvisOsd();
|
const osd = new JarvisOsd();
|
||||||
const session = new SessionPanel();
|
const session = new SessionPanel();
|
||||||
|
const name = { visible: true };
|
||||||
const bar = { children: [], add_child(child) { this.children.push(child); } };
|
const bar = { children: [], add_child(child) { this.children.push(child); } };
|
||||||
osd.attach(bar);
|
osd.attach(bar, name);
|
||||||
session.attach();
|
session.attach();
|
||||||
assert.equal(bar.children[0], osd.root);
|
assert.equal(bar.children[0], osd.root);
|
||||||
osd.setState('LISTENING');
|
osd.setState('LISTENING');
|
||||||
assert.equal(osd.root.visible, true);
|
assert.equal(osd.root.visible, true);
|
||||||
|
assert.equal(name.visible, false);
|
||||||
assert.match(osd.label.text, /Listening/);
|
assert.match(osd.label.text, /Listening/);
|
||||||
assert.equal(session.root.visible, false);
|
assert.equal(session.root.visible, false);
|
||||||
osd.setState('ARMED');
|
osd.setState('ARMED');
|
||||||
assert.equal(osd.root.visible, false);
|
assert.equal(osd.root.visible, false);
|
||||||
|
assert.equal(name.visible, true);
|
||||||
osd.setState('SPEAKING');
|
osd.setState('SPEAKING');
|
||||||
assert.equal(osd.root.visible, true);
|
assert.equal(osd.root.visible, true);
|
||||||
|
assert.equal(name.visible, false);
|
||||||
assert.match(osd.label.text, /Speaking/);
|
assert.match(osd.label.text, /Speaking/);
|
||||||
assert.equal(timers.size, 0);
|
assert.equal(timers.size, 0);
|
||||||
osd.setState('THINKING');
|
osd.setState('THINKING');
|
||||||
assert.match(osd.label.text, /Thinking/);
|
assert.match(osd.label.text, /Thinking/);
|
||||||
|
assert.equal(name.visible, false);
|
||||||
osd.setState('SLEEPING');
|
osd.setState('SLEEPING');
|
||||||
assert.match(osd.label.text, /Privacy/);
|
assert.match(osd.label.text, /Privacy/);
|
||||||
osd.showWake();
|
osd.showWake();
|
||||||
|
|||||||
Reference in New Issue
Block a user