# Agent (grok-class harness)

`BridgeSwarm.agent` is a Bare JS agent loop modeled on Grok Build: sample → tool calls → results → repeat. Inference is QVAC (`tools: true`). Multimodal models can **see** attached stills and a live desktop ring; image/video **generation** and click/keyboard computer-use tools are not implemented. The agent requires **Settings → Enable QVAC**; `create` / `prompt` fail until that toggle is on.

## Embed

```js
const session = await BridgeSwarm.agent.create({
  model: 'qwen3.5-4b',
  cwd: 'default',
  permissionMode: 'ask' // ask | allowlist | always-approve
})
session.on((ev) => { /* agent_message_chunk | tool_call | permission | ask_user | plan_approval | end */ })
await session.prompt('List the workspace and summarize', {
  planMode: false,
  permissionMode: 'ask',
})
await session.prompt('What is in this screenshot?', {
  images: [{ dataUrl: canvas.toDataURL('image/png') }],
})
await session.cancel()
await session.dispose()
```

Always-approve is only honored for origins listed under **Settings → Agent always-approve**.

Multimodal catalog models (Qwen 3.5/3.6, Gemma 4) load a matching mmproj (Q8_0 or F16). If a vision GGUF is already in memory without a projector, the next `prompt` reloads it with mmproj. `prompt(..., { images })` accepts data URLs or `{ dataBase64, mime }` (jpeg/png/webp/gif, max 4). A separate capture app can POST stills to `http://127.0.0.1:11435/v1/vision/frames` (up to 5 FPS). **HTTPS pages (Pip) cannot hit that loopback** — use `BridgeSwarm.qvac.pushDesktopFrame({ dataUrl })` over native messaging instead. Each agent turn sees the latest frame unless `prompt(..., { desktopVision: false })`. Stream frames are not written to session JSONL. The host writes images under `$BRIDGE_SWARM_STORAGE/qvac/vision/` and passes QVAC `attachments: [{ path }]`. Image/video **generation** is still not implemented. Text-only models (`gpt-oss-20b`, `qwen3-*` INST) refuse images.

## Tools

Must-have: `read_file`, `write_file`, `search_replace`, `grep`, `list_dir`, `run_terminal_cmd`, `todo_write`

- `search_replace` requires a unique `old_string` unless `replace_all` is true. An empty `old_string` creates a file only if it is missing or empty. The result includes ±3 lines of context.
- `grep` prefers `rg` when present (glob + `output_mode`: `content` | `files_with_matches` | `count`) and falls back to a JS walker.
- Independent tools in one model turn run in parallel. `write_file` / `search_replace` that share a path are serialized. `ask_user_question`, `exit_plan_mode`, `update_goal`, `enter_plan_mode`, and `task` stay sequential.
- Long tool results and `web_fetch` are truncated head+tail with a recovery marker.

Also: `task` / `send_subagent_message` / `get_task_output` / `wait_tasks` / `kill_task` (subagents share the loaded model; `wait_tasks` actually waits), `web_search`, `web_fetch` (opt-in), `memory_search` / `memory_get` / `memory_write`, `enter_plan_mode` / `exit_plan_mode`, `ask_user_question`, `update_goal`, `search_tool` / `use_tool` (MCP HTTP: `initialize` + `tools/list` on register, then `tools/call`; stdio requires a trusted origin and is not spawned by default)

Shell is cwd-jailed via `bare-subprocess` (host-internal, not a page pack). MCP HTTP tools must be public URLs (`net` policy). Handshake failures are surfaced once as a system reminder.

## Permissions

`permissionMode: 'ask'` (default) prompts the page. Agent Studio offers **Allow**, **Always allow** (persist this command/path prefix next to sessions), and **Deny**. Remembered `git status` / `git diff` prefixes skip the prompt without turning on full-shell YOLO. `session.permit(jobId, toolCallId, true | false | 'always')`.

## Subagents

`task({ prompt, subagent_type })` with `explore` (read-only, default) or `general` (same write gate as the parent). At most **2** concurrent child tasks (one GPU). The child history is stored on the task record; `send_subagent_message` continues that inner loop (up to 8 turns). `task` still **blocks the parent** until the child finishes.

## Custom tools from the page

Handlers stay in the tab (they are never sent to the host). The host only stores the JSON schema and, when the model calls the tool, waits for `toolResult`.

```js
const session = await BridgeSwarm.agent.create({
  model: 'qwen3.5-4b',
  tools: [
    {
      name: 'get_cart',
      description: 'Return the in-page shopping cart',
      parameters: { type: 'object', properties: {} },
      execute: () => window.cart,
    },
  ],
})
await session.addTool({
  name: 'highlight',
  description: 'Highlight a line in the page editor',
  parameters: { type: 'object', properties: { line: { type: 'number' } }, required: ['line'] },
  execute: (args) => { editor.highlight(args.line); return 'ok'; },
})
await session.prompt('Add a README.md then highlight line 1')
```

Built-in names (`read_file`, `write_file`, …) cannot be overwritten **while host workspace tools are on**. Max 32 custom tools per session. Events include `tool_request` when the page must run a handler.

## Disable host workspace (page / container agents)

By default the agent ships host-jail tools against `$BRIDGE_SWARM_STORAGE/agent/<origin-hash>/`. Embedders that own their own filesystem (a container, a panel, an in-page editor) should turn that off so the model never sees or shells the native host:

```js
const session = await BridgeSwarm.agent.create({
  model: 'qwen3.5-4b',
  hostWorkspace: false,          // or hostTools: false
  workspace: 'dlinux-container', // prompt label only — not a host path
  builtinTools: false,           // optional: drop todo/task/web/MCP builtins too
  tools: [
    {
      name: 'read_file',
      description: 'Read a file inside the container',
      parameters: { type: 'object', properties: { path: { type: 'string' } }, required: ['path'] },
      execute: (args) => container.read(args.path),
    },
    {
      name: 'run_terminal_cmd',
      description: 'Run a command in the container',
      parameters: { type: 'object', properties: { command: { type: 'string' } }, required: ['command'] },
      execute: (args) => container.exec(args.command),
    },
  ],
})
```

With `hostWorkspace: false`:

- Host `read_file` / `write_file` / `search_replace` / `grep` / `list_dir` / `run_terminal_cmd` / `memory_*` are **not** offered and will not execute on the host.
- Those names may be registered as page tools (handlers stay in the tab).
- `cwd` / `workspace` is a label for the system prompt, not a path under `$BRIDGE_SWARM_STORAGE`.
- Subagents do not get host filesystem tools.
- Session JSONL still lives under `$BRIDGE_SWARM_STORAGE/agent/sessions/` (harness metadata only).

`builtinTools: false` removes remaining host-executed builtins (`todo_write`, `task`, `web_search`, MCP, …). Pass an array of names to keep a subset.

Pages cannot grant extra host roots via `create` — extra absolute roots still come only from **Settings → Agent workspace roots**.

## Sandbox

Default cwd: `$BRIDGE_SWARM_STORAGE/agent/<origin-hash>/`. Extra absolute roots: Settings → Agent workspace roots (pushed to the host as grants).

## Sessions

JSONL under `$BRIDGE_SWARM_STORAGE/agent/sessions/`. History is compacted near **85%** of the model context with one tool-free summary complete (five sections: goal, done, files, open work, next). Degenerate summaries fall back to drop-oldest. Compacted history is rewritten so a reload does not restore the uncompacted log. After compact, the loop injects reminders (todos, plan mode, AGENTS.md) and a short `[memory]` keyword block. Each prompt also refreshes the system message (AGENTS.md) plus a cached `git status -sb` sidecar (~30s) when the cwd is a git work tree. `plan.md`, plan-mode snapshot, and goal status live on the session summary. Allow/deny patterns persist in `$BRIDGE_SWARM_STORAGE/agent/permission-rules.json`.

Call `session.dispose()` (or `BridgeSwarm.agent.destroy(sessionId)`) when the page is done — that deletes the session dir, drops page-tool schemas, and schedules a GPU unload if nothing else is running. The loaded model is also released after **5 minutes idle** (not mid-turn: permission / ask-user / generation hold VRAM until the turn ends). The next `prompt` or `qvac.load` brings the same model back.

Page-tool schemas are stored on the session summary so a later `load` / `prompt` restores them without stacking a second model.

## ACP-shaped events

Host emits `cap-chunk` with `pack: 'agent'` and `type` in `agent_message_chunk`, `agent_thought_chunk`, `tool_call`, `tool_result`, `permission`, `ask_user`, `plan_approval`, `plan_update`, `goal_update`, `end`.

`end.reason` is one of `stop` | `max_turns` | `stuck` | `cancelled` | `goal_complete` | `goal_blocked`.

## Plan mode

`enter_plan_mode` / `session.prompt(text, { planMode: true })` switches the session to plan mode. While active:

- `run_terminal_cmd` is not offered; `write_file` / `search_replace` may only edit the session `plan.md` (host session dir, streamed as `plan_update`).
- Other writes are rejected at execute time, including page-registered `write_file` / `search_replace` / `run_terminal_cmd`.
- The model is reminded to end the turn with `ask_user_question` or `exit_plan_mode`.

`exit_plan_mode` emits `plan_approval` and **blocks** until the page calls `session.planDecision('approve' | 'reject')`. Approve injects “plan approved, implement it”; reject stays in plan mode.

```js
session.on((ev) => {
  if (ev.type === 'plan_approval') {
    session.planDecision('approve')
  }
})
await session.prompt('Plan a README then implement it', { planMode: true })
```

## Ask the user

`ask_user_question` emits `ask_user` and **waits**. Reply with `session.answer(toolCallId, choice)`.

```js
session.on((ev) => {
  if (ev.type === 'ask_user') session.answer(ev.toolCallId, ev.options[0])
})
```

## Goals

Done is **not** the model stopping tool calls. Start a goal with `create({ goal: '…' })` or `prompt(text, { goal: '…' })`. The agent must `update_goal({ completed: true })` after todos are finished. Open todos reject that call (premature stop). A single verifier `complete` then either ends with `reason: 'goal_complete'` or injects gaps and continues. `update_goal({ blocked_reason })` ends with `goal_blocked`. Pass `{ verify: false }` to skip the verifier.

If a goal is active and the model stops with open work, the loop injects a continuation instead of `end stop`.

## Prompt options

`session.prompt(text, { planMode, permissionMode, system, webFetch, goal, verify })` forwards those fields to the host.

Models, devices, enable toggle, and `BridgeSwarm.qvac` chat (page-owned tools, no workspace shell) are documented in [QVAC.md](QVAC.md).