+3
-3
@@ -39,9 +39,9 @@ Workspace skills live in `skills/<name>/SKILL.md`. When a request matches a skil
|
||||
## Tools
|
||||
|
||||
- `read_file` / `list_dir` / `grep` / `write_file` / `search_replace` — workspace files.
|
||||
- `run_terminal_cmd` — local shell. Public HTTP via curl or wget is blocked; use web tools.
|
||||
- Web tools share one headed Playwright Chromium window. `web_search` / `google_search` / `wiki_search` / `hn_search` / `code_search` find links. `fetch_page` / `web_fetch` read a public page. For cookie banners, forms, logins, or leftover challenges, call `browser`. Before a multi-step browse, `read_file` `skills/browser/SKILL.md`.
|
||||
- `browser` actions: `navigate` (needs `url`), `snapshot`, `click` (`ref` from the last snapshot), `type` (`ref` + `text`, optional `submit`), `press` (`key`), `scroll` (`dy`), `wait` (`ms`). Snapshot or navigate first. Refs change after every click. Do not use `cu_observe` or the shell for websites.
|
||||
- `run_terminal_cmd` — local shell. Public HTTP via curl or wget is blocked; use `browser`.
|
||||
- The only web tool is `browser`, the headed Jarvis Chromium window. `web_search`, `web_fetch`, and the other search tools are removed. Before a multi-step browse, `read_file` `skills/browser/SKILL.md`.
|
||||
- `browser` actions: `navigate` (needs `url`; returns page text), `snapshot`, `click` (`ref` from the last snapshot), `type` (`ref` + `text`, optional `submit`), `press` (`key`), `scroll` (`dy`), `wait` (`ms`). To search, navigate to a public search url, then click a result ref. Snapshot or navigate first. Refs change after every click. Do not use `cu_observe` or the shell for websites.
|
||||
- Desktop and computer-use tools are registered by Jarvis. After Settings → Computer use → Allow now, call `cu_observe`, then `cu_click` / `cu_type`. Do not paste tool JSON into chat.
|
||||
- Webcam: Settings → Camera → Allow now. Call `webcam`. The still is attached; do not speak the file path. This is not `cu_observe`.
|
||||
- `ask_user_question` — wait for a user choice.
|
||||
|
||||
+6
-4
@@ -15,13 +15,15 @@ Local notes for this Jarvis session. This file is guidance, not an allowlist.
|
||||
## Shell
|
||||
|
||||
- `run_terminal_cmd` is a local user shell, not root.
|
||||
- Public HTTP via curl or wget is blocked. Use `web_search` / `web_fetch` in the Jarvis Chromium window.
|
||||
- Public HTTP via curl or wget is blocked. Use `browser` in the Jarvis Chromium window.
|
||||
|
||||
## Browser
|
||||
|
||||
- Search, fetch, and `browser` share one headed Playwright Chromium window.
|
||||
- `web_search` finds links. `fetch_page` reads a public page.
|
||||
- Cookie walls, forms, leftover challenges: call `browser`. `read_file` `skills/browser/SKILL.md` for the playbook.
|
||||
- `browser` is the only web tool. It drives one headed Playwright Chromium window.
|
||||
- `web_search`, `google_search`, `fetch_page`, `web_fetch`, `wiki_search`, `hn_search`, and `code_search` are removed. Do not call them.
|
||||
- To search, `navigate` to `https://duckduckgo.com/?q=QUERY` or a Google search url, then `click` a result ref.
|
||||
- `navigate` and `snapshot` return page `text`. Read it, then decide the next action.
|
||||
- Cookie walls, forms, leftover challenges: `read_file` `skills/browser/SKILL.md` for the playbook.
|
||||
- `browser` actions: `navigate` + `url`, `snapshot`, `click`/`type` with `ref` from the last snapshot, `press` + `key`, `scroll` + `dy`, `wait` + `ms`.
|
||||
- Snapshot or navigate before every click or type. Refs go stale after a click.
|
||||
- Do not use `cu_observe` or the shell for websites.
|
||||
|
||||
+20
-14
@@ -1,17 +1,13 @@
|
||||
---
|
||||
name: browser
|
||||
description: Drive the headed Jarvis Chromium window with the browser tool. Use for cookie walls, forms, logins, leftover bot checks, and any extra clicks after search or fetch.
|
||||
description: Drive the headed Jarvis Chromium window. This is the only web tool. Use it to search, open pages, click, and type. Do not use web_search, web_fetch, or computer-use for websites.
|
||||
---
|
||||
|
||||
# Browser
|
||||
|
||||
Search and fetch already run in this same Playwright Chromium window. Call `browser` when the page needs a click, a form, a cookie banner, or a leftover challenge. Do not use `cu_observe`, `cu_click`, curl, or wget for websites.
|
||||
`browser` is the only way to the public web. `web_search`, `google_search`, `fetch_page`, `web_fetch`, `wiki_search`, `hn_search`, and `code_search` are removed. Do not call them. Do not use `cu_observe`, `cu_click`, curl, or wget for websites.
|
||||
|
||||
## Choose a tool
|
||||
|
||||
1. `web_search` / `google_search` / `wiki_search` / `hn_search` / `code_search` — find public links.
|
||||
2. `fetch_page` / `web_fetch` — read one public http(s) page (JavaScript runs).
|
||||
3. `browser` — drive that window. Cookies persist across these tools.
|
||||
Chrome runs on this computer. After a navigate or snapshot, read the page `text` and decide the next click. If Groq is enabled, that thinking step uses Groq. The crawl itself stays local.
|
||||
|
||||
## `browser` actions
|
||||
|
||||
@@ -19,8 +15,8 @@ Call with `action` plus the fields for that action. Snapshot refs are strings li
|
||||
|
||||
| action | Fields | Result |
|
||||
| --- | --- | --- |
|
||||
| `navigate` | `url` (public http or https) | Opens the page and returns a snapshot |
|
||||
| `snapshot` | none | Current url, title, numbered `refs`, short aria text |
|
||||
| `navigate` | `url` (public http or https) | Opens the page and returns a snapshot plus readable `text` |
|
||||
| `snapshot` | none | Current url, title, numbered `refs`, page `text` |
|
||||
| `click` | `ref` from the last snapshot (or `selector` / `text`) | Clicks, then a fresh snapshot |
|
||||
| `type` | `ref` + `text`; optional `submit` true | Fills the field; `submit` presses Enter |
|
||||
| `press` | `key` (`Enter`, `Tab`, `Escape`, `Control+l`, …) | Key, then a snapshot |
|
||||
@@ -29,23 +25,33 @@ Call with `action` plus the fields for that action. Snapshot refs are strings li
|
||||
|
||||
Private, loopback, and metadata hosts are blocked before Chromium starts.
|
||||
|
||||
## Search
|
||||
|
||||
Do not call a search tool. Navigate to a public search url, then click a result:
|
||||
|
||||
- Web: `https://duckduckgo.com/?q=QUERY` or `https://www.google.com/search?q=QUERY`
|
||||
- Wikipedia: `https://en.wikipedia.org/w/index.php?search=QUERY`
|
||||
- Hacker News: `https://hn.algolia.com/?q=QUERY`
|
||||
- GitHub: `https://github.com/search?q=QUERY&type=repositories`
|
||||
- This computer's public IP: `https://ifconfig.me/ip`
|
||||
|
||||
## Loop
|
||||
|
||||
1. `navigate` or `snapshot` so you have fresh `refs`.
|
||||
2. Pick the ref whose `name` matches the control (Accept, Next, email, search box).
|
||||
1. `navigate` or `snapshot` so you have fresh `refs` and page `text`.
|
||||
2. Pick the ref whose `name` matches the control (Accept, Next, email, search box, a result title).
|
||||
3. `click` or `type` with that `ref`.
|
||||
4. Read the new snapshot. Refs from earlier snapshots are stale.
|
||||
5. Repeat until the page is usable, then `fetch_page` on the current url if you need the article text.
|
||||
5. Repeat until the page text answers the question, then speak the answer. Do not keep searching the same query.
|
||||
|
||||
## Cookie walls and challenges
|
||||
|
||||
- Cookie banner: `snapshot`, then `click` the Accept / Agree / I understand ref.
|
||||
- `challenge: true` or a Cloudflare / “just a moment” page: tell the user to finish the prompt in the visible Jarvis browser window. Do not guess. Then `snapshot` or `fetch_page` again.
|
||||
- `challenge: true` or a Cloudflare / “just a moment” page: tell the user to finish the prompt in the visible Jarvis browser window. Do not guess. Then `snapshot` again.
|
||||
- Login that needs a password: stop and ask the user. Never type credentials unless they just provided them for this site.
|
||||
|
||||
## Do not
|
||||
|
||||
- Speak refs, selectors, or tool JSON.
|
||||
- Call `browser` for a page you can already read with `fetch_page`.
|
||||
- Call `web_search`, `web_fetch`, `fetch_page`, or any other search tool.
|
||||
- Use computer-use tools on the Chromium window.
|
||||
- Keep searching the same query instead of opening a hit.
|
||||
|
||||
Reference in New Issue
Block a user