89 lines
8.4 KiB
Markdown
89 lines
8.4 KiB
Markdown
# HTTP clients: `curl` and `wget` on Bare OS
|
||
|
||
[Reference index →](README.md)
|
||
|
||
Bare OS exposes **`curl`** and **`wget`** as familiar command names, but they are **not** Daniel Stenberg’s libcurl or full GNU **wget**. Both are **JavaScript CLIs** in the booter that use the **Fetch API** (or a compatible implementation), support a **documented subset** of common flags, and honor the same **HTTP / DNS policy** environment variables as the rest of the session.
|
||
|
||
---
|
||
|
||
## Where to read more
|
||
|
||
- Flag parity and history — [`packages/bare-os-booter/CLI_PARITY.md`](../../packages/bare-os-booter/CLI_PARITY.md)
|
||
- Online **`man curl`** / **`man wget`** — Merged manual DB: [`packages/bare-os-coreutils/man/pages/curl.json`](../../packages/bare-os-coreutils/man/pages/curl.json), [`wget.json`](../../packages/bare-os-coreutils/man/pages/wget.json); narrative in [Handbook ch. 10](../../handbook/10-manpages-and-online-help.md)
|
||
- Security, allowlists, audit — [Developer guide — Security and trust](../../developer-guide/09-security-and-trust.md)
|
||
- Environment variables (tables) — [Environment variables and POSIX appendix](environment-and-posix-appendix.md#14-environment-variables-complete-list)
|
||
- Capability / proc map — [kernel-capabilities-index.md](./kernel-capabilities-index.md); kernel program net sketches — [kernel-extensions.md](./kernel-extensions.md) ( **`BARE_OS_NET_DELEGATE_SKETCH_JSON`** (legacy **`BARE_OS_GP2_NET_DELEGATE_SKETCH_JSON`**), **`BARE_OS_DNS_PROFILE`**, **`BARE_OS_DELEGATE_TRACE`** )
|
||
|
||
---
|
||
|
||
## Invocation: delegates run before `/bin`
|
||
|
||
When the shell or **`ctx.runBinCommand`** resolves an external command, [`runBinCommand` in `kernel-runner.js`](../../packages/bare-os-booter/lib/kernel-runner.js) consults the **host delegate registry** **first** (see [`host-delegate-registry.js`](../../packages/bare-os-booter/lib/host-delegate-registry.js)). Delegates are registered with **static imports** of **`curl-cli.js`**, **`wget-cli.js`**, **`git-cli.js`**, and **`systemctl-cli.js`** so Pear staging and module resolution stay predictable (avoid dynamic `**import('./…')`** of sibling CLI modules from deep paths).
|
||
|
||
Order for a simple command name:
|
||
|
||
1. **Host delegates** — basename match for **`git`**, **`curl`**, **`wget`**, and **`systemctl`** / **`bare-initctl`** / **`journalctl`**.
|
||
2. **Explicit paths** — **`/`** in **`argv[0]`** → VFS script load.
|
||
3. **`*.js` in `$PWD**` — before **`PATH`**.
|
||
4. **`PATH`** on the **system** drive only — normal **`/bin`** utilities.
|
||
|
||
**`BARE_OS_DELEGATE_ALLOW`** (host → session passthrough) may restrict which delegate **kinds** run. When set to a comma- or whitespace-separated list (**`git`**, **`curl`**, **`wget`**, **`systemctl`**), only those kinds are allowed; a denied delegate logs a message and exits with **126**. When unset or empty, all registered delegates are allowed. Optional **per-minute** rate limits use **`BARE_OS_DELEGATE_MAX_PER_MIN`** and per-kind `**BARE_OS_DELEGATE_*_MAX_PER_MIN`**. Optional **in-flight** caps use **`BARE_OS_DELEGATE_MAX_CONCURRENT`** (global) and **`BARE_OS_DELEGATE_<KIND>_MAX_CONCURRENT`** (e.g. **`CURL`**, **`GIT`**, **`WGET`**, **`SYSTEMCTL`**) to queue or reject parallel delegate runs. With `**BARE_OS_AUDIT=1**`, `**BARE_OS_DELEGATE_AUDIT_ONLY=1`** logs invocations and skips the host run (**exit 0**). Optional audit logging for delegate invocations is described in the security guide.
|
||
|
||
---
|
||
|
||
## `/bin/curl` and `/bin/wget` on the system image
|
||
|
||
The coreutils build still emits **`kernel/bin/curl`** and **`kernel/bin/wget`** so **`ls /bin`**, **`which curl`**, and the sorted **`/bin`** manifest stay complete. Those files are **placeholders**: if the runner ever executed them, they would print an error, because the booter is expected to **delegate** first. In normal operation you never run the stub bodies in [`packages/bare-os-coreutils/src/curl.js`](../../packages/bare-os-coreutils/src/curl.js) and [`wget.js`](../../packages/bare-os-coreutils/src/wget.js).
|
||
|
||
---
|
||
|
||
## How HTTP requests choose a `fetch` implementation
|
||
|
||
Delegated **`curl`** and **`wget`** share the resolution helpers in [`bare-os-ensure-bare-fetch.js`](../../packages/bare-os-booter/lib/bare-os-ensure-bare-fetch.js).
|
||
|
||
**Per-request function** — **`resolveBareOsFetchFn(ctx)`** returns, in order:
|
||
|
||
1. **`ctx.httpFetch`** when it is a function — typically the booter’s **policy-wrapped** fetch (HTTP allow/deny lists, optional audit).
|
||
2. Else **`ctx.bare.fetch`** when present — drive bundles may expose **`fetch`**; legacy bundles sometimes attach a CJS-shaped object, so the booter **unwraps** **`.default`** when needed (**`coerceBareFetchExport`**).
|
||
3. Else **`globalThis.fetch`** when defined.
|
||
|
||
**Priming globals on hosts without native fetch** — **`ensureBareFetchGlobals(ctx)`** (async) runs when the CLI needs globals installed:
|
||
|
||
1. If **`globalThis.fetch`** already exists, return.
|
||
2. Try **`ctx.bare.fetch`** (same **`.default`** unwrap) and copy **`fetch`** / **`Request`** / **`Response`** / **`Headers`** onto **`globalThis`** (and **`global`** on Node).
|
||
3. Otherwise try **`import`** of **`bare-fetch`** (via **`import.meta.resolve`** when available, then `**import(href)`**, for Pear-friendly resolution).
|
||
4. If still missing, try **`bare-https`** and use its exported **`fetch`**.
|
||
|
||
The booter may also call **`primeGlobalFetchFromBareLibrary(bareLibrary)`** during **`ctx`** assembly so **`globalThis.fetch`** exists before guest code runs when the system image supplies **`/lib/bare/bundles/fetch.js`**.
|
||
|
||
**TLS and CA behavior** for delegated **`curl`** can use optional **`init.bareOsCurlTls`** ( **`insecure`**, **`caPem`**, **`pinnedSha256`**) alongside **`BARE_OS_TLS_PIN_SHA256`**; see the security guide and kernel **`init`** documentation.
|
||
|
||
**Capability word 6 (rotation and HTTP hints)** — For operators running **bare-fetch** (or host **`fetch`**) behind the delegate, document **multi-pin** rotation as a comma- or JSON-list of hex digests (same semantics as single-pin, evaluated by the host). **HSTS preload** lists and **alt-svc** hints are **host-layer** concerns: reserved env names **`BARE_OS_TLS_PINS_JSON`**, **`BARE_OS_HSTS_PRELOAD_PINS_JSON`**, **`BARE_OS_CURL_ALT_SVC_JSON`** are listed in [kernel-extensions.md](./kernel-extensions.md) for Pear-side wiring; the stock in-guest booter does not parse them.
|
||
|
||
---
|
||
|
||
## Policy-related environment variables
|
||
|
||
These are summarized in the [environment appendix](environment-and-posix-appendix.md); the security guide explains threat model and audit interaction.
|
||
|
||
- **`BARE_OS_HTTP_ALLOWLIST`** / **`BARE_OS_HTTP_DENYLIST`** — Restrict outbound **http(s)** URLs for policy-wrapped fetch (delegated clients and **`ctx.httpFetch`**).
|
||
- **`BARE_OS_TLS_PIN_SHA256`** — Optional TLS certificate pinning hint forwarded for pinning-aware hosts.
|
||
- **`BARE_OS_DNS_ALLOWLIST`** — Optional host allowlist for **http(s)** URLs in delegated **`curl`** / **`wget`** (suffix forms such as `***.example.com`** supported).
|
||
- **`BARE_OS_DELEGATE_ALLOW`** — Limits which delegate kinds (**`curl`**, **`wget`**, …) may run.
|
||
|
||
---
|
||
|
||
## `agent` and the `web_fetch` tool
|
||
|
||
In-guest **`/bin/agent`** uses the same **`ctx.httpFetch`** implementation as delegated **`curl`** / **`wget`**: policy-wrapped **`fetch`** when the booter sets **`ctx.httpFetch`**, with **`BARE_OS_HTTP_ALLOWLIST`** / **`BARE_OS_HTTP_DENYLIST`** applying to **both** the OpenAI-compatible provider **`/chat/completions`** URL and URLs fetched by the assistant’s **`web_fetch`** tool. Operators must allowlist **every** host the session will call (API origin plus any `**http(s)`** sites you expect **`web_fetch`** to retrieve). **`web_fetch`** (and **`bare-fetch` 3**) may also surface **`response.type`** and **`Headers.getSetCookie()`** on the tool result when the host implementation provides them.
|
||
|
||
Configuration and tool semantics are **not** duplicated here — see guest **`man agent`** ([`packages/bare-os-coreutils/man/pages/agent.json`](../../packages/bare-os-coreutils/man/pages/agent.json) in the repo) and [User manual — ch.4](../../users-manual/04-shell-path-and-scripts.md).
|
||
|
||
---
|
||
|
||
## Related handbook sections
|
||
|
||
- [Handbook ch. 4 — Booter runtime](../../handbook/04-the-booter-runtime.md) ( **`ctx`**, HTTP policy overview, **`agent`**)
|
||
- [Handbook ch. 6 — Kernel and `/bin`](../../handbook/06-kernel-and-binaries.md) (delegated vs coreutils)
|
||
- [Handbook ch. 9 — POSIX utilities](../../handbook/09-posix-utilities-shell-and-vfs.md) (catalog classification; **`agent`** / **`chat`** as non-POSIX Tier-1)
|