feat(kernel): wave 2 guest OS surface and operator parity
- Extend capability model (featureBits2) and align ADR, protocol, /proc, verify scripts - Add seed RPC + /proc mirrors for replication queue, MBR failover hints, optional attestation - Pear bridge: IPC request/response, mirror-drive / HDMS pairing hints, dev diagnostics - Initd: IdleSec for socket units, units.d drop-ins, richer readiness (e.g. exec:) - Cron: @reboot and JitterSec-style scheduling - VFS: Linux-shaped /proc stubs (cgroups, tcp), bounded vfs.watch on safe pseudo paths - Shell: gated parameter expansion v2; /bin/env -S and --env-file (staged script without ESM export) - Schemas under docs/schemas; expand contract tests; kernel.ext.d → /proc extensions registry - Refresh handbook, developer-guide, reference index, package READMEs; keep seeder kernel tree in sync
This commit is contained in:
Raven Scott
@@ -67,12 +67,16 @@ The following are set on `ctx` before the kernel starts (unless noted as overwri
|
||||
| **`bareOsVerifyBootManifestSignature(manifestBytes, signatureBytes, publicKeyHex?)`** | Ed25519 verify helper used when **`BARE_OS_BOOT_MANIFEST_SIGN=1`**; public key from arg or **`BARE_OS_BOOT_MANIFEST_PUBKEY_HEX`**. |
|
||||
| **`bareOsRequestMirror(opts?)`** / **`bareOsExportPersonalSnapshot(opts?)`** | **`async`** host bridges returning **`{ ok, hint }`**; on Node emit **`bare-os:mirror-request`** / **`bare-os:export-personal-snapshot`**. |
|
||||
| **`bareOsPearIpcEmit(channel, payload)`** | **`boolean`** — forwards structured **`payload`** to the host when registered (**`bare-os:pear-ipc`** on Node). Align channel names with your **[pear-ipc](https://github.com/holepunchto/pear-ipc)** consumer. |
|
||||
| **`bareOsPearIpcRequest(channel, payload, opts?)`** | **`Promise<unknown>`** — correlates **`bareOsIpcReqId`** on **`payload`**; host must **`process.emit('bare-os:pear-ipc-response', { bareOsIpcReqId, result?, error? })`** before **`opts.timeoutMs`** (default 30s). |
|
||||
| **`bareOsEmitMirrorDriveHint(opts?)`** | Emits **`bare-os:mirror-drive-hint`** on the host with **`label`**, **`key`**, **`ts`** (mirror-drive–style operator hint). |
|
||||
| **`bareOsRegisterKernelExtensionRecord(rec)`** | Appends **`{ dropin, script }`** for **`/proc/bare_os/extensions.json`** when the stock kernel loads **`kernel.ext.d`** scripts. |
|
||||
| **`bareOsDiagnosticsSubscribe(fn)`** / **`bareOsDiagnosticsEmit(ev)`** | When **`BARE_OS_DIAGNOSTICS_SUBSCRIBE=1`**, subscribe to structured booter/kernel diagnostics events (dev-oriented). |
|
||||
| **`bareOsHostStats`** _(optional)_ | When the **`bare-os`** npm module loads on the host, a **frozen** snapshot: **`hostname`**, **`loadavg`**, **`cpus`**, **`networkInterfaces`**, optional **`memoryUsage`**, **`peerCount`** (swarm peers during session build), **`atMs`**. |
|
||||
| **`httpFetch`** _(optional)_ | When the booter can build a policy-wrapped **`fetch`**, it sets this field; delegated **`curl`** / **`wget`** prefer **`resolveBareOsFetchFn`**, which uses **`ctx.httpFetch`** first, then **`ctx.bare.fetch`** (including **`/lib/bare/bundles`** merge, with **`.default`** unwrap), then **`globalThis.fetch`**. On hosts without native fetch, **`ensureBareFetchGlobals`** may install **`bare-fetch`** or **`bare-https`**. Optional **HTTP allow/deny** (**`BARE_OS_HTTP_ALLOWLIST`**, **`BARE_OS_HTTP_DENYLIST`**) and audit hooks when **`BARE_OS_AUDIT`** is on. See [HTTP: curl and wget](../docs/reference/http-curl-and-wget.md). |
|
||||
|
||||
Kernel boot composition lives on the **system image** (`/boot/init.js`, `/etc/bare-os/rc`, `/etc/bare-os/rc.d/`, optional **`/etc/bare-os/rc.local`**, optional **`/etc/bare-os/kernel.d/`** (same digit-prefix rules as **`rc.d`**), optional **`/etc/bare-os/profile`** / **`rc.profile.*`**, **`/etc/bare-os/onboot`**), not on `ctx`—extend the image or hooks like **`registerKernelShutdownHook`** rather than adding boot fields to the context object. The booter seeds **`ctx.env`** from the host for **`BARE_OS_PIPELINE_*`**, **`BARE_OS_SHELL_STREAMING`**, **`BARE_OS_SHELL_STREAMING_MULT`**, **`BARE_OS_SHELL_CMDSUBST`**, **`BARE_OS_SHELL_CMDSUBST_MAX_BYTES`**, boot profile / audit / IPC / HTTP policy keys (**`BARE_OS_IPC_CHANNEL_MAX_BYTES`**, …), **`BARE_OS_VFS_WATCH`**, **`BARE_OS_VFS_UNION_PREFIXES`**, **`BARE_OS_VFS_UNION_WRITE_DENY`**, **`BARE_OS_VFS_BIN_CACHE`**, **`BARE_OS_IMAGE_DIGEST`**, Pear channel fields, **`BARE_OS_BOOT_MANIFEST`**, **`BARE_OS_BOOT_MANIFEST_SIGN`**, **`BARE_OS_BOOT_MANIFEST_PUBKEY_HEX`**, **`BARE_OS_BOOT_POLICY`**, **`BARE_OS_SANDBOX_SCRIPT`**, **`BARE_OS_SANDBOX_WORKER`**, **`BARE_OS_INITD_MAX_PARALLEL`**, **`BARE_OS_INITD_JOURNAL_MAX_LINES`**, **`BARE_OS_URANDOM_CRYPTO`**, **`BARE_OS_TELEMETRY_NDJSON`**, **`BARE_OS_SEED_RPC_HANDSHAKE`**, **`BARE_OS_SEED_CAP_STRICT`**, **`BARE_OS_SEED_CAP_FAIL`**, **`BARE_OS_BLIND_BOOTSTRAP_URL`**, **`BARE_OS_BLIND_BOOTSTRAP_JSON`**, **`BARE_OS_MIRROR_READ_KEY`**, **`BARE_OS_FIND_EXEC_MAX`**, **`BARE_OS_YES_MAX_LINES`**, **`BARE_OS_SHUF_MAX_LINES`**, **`BARE_OS_SPLIT_MAX_FILES`**, **`BARE_OS_NPROC`**, **`TERM`**, **`COLORTERM`**, **`PEAR_CHANNEL`**, and the rest of the passthrough table in [environment appendix §14](../docs/reference/environment-and-posix-appendix.md#14-environment-variables-complete-list); always sets **`BARE_OS_BOOT_PROFILE_RESOLVED`** and **`BARE_OS_SESSION_ID`**. When **`ctx.httpFetch`** handles **`curl`**, check optional **`init.bareOsCurlTls`** (`insecure`, **`caPem`**, **`pinnedSha256`**) for **`--cacert`** / **`-k`** semantics.
|
||||
|
||||
After [`createVfs`](../packages/bare-os-booter/lib/vfs.js), **`ctx.vfs.watch(logicalPath)`** returns a Hyperdrive watcher when **`BARE_OS_VFS_WATCH`** is not disabled. **`ctx.bareOsIpc`** exposes FIFO **`push`/`take`** (optional per-channel byte caps from **`BARE_OS_IPC_CHANNEL_MAX_BYTES`**), optional JSON-RPC **`pushJson`/`takeJson`** (max line size, optional RPC token), **`fanoutPublish`/`fanoutSubscribe`**, **`createDuplexBridge`**, **`duplexJsonRoundTrip`** (one JSON request / one JSON reply over a duplex side — useful for unit-to-unit or guest–helper protocols without pulling **`bare-rpc`** into **`/bin`**), and **`stats`** (see [`bare-os-ipc.js`](../packages/bare-os-booter/lib/bare-os-ipc.js)).
|
||||
After [`createVfs`](../packages/bare-os-booter/lib/vfs.js), **`ctx.vfs.watch(logicalPath)`** returns a Hyperdrive watcher when **`BARE_OS_VFS_WATCH`** is not disabled. With **`BARE_OS_VFS_WATCH_PSEUDO=1`**, **`vfs.watch`** may also poll coalesced **`/proc/bare_os/metrics_live.json`** (interval from **`BARE_OS_PROC_POLL_MS`**). **`ctx.bareOsIpc`** exposes FIFO **`push`/`take`** (optional per-channel byte caps from **`BARE_OS_IPC_CHANNEL_MAX_BYTES`**), optional JSON-RPC **`pushJson`/`takeJson`** (max line size, optional RPC token), **`fanoutPublish`/`fanoutSubscribe`**, **`createDuplexBridge`**, **`duplexJsonRoundTrip`** (one JSON request / one JSON reply over a duplex side — useful for unit-to-unit or guest–helper protocols without pulling **`bare-rpc`** into **`/bin`**), and **`stats`** (see [`bare-os-ipc.js`](../packages/bare-os-booter/lib/bare-os-ipc.js)).
|
||||
|
||||
**Initd / long-running services:** prefer **`duplexJsonRoundTrip`** or **`pushJson`/`takeJson`** for structured messages with byte limits already enforced by IPC options. A dedicated **`bare-rpc`** dependency is optional on the host or in **`ctx.bare`** if you need richer framing; the stock image documents the FIFO-level building blocks only.
|
||||
|
||||
|
||||
@@ -23,9 +23,11 @@ npm test
|
||||
|
||||
## Brittle: `brittle-node` vs `brittle-bare`
|
||||
|
||||
- **`brittle-node`** runs most of **`packages/bare-os-booter/test.js`**—Hyperdrive, VFS, shell tokenizer, **`runBinCommand`** against real `/bin` bytes on disk.
|
||||
- **`brittle-node`** runs most of **`packages/bare-os-booter/test.js`**—Hyperdrive, VFS, shell tokenizer, **`runBinCommand`** against real `/bin` bytes on disk (includes wave-2 contract checks: **`@reboot`** / jitter parsing, param expansion v2, **`/proc`** stubs).
|
||||
- **`brittle-bare`** runs **`test.identity.js`** and protocol tests that need the **Bare** runtime (e.g. **bare-crypto** native pieces).
|
||||
|
||||
**Diagnostics:** with **`BARE_OS_DIAGNOSTICS_SUBSCRIBE=1`**, **`ctx.bareOsDiagnosticsSubscribe`** receives structured booter events (phase timing, optional kernel echoes) for local debugging—do not enable in untrusted production shells.
|
||||
|
||||
CI installs **Bare** globally for parity (see `.github/workflows/ci.yml`). If identity tests fail locally, ensure **`bare`** is installed and on `PATH`.
|
||||
|
||||
---
|
||||
|
||||
@@ -14,6 +14,8 @@ The **system** Hyperdrive is the **OS image**. You normally obtain it by **repli
|
||||
|
||||
**Boot manifest integrity:** **`BARE_OS_BOOT_MANIFEST=1`** makes the stock kernel compare **`/etc/bare-os/boot.manifest.json`** to an expected digest (**`ctx.bareOsBootFileSha256Hex`**). **`BARE_OS_BOOT_MANIFEST_SIGN=1`** adds **Ed25519** verification of the **raw manifest bytes** against **`/etc/bare-os/boot.manifest.sig`**, using **`ctx.bareOsVerifyBootManifestSignature`** and **`BARE_OS_BOOT_MANIFEST_PUBKEY_HEX`** (64 hex chars). Rotation is operational: ship a new public key in host env and a matching signature file on the system image. This does **not** replace image trust—you still choose peers and seed sources carefully.
|
||||
|
||||
**Seed capability attestation (optional):** the seeder may answer **`bare_os.capability_attestation`** with JSON from host env **`BARE_OS_SEED_CAPABILITY_ATTESTATION_JSON`** (for example a distro or **hyper-multisig** bundle). The guest does **not** verify signatures; operators verify on the host or out-of-band. **`BARE_OS_BIN_WORKER_OFFLOAD`** is a reserved gate for future isolated **`/bin`** execution—stock paths remain in-process unless a host integration supplies a worker bridge.
|
||||
|
||||
**Audit:** **`BARE_OS_AUDIT=1`** logs **`execLine`** activity to **`/var/log/bare-os/audit.log`**. **`BARE_OS_AUDIT_JSON=1`** uses JSON lines with **`auditSchemaVersion: 2`** on **`execLine`**, **delegate**, and **`httpFetch`** records; **`BARE_OS_AUDIT_REDACT=1`** (or **`true`**) masks common secret-like **`VAR=value`** patterns; **`BARE_OS_AUDIT_REDACT=TOKEN,PASSWORD`** masks named keys.
|
||||
|
||||
**Delegated HTTP:** when the booter sets **`ctx.httpFetch`** from the host **`fetch`**, **`BARE_OS_HTTP_ALLOWLIST`** and **`BARE_OS_HTTP_DENYLIST`** restrict **`http`/`https`** URLs for **`curl`/`wget`** (host-pattern globs). Failed checks throw before the request; with audit on, allow/deny outcomes can be logged.
|
||||
|
||||
@@ -10,20 +10,21 @@ This chapter ties the Hyperdrive-resident kernel ([`kernel/init.js`](../kernel/i
|
||||
|
||||
## Init, timers, and sockets
|
||||
|
||||
- **bare-initd** — User drop-ins under `~/.config/bare-os/units/<name>.unit` support `SocketActivationIpc=<fifo>`; the unit’s `start` runs after the first byte is read on that logical FIFO under `/run/bare-os/ipc/…`.
|
||||
- **Timers** — Files in `~/.config/bare-os/timers/*.timer` (max **8**) with a `[Timer]` section: either `OnCalendar=` (five cron fields) + `ExecLine=` on the same minute tick as `~/.crontab`, or `EveryMs=` (**1000**–**86400000**) + `ExecLine=` on a fixed `setInterval`. Copy from `/etc/bare-os/timers/*.timer.example` on the system image when present.
|
||||
- **System cron** — Optional image file `/etc/bare-os/crontab` (see [crontab.example](../kernel/etc/bare-os/crontab.example)) is merged with user crontab entries.
|
||||
- **bare-initd** — User drop-ins under `~/.config/bare-os/units/<name>.unit` support `SocketActivationIpc=<fifo>`; the unit’s `start` runs after the first byte is read on that logical FIFO under `/run/bare-os/ipc/…`. Optional `IdleSec=N` (with `stop`) stops the unit after N seconds without further IPC traffic; optional fragments `~/.config/bare-os/units.d/<name>/*.conf` merge before `~/.config/bare-init/units/` overrides. `ReadinessPath=` may be `exec:<execLine>` for a bounded readiness gate.
|
||||
- **Timers** — Files in `~/.config/bare-os/timers/*.timer` (max **8**) with a `[Timer]` section: either `OnCalendar=` (five cron fields) + `ExecLine=` on the same minute tick as `~/.crontab`, or `EveryMs=` (**1000**–**86400000**) + `ExecLine=` on a fixed `setInterval`. Optional `JitterSec=` spreads the first (or periodic) `execLine` within **0…N** seconds. Copy from `/etc/bare-os/timers/*.timer.example` on the system image when present.
|
||||
- **System cron** — Optional image file `/etc/bare-os/crontab` (see [crontab.example](../kernel/etc/bare-os/crontab.example)) is merged with user crontab entries. Lines may use `@reboot` + command (once per `bare-cron` start) and optional `JitterSec=N` prefix on the command for calendar lines.
|
||||
|
||||
## Observability
|
||||
|
||||
- **`ctx.bareOsSubscribeBootEvent`** — Same structured events as `BARE_OS_BOOT_TRACE=ndjson` (phase, ms, sessionId); the booter also emits **`booter:*`** phases (**`vfs`**, **`ctx`**, **`repl`**, **`initd`**, **`kernel_invoke`**).
|
||||
- **`/proc/bare_os_quotas`** — Pipeline limits, `BARE_OS_EXEC_MAX_DEPTH`, IPC caps, session stats.
|
||||
- **`/proc/bare_os_resources`** / **`ctx.bareOsGetResourceStatus()`** — Unified snapshot for operators.
|
||||
- **`/proc/bare_os_features`** — Documented kernel-feature bitmask (see `bare-os-protocol` exports).
|
||||
- **`ctx.vfs.watch(path)`** — Hyperdrive-backed watch when `BARE_OS_VFS_WATCH` is not `0`; returns `{ watcher, destroy, … }`.
|
||||
- **`/proc/bare_os_features`** — Documented kernel-feature words **`bits`** / **`bits2`** (see `bare-os-protocol` exports).
|
||||
- **`ctx.vfs.watch(path)`** — Hyperdrive-backed watch when `BARE_OS_VFS_WATCH` is not `0`; returns `{ watcher, destroy, … }`. With `BARE_OS_VFS_WATCH_PSEUDO=1`, may watch coalesced `/proc/bare_os/metrics_live.json`.
|
||||
|
||||
## Pear / Git / HTTP
|
||||
|
||||
- **Mirror-drive hints** — `ctx.bareOsEmitMirrorDriveHint({ label?, key? })` emits `bare-os:mirror-drive-hint` on Node-style hosts for mirror-drive–style workflows (labels/keys only; trust boundaries unchanged).
|
||||
- **Release metadata** — Host can set `BARE_OS_PEAR_CHANNEL`, `BARE_OS_PEAR_RELEASE`, and `BARE_OS_IMAGE_DIGEST`; they appear in `/run/bare-os/boot.json`.
|
||||
- **`ctx.bareOsRequestPearReload()`** — Returns hints and env strings; the host **`pear-runtime` / `pear-runtime-updater`** must perform any real reload.
|
||||
- **`git-pear`** — `/bin/git-pear help` documents Git-in-Pear (`gip-transport`, `gip-remote`, `git+pear://` remotes).
|
||||
|
||||
@@ -56,6 +56,10 @@ This chapter ties together **Holepunch `bare-*` packages**, the **Pear** host ru
|
||||
|
||||
**`ctx.bareOsPearIpcEmit(channel, payload)`** is a host bridge for [pear-ipc](https://github.com/holepunchto/pear-ipc)-style messaging. Well-known **`channel`** strings are listed in **`packages/bare-os-booter/lib/bare-os-pear-ipc-registry.js`** and mirrored as JSON under **`/proc/bare_os/pear_ipc.json`** (and the flat **`/proc/bare_os_pear_ipc_registry`** alias) so operators and kernels share one vocabulary for reload, mirror, and telemetry hints.
|
||||
|
||||
**`ctx.bareOsPearIpcRequest(channel, payload, { timeoutMs? })`** correlates **`bareOsIpcReqId`** on **`payload`** and awaits **`process.emit('bare-os:pear-ipc-response', { bareOsIpcReqId, result?, error? })`** on Node-style hosts. Use for request/response pairs alongside **`bareOsPearIpcEmit`**.
|
||||
|
||||
**Mirror-drive hints:** **`ctx.bareOsEmitMirrorDriveHint({ label?, key? })`** emits **`bare-os:mirror-drive-hint`** for Holepunch-style snapshot workflows (see [kernel extensions](../docs/reference/kernel-extensions.md)).
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# ADR 001: Kernel feature bit governance (v2)
|
||||
# ADR 001: Kernel feature bit governance (v3)
|
||||
|
||||
## Status
|
||||
|
||||
@@ -19,9 +19,11 @@ Bare OS advertises optional booter and seed-channel capabilities as a versioned
|
||||
|
||||
3. **`/proc/bare_os_features`** — Must include `doc`, effective `bits` (stock mask minus env-disabled features such as crypto urandom), and optional `seedHandshake` summary. The unified tree under `/proc/bare_os/` duplicates the same content for stable paths (see `docs/reference/kernel-extensions.md`).
|
||||
|
||||
4. **Bits 28–30 (assigned)** — **28:** `BARE_OS_FEATURE_SEED_MANIFEST_HINTS_RPC` (`bare_os.manifest_hints`). **29:** `BARE_OS_FEATURE_LIFECYCLE_TELEMETRY_V3` (NDJSON / boot events `lifecycleSchemaVersion: 3`). **30:** `BARE_OS_FEATURE_BOOT_POLICY_V2` (extended `boot.policy.json` fields). Further bits require a new ADR revision if the integer space is exhausted (avoid `1 << 31` in JS without `>>> 0` discipline).
|
||||
4. **Bits 28–30 (assigned, word 1)** — **28:** `BARE_OS_FEATURE_SEED_MANIFEST_HINTS_RPC` (`bare_os.manifest_hints`). **29:** `BARE_OS_FEATURE_LIFECYCLE_TELEMETRY_V3` (NDJSON / boot events `lifecycleSchemaVersion: 3`). **30:** `BARE_OS_FEATURE_BOOT_POLICY_V2` (extended `boot.policy.json` fields). **No further bits** are assigned on word 1 without revisiting this ADR (avoid `1 << 31` in JS without `>>> 0` discipline).
|
||||
|
||||
5. **Second capability word (`bits2`)** — Wave 2 adds **`BARE_OS_KERNEL_FEATURES_STOCK_V2`** and named **`BARE_OS_FEATURE2_*`** constants (bits 0–31, always use **`>>> 0`** when masking). Seed JSON **`bare_os.capabilities`** and **`/proc/bare_os_features`** include **`bits2`** alongside **`bits`**. **`BARE_OS_SEED_CAP_STRICT`** may require both words when the booter advertises stock v2. New guest-visible behavior on word 2 follows the same documentation rule as word 1: [`docs/reference/kernel-capabilities-index.md`](../../docs/reference/kernel-capabilities-index.md), [`docs/reference/kernel-extensions.md`](../../docs/reference/kernel-extensions.md), and relevant handbook chapters.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Seeder RPC `bare_os.capabilities` must return `protocolPackageVersion` and `featureBitsDoc` alongside `bits` for operator clarity.
|
||||
- CI should keep seeder and booter copies of the stock mask aligned (`BARE_OS_KERNEL_FEATURES_STOCK_V1`).
|
||||
- Seeder RPC `bare_os.capabilities` must return `protocolPackageVersion` and `featureBitsDoc` alongside `bits` (and `bits2` when used) for operator clarity.
|
||||
- CI should keep seeder and booter copies of the stock masks aligned (`BARE_OS_KERNEL_FEATURES_STOCK_V1`, `BARE_OS_KERNEL_FEATURES_STOCK_V2`).
|
||||
|
||||
Reference in New Issue
Block a user