Updates

2026-04-04 18:46:00 -04:00
parent d75f4e7383
commit 92368de550
14 changed files with 796 additions and 504 deletions
@@ -0,0 +1,15 @@
+# ADR 0001: Kernel subsystem boundaries (booter vs image)
+
+- **Status:** Accepted  
+- **Date:** 2026-04-04  
+- **Context:** The booter (`packages/bare-os-booter`) grew a large `index.js` while also owning VFS, swarm, IPC, identity, and `ctx` construction. Contributors need a stable mental model for where new code belongs.  
+- **Decision:**  
+  1. Treat **boot** (peer wait, MBR, replication, kernel bytes load, teardown ordering) as distinct from **executeKernel** (`ctx` assembly + REPL + initd).  
+  2. Centralize **stock kernel capability word** construction and **seed strict rows** in [`bare-os-capability-registry.js`](../../packages/bare-os-booter/lib/bare-os-capability-registry.js).  
+  3. Centralize **network/offline kernel loading** (seed RPC wave, MBR iteration) in [`bare-os-kernel-loader.js`](../../packages/bare-os-booter/lib/bare-os-kernel-loader.js).  
+  4. Centralize **Pear-safe teardown** (HDMS → swarm → drives → store) in [`bare-os-lifecycle-manager.js`](../../packages/bare-os-booter/lib/bare-os-lifecycle-manager.js).  
+  5. Document the full subsystem map in [`docs/architecture/kernel-subsystems.md`](../architecture/kernel-subsystems.md).  
+- **Consequences:**  
+  - `index.js` remains the orchestration root but delegates loader/capability/teardown modules.  
+  - Future refactors should move **`executeKernel`** into a dedicated module when ready, without changing public `ctx` contracts.  
+  - ADRs for larger changes should cross-link this boundary map.
@@ -0,0 +1,9 @@
+# Architecture Decision Records
+
+Short, durable decisions for kernel/booter structure and public contracts.
+
+| ADR | Title |
+| --- | --- |
+| [0001](0001-kernel-subsystem-boundaries.md) | Kernel subsystem boundaries (loader, capabilities, lifecycle) |
+
+Add new ADRs with the next sequential number and link them here.
@@ -0,0 +1,28 @@
+# Kernel subsystem map (booter + guest image)
+
+This document is the **boundary map** for the Bare operating system runtime: which responsibilities live in which layer, and which modules are the canonical entry points.
+
+## Subsystems
+
+| Subsystem | Role | Canonical modules |
+| --- | --- | --- |
+| **boot** | Swarm join, MBR, Hyperdrive replication, timeout, offline LKG | [`packages/bare-os-booter/index.js`](../../packages/bare-os-booter/index.js), [`bare-os-kernel-loader.js`](../../packages/bare-os-booter/lib/bare-os-kernel-loader.js), [`bare-os-lifecycle-manager.js`](../../packages/bare-os-booter/lib/bare-os-lifecycle-manager.js) |
+| **vfs** | Two-drive routing, pseudo `/proc`/`/sys`/`/run`/`/dev`, policy | [`vfs.js`](../../packages/bare-os-booter/lib/vfs.js), [`vfs-posix-meta.js`](../../packages/bare-os-booter/lib/vfs-posix-meta.js) |
+| **process** | Synthetic process table, jobs/shell coordination | [`bare-os-process-table.js`](../../packages/bare-os-booter/lib/bare-os-process-table.js), [`shell.js`](../../packages/bare-os-booter/lib/shell.js) |
+| **ipc** | FIFOs, JSON-RPC, fan-out | [`bare-os-ipc.js`](../../packages/bare-os-booter/lib/bare-os-ipc.js) |
+| **net** | Hyperswarm, Protomux, peer disk I/O | [`swarm-disk.js`](../../packages/bare-os-booter/lib/swarm-disk.js), [`bare-os-swarm-connection-manager.js`](../../packages/bare-os-booter/lib/bare-os-swarm-connection-manager.js) |
+| **security** | Identity, vault, ACL evaluation, audit chain | [`identity-session.js`](../../packages/bare-os-booter/lib/identity-session.js), [`identity-account.js`](../../packages/bare-os-booter/lib/identity-account.js), [`bare-os-acl-eval.js`](../../packages/bare-os-booter/lib/bare-os-acl-eval.js), [`bare-os-audit-chain.js`](../../packages/bare-os-booter/lib/bare-os-audit-chain.js) |
+| **capabilities** | Stock kernel capability words, seed strict matrix | [`bare-os-capability-registry.js`](../../packages/bare-os-booter/lib/bare-os-capability-registry.js), [`bare-os-protocol`](../../packages/bare-os-protocol/) |
+
+## Guest image (`kernel/`)
+
+| Stage | Source |
+| --- | --- |
+| Boot policy, rc, extensions | [`kernel/init.js`](../../kernel/init.js) |
+| Stock `/bin` | [`packages/bare-os-coreutils`](../../packages/bare-os-coreutils/) (seeded into image) |
+
+## Related
+
+- [KERNEL_CONTRACT.md](KERNEL_CONTRACT.md) — trust and replication contract.
+- [POSIX conformance matrix](../reference/conformance-matrix.md) — Issue 7 tracking.
+- [Environment and POSIX appendix](../reference/environment-and-posix-appendix.md) — env vars and userland gaps.
@@ -13,6 +13,9 @@ This directory holds the split **file-by-file inventory** that used to live in t
 | File-level inventory | This directory (`docs/reference/`) |
 | JSON Schemas | [`docs/schemas/`](../schemas/) |
 | Kernel contract (boundaries) | [`docs/architecture/KERNEL_CONTRACT.md`](../architecture/KERNEL_CONTRACT.md) |
+| Kernel subsystem map | [`docs/architecture/kernel-subsystems.md`](../architecture/kernel-subsystems.md) |
+| POSIX conformance matrix | [`docs/reference/conformance-matrix.md`](conformance-matrix.md) |
+| ADRs | [`docs/adr/`](../adr/) |
 | Audit baseline | [`docs/audit/PLACEHOLDER_BASELINE.md`](../audit/PLACEHOLDER_BASELINE.md) |
 | OTA / bundles (operators) | [`docs/deployment/OTA_AND_BUNDLES.md`](../deployment/OTA_AND_BUNDLES.md) |
 | CI / verifiers | [`scripts/README.md`](../../scripts/README.md) |
@@ -41,6 +44,7 @@ This directory holds the split **file-by-file inventory** that used to live in t
 - **Environment variables and POSIX appendix** (former §14, §14a) — [Environment variables and POSIX appendix](environment-and-posix-appendix.md)
 - **Kernel feature bits and handshake** — [Kernel extensions](kernel-extensions.md)
 - **Bits, env, `/proc`, and `ctx` map** — [Kernel capabilities index](kernel-capabilities-index.md)
+- **`ctx` API versioning** — [ctx-api-versioning.md](ctx-api-versioning.md)
 - **Version alignment** — [Compatibility matrix](compatibility-matrix.md)
 - **Legacy ↔ canonical names** — [Naming alias matrix](naming-alias-matrix.md)
 - **Kernel program status tables** — [Feature roadmap status](feature-roadmap.md)
@@ -51,6 +55,8 @@ This directory holds the split **file-by-file inventory** that used to live in t
 - **Kernel extensions (generated TOC)** — [kernel-extensions-generated-toc.md](kernel-extensions-generated-toc.md)
 - **OTA channel sketch** — [ota-channels.md](ota-channels.md)
 - **POSIX conformance dashboard (sketch)** — [conformance-dashboard.md](conformance-dashboard.md)
+- **POSIX conformance matrix (subsystem table)** — [conformance-matrix.md](conformance-matrix.md)
+- **Kernel subsystem boundaries** — [kernel-subsystems.md](../architecture/kernel-subsystems.md)
 - **Machine-readable data stubs** — [../data/README.md](../data/README.md)
 - **Kernel program (governed expansion)** — [developer-guide/kernel-program.md](../../developer-guide/kernel-program.md)
 - **Out of scope and tooling** (former §§15–16) — [Out of scope and tooling](out-of-scope-and-tooling.md)
@@ -2,4 +2,6 @@

 Maps **`/bin`** commands and flags to **supported**, **partial**, and **unsupported** against Issue 7 expectations. Authoritative behavior remains in `packages/bare-os-coreutils/src/*` and [handbook ch.9](../../handbook/09-posix-utilities-shell-and-vfs.md).

+**Structured matrix (subsystem × status):** [conformance-matrix.md](conformance-matrix.md).
+
 CI today: coreutils tests and man coverage (`scripts/verify-man-coverage.mjs`). This dashboard will grow as golden fixtures land (shell test corpus).
@@ -0,0 +1,63 @@
+# POSIX / SUSv4 conformance matrix (Bare OS)
+
+**Normative reference:** IEEE Std 1003.1-2017 (POSIX.1) / Issue 7 Base Specifications — [Open Group index](https://pubs.opengroup.org/onlinepubs/9699919799/).
+
+Bare OS targets **maximal POSIX-like** behavior on the **Bare** runtime with a **Hyperdrive-backed VFS** and **JavaScript userland**. Full certification is not claimed; this matrix records **intent**, **implementation surface**, and **known deltas**.
+
+## Legend
+
+| Status | Meaning |
+| --- | --- |
+| **Full** | Matches Issue 7 for the scoped flags/operands we document |
+| **Partial** | Subset or deliberate bound (env caps, quotas, GNU gaps) |
+| **Synthetic** | Emulated semantics (e.g. `df`, process table) |
+| **N/A** | Not applicable on this architecture (e.g. host `fork`) |
+
+## Shell and language (XCU `sh`)
+
+| Area | Status | Notes |
+| --- | --- | --- |
+| Pipelines, lists, redirects | Partial | See [environment appendix §14a](environment-and-posix-appendix.md#14a-posix-userland-appendix-implemented-vs-gaps) |
+| Parameter expansion | Partial | `BARE_OS_SHELL_PARAM_EXPANSION`, `BARE_OS_SHELL_PARAM_EXPANSION_V2` |
+| Command substitution | Partial | Bounded `$(…)` when `BARE_OS_SHELL_CMDSUBST` |
+| Job control | Partial | `jobs` / `fg` / `wait`, background `&` |
+| Full POSIX `sh` grammar | N/A | No subshells; not a standalone POSIX `sh` binary |
+
+## File system / VFS (XBD path + XSH-like ops)
+
+| Area | Status | Notes |
+| --- | --- | --- |
+| Path resolution | Partial | Symlinks, modes, personal vs system drive; see `vfs.js` |
+| `chmod`, `umask`, ownership display | Partial | `chown`/`chgrp` on personal metadata |
+| `mkfifo` | Partial | IPC under `/run/bare-os/ipc/` (see man pages) |
+| `sync` / durability | Synthetic | Documented no-op or policy-bound flush |
+
+## Utilities (XCU)
+
+| Bucket | Status | Source of truth |
+| --- | --- | --- |
+| Tier-1 `/bin` corpus | Partial → Full (per command) | `packages/bare-os-coreutils/src/*`, man JSON |
+| `awk` / `sed` | Partial | Not byte-identical to GNU on all inputs |
+| `find`, `xargs`, `test` | Partial | Caps via env (`BARE_OS_FIND_EXEC_MAX`, etc.) |
+
+## Process / signals (XSH)
+
+| Area | Status | Notes |
+| --- | --- | --- |
+| Real host PIDs | N/A | Synthetic table in `/proc`-style JSON |
+| `kill` / wait semantics | Partial | Tied to shell and delegates |
+
+## Networking (XNS subset)
+
+| Area | Status | Notes |
+| --- | --- | --- |
+| Sockets as POSIX BSD | Partial | Delegates / fetch policy where used |
+| P2P swarm | Synthetic | Hyperswarm + Protomux; not POSIX sockets |
+
+## CI and evidence
+
+- Booter + coreutils: `packages/bare-os-booter/test.js`, `packages/bare-os-coreutils` tests.
+- Man coverage: `scripts/verify-man-coverage.mjs`.
+- Dashboard sketch: [conformance-dashboard.md](conformance-dashboard.md).
+
+**Maintenance:** When behavior changes, update this matrix, [environment-and-posix-appendix.md](environment-and-posix-appendix.md), and the relevant man page JSON.
@@ -0,0 +1,19 @@
+# `ctx` API versioning policy
+
+## Current version
+
+- **`ctx.bareOsCtxApiVersion`** — semver string defined in [`bare-os-ctx-api.js`](../../packages/bare-os-booter/lib/bare-os-ctx-api.js).
+- **`BARE_OS_REQUIRE_CTX_API_MIN`** / **`BARE_OS_BOOT_ABI_STRICT`** — guest kernel may refuse boot if the booter is too old (see [environment appendix §14](environment-and-posix-appendix.md#14-environment-variables-complete-list)).
+
+## Rules for contributors
+
+1. **Additive changes** (new optional `ctx` methods, new env passthrough keys) — bump **patch** when behavior is backward compatible for existing kernels.
+2. **Breaking changes** (renamed methods, stricter required behavior, removed hooks) — bump **minor** or **major** per semver; update stock `kernel/init.js` and docs in the same change.
+3. **Deprecation** — document in this file and in [`package-bare-os-booter.md`](package-bare-os-booter.md); keep a compatibility shim for at least one release when feasible.
+4. **Protocol alignment** — capability words and wire formats belong in **`bare-os-protocol`**; bump **`BARE_OS_PROTOCOL_PACKAGE_VERSION`** consumers when wire or strict matrix changes.
+
+## Related
+
+- [Kernel subsystem map](../architecture/kernel-subsystems.md)
+- [Kernel capabilities index](kernel-capabilities-index.md)
+- [Compatibility matrix](compatibility-matrix.md)
@@ -21,9 +21,18 @@ Former **DOCUMENTATION.md** §§12.1–12.9 (runtime, VFS, shell, identity). [Re
 - **`packageRootDir(metaUrl)`** — Same as seeder (Pear RTI / `swapDir` / `cwd`).
 - **`defaultBootCorestorePath`** / **`defaultLocalSeedCorestorePath`** — `BARE_OS_BOOT_STORE` / `BARE_OS_LOCAL_SEED`, or under **`hostDataRoot()`** (`BARE_OS_HOST_DATA` or `~/.bare-os`): `corestore/booter` and `corestore/seeder` respectively. Same signature stability as the seeder helper.

+### 12.2a Boot modularity (loader, capabilities, lifecycle)
+
+- **[`lib/bare-os-kernel-loader.js`](../../packages/bare-os-booter/lib/bare-os-kernel-loader.js)** — **`loadOsFromPeers`**, **`loadOsFromOfflineLkg`**: seed capability handshake + optional RPC wave, MBR read, system Hyperdrive open/replicate, `/boot/init.js` fetch, personal drive mount (when not lazy). Invoked from `index.js` inside the boot timeout race.
+- **[`lib/bare-os-capability-registry.js`](../../packages/bare-os-booter/lib/bare-os-capability-registry.js)** — **`KERNEL_CAPABILITY_SEED_STRICT_ROWS`**, **`buildStockKernelCapabilityWords`**, **`freezeKernelCapabilityWordsFromCapabilities`**: single place for stock capability word math used by the booter and loader.
+- **[`lib/bare-os-lifecycle-manager.js`](../../packages/bare-os-booter/lib/bare-os-lifecycle-manager.js)** — **`exitHostProcess`**, **`teardownBareOsBootResources`**: `Bare.exit` deferral and ordered HDMS → swarm → drives → Corestore teardown (Pear heap safety).
+- **[`lib/bare-os-kernel-errno.js`](../../packages/bare-os-booter/lib/bare-os-kernel-errno.js)** — POSIX-inspired **`BARE_OS_ERRNO`** constants and name helpers for future unified error reporting across VFS/utilities.
+
+See also [Kernel subsystem map](../architecture/kernel-subsystems.md) and [ADR 0001](../adr/0001-kernel-subsystem-boundaries.md).
+
 ### 12.3 [packages/bare-os-booter/index.js](../../packages/bare-os-booter/index.js)

-**Imports** — Hyperswarm, Protomux, protocol, `./lib/swarm-disk.js`, `./lib/kernel-runner.js`, `./lib/vfs.js`, `./lib/shell.js`, `./lib/paths.js`, `./lib/bare-os-ipc.js`, `./lib/bare-os-runtime-caps.js`, stdio/readline/repl/boot-splash helpers, `./lib/identity-session.js`.
+**Imports** — Hyperswarm, Protomux, protocol, `./lib/swarm-disk.js`, `./lib/kernel-runner.js`, `./lib/bare-os-kernel-loader.js`, `./lib/bare-os-capability-registry.js`, `./lib/bare-os-lifecycle-manager.js`, `./lib/vfs.js`, `./lib/shell.js`, `./lib/paths.js`, `./lib/bare-os-ipc.js`, `./lib/bare-os-runtime-caps.js`, stdio/readline/repl/boot-splash helpers, `./lib/identity-session.js`.

 **`bootStorePath()`** — `defaultBootCorestorePath(_pkg, import.meta.url)`.

@@ -94,7 +103,7 @@ Former **DOCUMENTATION.md** §§12.1–12.9 (runtime, VFS, shell, identity). [Re

 **`search(query)`**

- Fan-out message 3 to all peers with shared id (actually each peer gets same id from counter — **one id per `search()` call**, registered in `pendingSearches`; concurrent searches could collide in id space; documented as current behavior).
+- Fan-out message 3 to all peers with a **single correlation id** per `search()` call. **`pendingSearches`** holds an aggregate state: each peer response appends matches until all peers have answered or a **3s** timeout fires, then the map entry is cleared. Concurrent searches use distinct ids (`searchIdCounter`), so operations do not overwrite each other’s callbacks.

 **`rpc(module, method, args?, timeoutMs?)`**