bare-operating-system/docs/README.md

# Bare OS — documentation home

**Bare OS** is an experimental, **P2P-first** system image: the operating tree lives on a replicated **Hyperdrive**, distributed via **Hyperswarm** and **Protomux**, with a separate **personal** drive for identity and mutable state. This page is the **navigation hub** for every documentation tree in the repository.

> [!CAUTION]
> **Experimental research software** — not a production operating system. Trust boundaries and security limits matter.

> **Important**  
> Same message on forges that do not render GitHub-style alerts: this project is **research software**, not a production OS. Full framing: [handbook preface](../handbook/00-preface.md).

**First run:** **[Get started](get-started.md)** · **Doc map:** **[Sitemap](sitemap.md)** · Evergreen explainers: **[Concepts](concepts/README.md)**.

## On this page

- [Get started](get-started.md) · [Sitemap](sitemap.md)
- [Pick your depth](#pick-your-depth)
- [Architecture snapshot](#architecture-snapshot)
- [Two-drive and swarm path](#two-drive-and-swarm-path)
- [Boot sequence logical order](#boot-sequence-logical-order)
- [POSIX + P2P reading order](#posix--p2p-reading-order)
- [Contract spine](#contract-spine-quick-cross-links)
- [POSIX + P2P consolidated dashboard](#posix--p2p-consolidated-dashboard)
- [Documentation map](#documentation-map)
- [Topic index](#topic-index-canonical-reference-per-subject)
- [Contract bump checklist](#contract-bump-checklist)

## Pick your depth

| Time | Goal | Path |
| --- | --- | --- |
| **5 min** | See it run | [Get started](get-started.md) → [User manual — ch.3](../users-manual/03-running-seeder-and-booter.md) |
| **30 min** | Two drives + P2P boot | [Concepts](concepts/README.md) → [Handbook ch.3](../handbook/03-protocol-and-disk.md) |
| **2 h** | Extend `/bin` or kernel | [Developer guide §1–6](../developer-guide/01-two-runtimes-host-vs-image.md) + [KERNEL_CONTRACT](architecture/KERNEL_CONTRACT.md) |
| **Release** | Avoid CI surprises | [scripts/README.md](../scripts/README.md) + [Release checklist](release-checklist.md) |

## Architecture snapshot

End-to-end data path from swarm connection to guest syscalls (canonical overview; details in the [handbook](../handbook/README.md), [concepts — P2P](concepts/p2p-swarm-and-protomux.md), and [kernel contract](architecture/KERNEL_CONTRACT.md)):

```mermaid
flowchart TB
  subgraph swarmLayer [P2P transport]
    HS[Hyperswarm topic]
    MUX[Protomux on encrypted stream]
    CH1["Channel bare-os-v1"]
    CH2["Optional bare-os-app-v1"]
  end
  subgraph booterHost [Pear or Node booter]
    DISK[SwarmDisk + Corestore]
    CTX["ctx: vfs IPC syscall swarm hooks"]
  end
  subgraph guestImage [Hyperdrive system image]
    INIT["/boot/init.js stock kernel"]
    BIN["/bin coreutils"]
  end
  HS --> MUX
  MUX --> CH1
  MUX --> CH2
  CH1 --> DISK
  DISK --> CTX
  INIT --> CTX
  BIN --> CTX
```

## Two-drive and swarm path

How the **personal** drive sits beside the **system** image and how the booter reaches both after swarm replication (narrative: [concepts — Two drives](concepts/two-drive-model.md), [concepts — Boot](concepts/boot-and-init-timeline.md)):

```mermaid
flowchart TB
  subgraph p2p [Swarm]
    T[Hyperswarm topic]
    M[Protomux bare-os-v1]
  end
  subgraph booter [Booter host]
    SD[SwarmDisk Corestore]
    VFS[VFS router]
  end
  subgraph drives [Hyperdrives]
    SYS[System drive read policy]
    PER[Personal drive home and .bare]
  end
  T --> M
  M --> SD
  SD --> SYS
  SD --> PER
  SYS --> VFS
  PER --> VFS
  VFS --> SH[Shell and /bin]
```

## Boot sequence (logical order)

High-level order only — not every Protomux message or RPC. **Text fallback (for `man` ingest and plain-text readers):** the seeder publishes **MBR block 0** and drive blocks on the swarm; the booter **discovers peers**, **reads the MBR**, **opens and replicates the system Hyperdrive**, **opens or creates the personal Hyperdrive**, **builds `ctx`**, then **runs `/boot/init.js`**. For bytes and message IDs, read [Handbook ch.3](../handbook/03-protocol-and-disk.md) and the [protocol reference](reference/package-bare-os-protocol.md).

```mermaid
sequenceDiagram
  participant Seeder
  participant Swarm as Hyperswarm_topic
  participant Booter
  participant Sys as System_Hyperdrive
  participant Per as Personal_Hyperdrive
  Seeder->>Swarm: publish_MBR_and_blocks
  Booter->>Swarm: discover_and_replicate
  Swarm-->>Booter: MBR_block_0
  Booter->>Sys: open_and_replicate
  Booter->>Per: open_or_create
  Booter->>Booter: build_ctx_and_run_boot_init
```

## POSIX + P2P reading order

Read **once** in this order when you need the full contract picture; then use the [consolidated dashboard](#posix--p2p-consolidated-dashboard) as your bookmark.

| Step | Focus | Human doc | Machine / generated |
| --- | --- | --- | --- |
| 1 | Declared stance | [POSIX declared profile](architecture/POSIX_DECLARED_PROFILE.md) | [`posix-compliance-matrix.json`](reference/posix-compliance-matrix.json) |
| 2 | Dashboard | [POSIX compliance dashboard](reference/posix-dashboard.md) | `pretest` → `gen-posix-dashboard.mjs` |
| 3 | Utilities + shell | [Handbook ch.9](../handbook/09-posix-utilities-shell-and-vfs.md) | [`syscalls.example.json`](../kernel/etc/bare-os/syscalls.example.json) |
| 4 | Syscall facade | [POSIX syscall facade map](reference/posix-syscall-facade-map.md) | [`bare-os-syscalls.schema.json`](schemas/bare-os-syscalls.schema.json) |
| 5 | P2P vs POSIX boundary | [Handbook ch.12](../handbook/12-p2p-stack-and-posix-boundaries.md) | [Capability words](reference/capability-words.md) |

**Environment variables** (full inventory, do not duplicate in prose): [Environment and POSIX appendix](reference/environment-and-posix-appendix.md).

### Compared at a glance

| | Bare OS | Classic OS image | Container on a host |
| --- | --- | --- | --- |
| **Distribution** | P2P Hyperdrive + swarm | ISO / image mirror | Registry pull |
| **Mutable user state** | Personal Hyperdrive | Local disk | Writable layers / volumes |
| **Guest execution** | `ctx` + VFS over drives | Kernel + real HW/VM | Namespaces + host kernel |
| **POSIX** | Declared profile + facade | Full kernel | Often libc + syscall pass-through |

---

## Contract spine (quick cross-links)

Normative and machine-readable contracts that should move together when behavior changes:

| Surface | Role |
| --- | --- |
| [POSIX declared profile](architecture/POSIX_DECLARED_PROFILE.md) | Narrative stance, non-goals, env gates (`BARE_OS_POSIX_PROFILE_VERSION`). |
| [`ctx` API version](reference/compatibility-matrix.md) | Guest-visible `ctx` semver in `bare-os-ctx-api.js`. |
| [Protocol / feature bits](reference/compatibility-matrix.md) | `bare-os-protocol` package version, kernel feature words, Protomux channel names. |
| [Telemetry / audit](schemas/) | NDJSON and JSON Schema shards under `docs/schemas/`; handbook ops ch.7. |
| [`/proc` and HRPC](reference/package-bare-os-booter.md) | Proc JSON schema bumps, `disk.os` allowlist, `replication_operator_sketch` versions. |

```mermaid
flowchart LR
  P[POSIX profile + syscalls.json] --> M[posix-compliance-matrix.json]
  C[ctx API semver] --> CM[compatibility-matrix.md]
  PR[bare-os-protocol] --> FB[kernel feature bits]
  B["booter /proc builders"] --> HRPC[ctx.bareOsHrpcRequest table]
  K["/boot/init.js boot policy"] --> T[telemetry NDJSON]
```

Run **`npm run pretest`** after contract edits so generated dashboards and parity scripts stay aligned.

---

## POSIX + P2P consolidated dashboard

Single entry point for **declared POSIX-like behavior**, machine-readable matrices, and **P2P / capability** alignment:

- [Declared POSIX profile](architecture/POSIX_DECLARED_PROFILE.md) — normative narrative and non-goals.
- [POSIX compliance dashboard](reference/posix-dashboard.md) — generated summary from [`posix-compliance-matrix.json`](reference/posix-compliance-matrix.json) (run **`npm run pretest`** / **`gen-posix-dashboard.mjs`**).
- [POSIX Issue 7 traceability index](reference/posix-issue7-traceability.md) — maps XBD/XSH/XCU to **`/proc`**, **`ctx`**, and handbook chapters.
- [Holepunch lockfile drift dashboard](audit/holepunch-lockfile-drift-dashboard.md) — human table from **`report-holepunch-lockfile-drift.mjs`** (also **`npm run audit:holepunch-clones`**).
- [POSIX utilities matrix (handbook)](../handbook/09-posix-utilities-shell-and-vfs.md) — Tier-1 **`/bin`** catalog and shell semantics.
- [Syscall / socket bridge contract](reference/syscall-socket-contract.md) — **`ctx.bareOsSyscall`** and optional socket FD bridge.
- [Capability words](reference/capability-words.md) and [compatibility matrix](reference/compatibility-matrix.md) — ctx API and kernel feature-bit alignment.
- [Feature roadmap (canonical JSON)](data/feature-roadmap-canonical.json) and [feature roadmap (human)](reference/feature-roadmap.md) — planned work vs shipped surfaces.

---

## Maintainer quick path (build + parity)

When you change **`kernel/lib/init/`**, **`kernel/lib/boot/`**, **`kernel/lib/bare/`**, or **`kernel/bin/`**:

1. Run **`npm run build -w bare-os-coreutils`** and **`npm run build -w bare-os-bare-libs`** as needed.
2. Run **`npm run bundle:kernel`** (or full **`npm test`**, which bundles and verifies).
3. Keep **`packages/bare-os-seeder/kernel/`** identical to **`kernel/`** — **`scripts/verify-kernel-seeder-parity.mjs`** enforces this in **`pretest`**.

Details: [scripts/README.md](../scripts/README.md) (maintainer build order), [developer guide ch.7](../developer-guide/README.md).

## Choose your path

| I want to… | Start here |
| --- | --- |
| **Orient** quickly (two drives, boot, P2P, POSIX) | [Concepts](concepts/README.md) |
| **Run** the seeder and booter and use the shell | [User manual](../users-manual/README.md) |
| **Understand** architecture, protocol, and design intent | [Handbook](../handbook/README.md) |
| **Extend** `/bin`, scripts, Pear apps, or `ctx` | [Developer guide](../developer-guide/README.md) |
| **Look up** paths, env vars, wire formats, matrices | [Reference hub](reference/README.md) |
| **Define a term** or compare version fields | [Glossary](glossary.md) · [Compatibility matrix](reference/compatibility-matrix.md) |
| **Vault / identity posture** (operator) | [Vault threat model](security/vault-threat-model.md) |
| **Boot provenance and block-0 trust** (operator) | [Boot trust model](security/boot-trust-model.md) |
| **Fix a problem** quickly | [Troubleshooting router](troubleshooting.md) · [FAQ](faq.md) |
| **Edit docs** in this repo | [Contributing to documentation](CONTRIBUTING-DOCS.md) |

```mermaid
flowchart LR
  subgraph roles [By role]
    Run[Run and use]
    Think[Understand design]
    Build[Extend and ship]
    Spec[Specify and verify]
  end
  subgraph trees [Doc trees]
    UM[users-manual]
    HB[handbook]
    DG[developer-guide]
    RF[docs/reference + architecture]
  end
  Run --> UM
  Think --> HB
  Build --> DG
  Spec --> RF
  HB --> RF
  DG --> RF
```

---

## Documentation map

### User manual ([`users-manual/`](../users-manual/README.md))

Practical, sequential guide for operators.

| Chapter | Topic |
| --- | --- |
| [README](../users-manual/README.md) | Paths through the docs, how trees relate |
| [01 — What this is](../users-manual/01-what-this-is.md) | Two drives, P2P image, experimental status |
| [02 — Install](../users-manual/02-install-and-repository-layout.md) | Clone, `npm ci`, monorepo layout |
| [03 — Seeder and booter](../users-manual/03-running-seeder-and-booter.md) | Node vs Pear, two terminals |
| [04 — Shell and PATH](../users-manual/04-shell-path-and-scripts.md) | Commands and scripts without full `ctx` depth |
| [05 — Home, identity, vault](../users-manual/05-home-identity-and-vault.md) | Guest, `login`, vault snapshots |
| [06 — Help and man](../users-manual/06-help-man-and-documentation-map.md) | `man`, `help`, where each doc tree lives |
| [07 — Troubleshooting](../users-manual/07-troubleshooting-and-operations.md) | Common failures, env vars, CI |
| [08 — Further reading](../users-manual/08-further-reading.md) | Curated deep links |

### Handbook ([`handbook/`](../handbook/README.md))

Narrative spine: *why* and *how* the system fits together.

| Chapter | Topic |
| --- | --- |
| [00 — Preface](../handbook/00-preface.md) | Thesis, security frame, contributors |
| [01 — Introduction](../handbook/01-introduction.md) | Goals, vocabulary, clone-to-prompt |
| [02 — Blueprints](../handbook/02-blueprints.md) | Layers, trust, boot path |
| [03 — Protocol and disk](../handbook/03-protocol-and-disk.md) | MBR, swarm, Protomux, failure modes |
| [04 — Booter runtime](../handbook/04-the-booter-runtime.md) | `ctx`, VFS, shell, bridges |
| [05 — Identity, vault, HDMS](../handbook/05-identity-vault-and-hdms.md) | Accounts, vault, mounts |
| [06 — Kernel and binaries](../handbook/06-kernel-and-binaries.md) | `/boot/init.js`, coreutils |
| [07 — Operations](../handbook/07-operations-and-development.md) | CI, Pear, env, troubleshooting |
| [08 — Git on Bare OS](../handbook/08-git-on-bare-os.md) | isomorphic-git, HTTP modes |
| [09 — POSIX, shell, VFS](../handbook/09-posix-utilities-shell-and-vfs.md) | Coverage and divergences |
| [10 — Man pages and help](../handbook/10-manpages-and-online-help.md) | JSON DB, handbook ingest |
| [11 — Kernel program and research](../handbook/11-kernel-program-and-research.md) | Roadmap context, capability words |
| [12 — P2P stack vs POSIX](../handbook/12-p2p-stack-and-posix-boundaries.md) | Hypercore, swarm, Protomux vs declared POSIX profile |

### Developer guide ([`developer-guide/`](../developer-guide/README.md))

How-to for code inside and around the image.

| Doc | Topic |
| --- | --- |
| [01 — Two runtimes](../developer-guide/01-two-runtimes-host-vs-image.md) | Host vs in-image execution |
| [02 — `ctx`](../developer-guide/02-the-context-object.md) | Capability object map |
| [03 — Kernel boot](../developer-guide/03-kernel-boot-init.md) | `start(ctx)` |
| [04 — User scripts and PATH](../developer-guide/04-user-scripts-and-path.md) | `run(ctx, argv)` |
| [05 — Modules](../developer-guide/05-modules-and-imports.md) | No ESM on drive |
| [06 — Extending `/bin`](../developer-guide/06-extending-bin-coreutils.md) | Coreutils pipeline |
| [07 — Apps beyond shell](../developer-guide/07-apps-beyond-the-shell.md) | initd, cron, delegates |
| [08 — Testing](../developer-guide/08-testing-and-debugging.md) | `npm test`, Pear dev |
| [09 — Security](../developer-guide/09-security-and-trust.md) | Trust boundaries |
| [10 — Glossary and FAQ](../developer-guide/10-glossary-and-faq.md) | Dev-focused Q&A |
| [11 — Pear cookbook](../developer-guide/11-kernel-pear-cookbook.md) | Pear integration patterns |
| [12 — Bare modules](../developer-guide/12-bare-modules-and-pear-ecosystem.md) | `ctx.bare`, manifests |
| [13 — Privacy and telemetry](../developer-guide/13-privacy-telemetry-pii.md) | Scrub lists, PII posture |
| [20 — Guest TUI](../developer-guide/20-tui-and-sdk.md) | `ctx.tui` |
| [21 — Discord bots](../developer-guide/21-discord-bots.md) | `ctx.bare.discordJS`, `/bin/discord-bot` |
| [Extras](../developer-guide/README.md#reading-order) | Phase alignment, kernel program, naming, node→Bare map, ADRs |

### Reference and contracts (this tree)

| Resource | Purpose |
| --- | --- |
| [Reference README](reference/README.md) | Topic index (packages, env, data flow) |
| [KERNEL_CONTRACT](architecture/KERNEL_CONTRACT.md) | Booter vs image, boot steps, `ctx` API pointer |
| [POSIX_DECLARED_PROFILE](architecture/POSIX_DECLARED_PROFILE.md) | Issue 7–like contract: shell, XCU, VFS, proc “syscall” models |
| [kernel-subsystems](architecture/kernel-subsystems.md) | Subsystem boundaries |
| [ADRs](adr/README.md) | Architecture decisions |
| [Audit baseline](audit/PLACEHOLDER_BASELINE.md) | Incomplete surfaces (classified) |
| [OTA and bundles](deployment/OTA_AND_BUNDLES.md) | Release packaging notes |
| [Release checklist](release-checklist.md) | Pre-tag steps |
| [Schemas](schemas/) | Boot policy, telemetry, extension manifests |
| [Themes](themes/README.md) | Terminal theme packs |
| [Machine-readable data](data/README.md) | Roadmap JSON, terminology allowlist |
| [Holepunch catalog](bare-holepunch-catalog.json) | Optional `ctx.bare` bundle metadata |

### Topic index (canonical reference per subject)

When updating behavior, edit the **canonical** row first; handbook, user manual, and developer guide should **summarize and link** here instead of copying versioned tables or env inventories.

| Topic | Canonical spec / reference | Narrative (handbook) | Operator (users manual) | How-to (developer guide) |
| --- | --- | --- | --- | --- |
| Environment variables | [environment-and-posix-appendix.md](reference/environment-and-posix-appendix.md) | [Ch. 7 — Operations](../handbook/07-operations-and-development.md) | [Ch. 7 — Troubleshooting](../users-manual/07-troubleshooting-and-operations.md) | [Ch. 11 — Pear cookbook](../developer-guide/11-kernel-pear-cookbook.md) |
| `ctx` / booter runtime | [package-bare-os-booter.md](reference/package-bare-os-booter.md), [ctx-api-versioning.md](reference/ctx-api-versioning.md), [shell-completion-and-repl-editor.md](reference/shell-completion-and-repl-editor.md) | [Ch. 4 — Booter runtime](../handbook/04-the-booter-runtime.md) | — | [Ch. 2 — `ctx`](../developer-guide/02-the-context-object.md) |
| Protocol / MBR / Protomux | [package-bare-os-protocol.md](reference/package-bare-os-protocol.md) | [Ch. 3 — Protocol and disk](../handbook/03-protocol-and-disk.md) | [Ch. 1–3](../users-manual/01-what-this-is.md) | — |
| Identity / vault / HDMS | [vault-threat-model.md](security/vault-threat-model.md), [package-bare-os-booter.md](reference/package-bare-os-booter.md) (guest `/.bare` policy) | [Ch. 5](../handbook/05-identity-vault-and-hdms.md) | [Ch. 5](../users-manual/05-home-identity-and-vault.md) | [Ch. 9 — Security](../developer-guide/09-security-and-trust.md) |
| POSIX / syscalls / sockets | [POSIX_DECLARED_PROFILE.md](architecture/POSIX_DECLARED_PROFILE.md), [posix-syscall-facade-map.md](reference/posix-syscall-facade-map.md), [syscall-socket-contract.md](reference/syscall-socket-contract.md) | [Ch. 9](../handbook/09-posix-utilities-shell-and-vfs.md) | — | [Ch. 8 — Testing](../developer-guide/08-testing-and-debugging.md) |
| Kernel program / capability words | [feature-roadmap.md](reference/feature-roadmap.md), [kernel-capabilities-index.md](reference/kernel-capabilities-index.md), [kernel-program.md](../developer-guide/kernel-program.md) | [Ch. 11–12](../handbook/11-kernel-program-and-research.md) | — | [kernel-program.md](../developer-guide/kernel-program.md) |
| Tier-1 `/bin` / coreutils | [package-bare-os-coreutils-and-ci.md](reference/package-bare-os-coreutils-and-ci.md), **`packages/bare-os-coreutils/lib/commands.mjs`** | [Ch. 6–9](../handbook/06-kernel-and-binaries.md) | [Ch. 4](../users-manual/04-shell-path-and-scripts.md) | [Ch. 6 — Extending `/bin`](../developer-guide/06-extending-bin-coreutils.md) |
| **`agent`** (QVAC / REST coding agent) and **`chat`** (swarm chat) | [HTTP: curl and wget](reference/http-curl-and-wget.md) (**`ctx.httpFetch`** for REST + **`web_fetch`**); **`man agent`** / **`man chat`** ([`packages/bare-os-coreutils/man/pages/`](../packages/bare-os-coreutils/man/pages/)); [bare-os-coreutils README](../packages/bare-os-coreutils/README.md) | [Ch. 4](../handbook/04-the-booter-runtime.md), [Ch. 6](../handbook/06-kernel-and-binaries.md), [Ch. 9](../handbook/09-posix-utilities-shell-and-vfs.md) | [Ch. 4 — Shell](../users-manual/04-shell-path-and-scripts.md), [Ch. 6 — `man`](../users-manual/06-help-man-and-documentation-map.md) | [Ch. 2 — `ctx`](../developer-guide/02-the-context-object.md) (`httpFetch`, `bareOsQvac*`), [Ch. 6](../developer-guide/06-extending-bin-coreutils.md) (preamble), [Ch. 21](../developer-guide/21-discord-bots.md) (`/agent`) |
| Discord bots | [Environment appendix — `DISCORD_*`](reference/environment-and-posix-appendix.md); **`man discord-bot`** | — | — | [Ch. 21 — Discord bots](../developer-guide/21-discord-bots.md) |

### Contract bump checklist

When you change **`BARE_OS_CTX_API_VERSION`**, **`/proc`** JSON schemas, **`BARE_OS_PROTOCOL_*`** / package semver, **`BARE_OS_POSIX_PROFILE_VERSION`**, or kernel capability words, update every dependent artifact in one pass:

1. [README.md](../README.md) and [DOCUMENTATION.md](../DOCUMENTATION.md) if the version string or install story changes.
2. [Handbook](../handbook/README.md) chapters touched by behavior (especially ch.3–4, 6, 9).
3. [Users manual](../users-manual/README.md) and [Developer guide](../developer-guide/README.md) for operator or extension workflows.
4. [docs/reference/compatibility-matrix.md](reference/compatibility-matrix.md) and any affected topic pages under [docs/reference/](reference/README.md).
5. Package changelogs: [packages/bare-os-booter/CHANGELOG.md](../packages/bare-os-booter/CHANGELOG.md), [packages/bare-os-protocol/CHANGELOG.md](../packages/bare-os-protocol/CHANGELOG.md) when applicable.
6. [docs/schemas/](schemas/) and [docs/architecture/POSIX_DECLARED_PROFILE.md](architecture/POSIX_DECLARED_PROFILE.md) when proc, telemetry, or POSIX profiles move.
7. [kernel/etc/bare-os/syscalls.example.json](../kernel/etc/bare-os/syscalls.example.json) and the [seeder mirror](../packages/bare-os-seeder/kernel/) after kernel contract edits; run **`node scripts/verify-kernel-seeder-parity.mjs`**.
8. Regenerate **`docs/reference/ctx-client-helper.generated.ts`** via root **`pretest`** ( **`gen-ctx-client-helper.mjs`** ).

---

## Root stubs and Pear

- First run: [Get started](./get-started.md) · Doc map: [Sitemap](./sitemap.md).
- Legacy monolith map: [DOCUMENTATION.md](../DOCUMENTATION.md) (points here and to reference).
- Pear channels and host env: [PEAR-RUN.md](./PEAR-RUN.md).
- Guest Pear + App Store (in-shell author, release, install, launch): [guides/guest-pear-and-appstore-workflow.md](./guides/guest-pear-and-appstore-workflow.md).
- Repository quick start: [README.md](../README.md).

---

## Contributing

Code and automation: [scripts/README.md](../scripts/README.md). Documentation style and verifiers: [CONTRIBUTING-DOCS.md](CONTRIBUTING-DOCS.md).