snxraven/bare-operating-system
1
1
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs
CI / test (push) Failing after 4m59s
Details

Browse Source
This commit is contained in:
Raven Scott
2026-04-02 21:39:31 -04:00
parent 1af6dbc9b3
commit c3b3dc188e
2 changed files with 441 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DOCUMENTATION.md
+431
View File
@@ -0,0 +1,431 @@
# bare-operating-system — full codebase reference
This document describes **every first-party file** in the repository: purpose, contents, dependencies, and how pieces connect. It does **not** enumerate third-party code under `node_modules/` or VCS metadata under `.git/`. Runtime directories `data/` and `packages/*/.test-data/` are **gitignored** and hold Corestore/RocksDB state when you run the apps or tests; they are not source files.
---
## 1. Repository tree (source only)
```
bare-operating-system/
├── package.json # Root workspace manifest
├── package-lock.json # Locked dependency tree (npm, all workspaces)
├── README.md # User-facing overview and runbook
├── DOCUMENTATION.md # This file
├── LICENSE # Apache-2.0 notice
├── .gitignore # Ignore rules
├── .prettierrc # Prettier formatting defaults
├── .github/
│ └── workflows/
│ └── ci.yml # GitHub Actions CI
├── kernel/ # Files staged into the system Hyperdrive
│ ├── init.js
│ ├── bin/
│ │ ├── echo
│ │ └── help
│ └── etc/
│ └── os-release
└── packages/
├── bare-os-protocol/ # Shared protocol + MBR + seed channel
│ ├── package.json
│ ├── index.js
│ ├── constants.js
│ ├── test.js
│ └── lib/
│ ├── messages.js
│ └── channel.js
├── bare-os-seeder/ # Publishes OS drive + MBR
│ ├── package.json
│ └── index.js
└── bare-os-booter/ # Boots from peers or local seed
├── package.json
├── index.js
├── test.js
└── lib/
├── swarm-disk.js
└── kernel-runner.js
```
---
## 2. Root: [package.json](package.json)
| Field | Value / meaning |
| ----------------- | -------------------------------------------------------- |
| `name` | `bare-operating-system` |
| `private` | `true` — not published as a single npm package |
| `type` | `module` — ESM |
| `workspaces` | `["packages/*"]` — npm workspaces for the three packages |
| `scripts.test` | Runs `npm run test --workspaces --if-present` |
| `scripts.format` | `prettier --write .` |
| `scripts.lint` | `prettier --check .` |
| `engines.node` | `>=20` |
| `devDependencies` | `prettier@^3.4.2` |
No runtime dependencies at the root; all stack deps live in workspace packages.
---
## 3. Root: [package-lock.json](package-lock.json)
- **Format**: npm lockfile v3 (`lockfileVersion: 3`).
- **Role**: Pins exact versions of the full install graph (root + `packages/bare-os-protocol`, `bare-os-seeder`, `bare-os-booter` and all transitive dependencies: `hyperdrive`, `corestore`, `hyperswarm`, `protomux`, `brittle`, Bare-related packages, native addons such as `rocksdb-native`, etc.).
- **Workspaces**: Lists workspace package paths and links workspace packages to `"node_modules/bare-os-protocol"` etc.
- **Not reproduced here line-by-line** — it is thousands of lines; use `npm ls` or open the file for the exact tree.
---
## 4. Root: [README.md](README.md)
User-oriented documentation: project goal, layout table, prerequisites (Node 20+, Pear/Bare), `npm ci`, `npm test`, how to run seeder and booter with `node index.js`, environment variables, Corestore path semantics (repo-root `data/`), placeholder `pear://` table, protocol summary (`bare-os-v1`, MBR layout), license pointer.
---
## 5. Root: [LICENSE](LICENSE)
Short Apache License, Version 2.0 header: copyright year 2026, standard AS-IS disclaimer, link to <http://www.apache.org/licenses/LICENSE-2.0>.
---
## 6. Root: [.gitignore](.gitignore)
Ignores:
- `node_modules/`
- `data/` (Corestore for seeder/booter at repo root)
- `coverage/`
- `.DS_Store`
- `*.log`
- `.pear/`
- `packages/*/.test-data/` (brittle test Corestore dirs)
---
## 7. Root: [.prettierrc](.prettierrc)
JSON: `semi: false`, `singleQuote: true`, `trailingComma: "none"`.
---
## 8. CI: [.github/workflows/ci.yml](.github/workflows/ci.yml)
- **Triggers**: `push` and `pull_request` to `main`.
- **Job `test`**: `ubuntu-latest`, checkout, `actions/setup-node@v4` with Node 20 and npm cache on `package-lock.json`, then `npm ci` and `npm test`.
---
## 9. Kernel sources (staged into Hyperdrive)
These files are **read from disk by the seeder** and written into the system drive with **no temp directory**; paths are determined by [packages/bare-os-seeder/index.js](packages/bare-os-seeder/index.js) `stageKernelTree()`.
### 9.1 [kernel/init.js](kernel/init.js)
- **Staged as**: `/boot/init.js`
- **Contract**: Must define a top-level `async function start(ctx)` (see [kernel-runner.js](packages/bare-os-booter/lib/kernel-runner.js)).
- **Behavior**:
- Reads `/etc/os-release` from `drive`, logs UTF-8 string via `b4a.toString`.
- Logs a one-line help string.
- Loop: `readLine('bare-os> ')`. If `null`, break. Empty line skips. `exit` breaks. Otherwise `await execLine(t)` (booter dispatches to `/bin/<cmd>`).
### 9.2 [kernel/bin/echo](kernel/bin/echo)
- **Staged as**: `/bin/echo`
- **Contract**: `async function run(ctx, argv)` — argv[0] is command name.
- **Behavior**: `ctx.console.log(argv.slice(1).join(' '))`.
### 9.3 [kernel/bin/help](kernel/bin/help)
- **Staged as**: `/bin/help`
- **Contract**: `async function run(ctx, _argv)`.
- **Behavior**: Logs `Commands: help, echo <text>, exit`.
### 9.4 [kernel/etc/os-release](kernel/etc/os-release)
- **Staged as**: `/etc/os-release`
- **Format**: Plain text key=value lines (familiar from Linux):
```
NAME="BareOS"
VERSION="0.1.0"
VARIANT="hyperdrive-only"
```
---
## 10. Package: `bare-os-protocol`
### 10.1 [packages/bare-os-protocol/package.json](packages/bare-os-protocol/package.json)
| Item | Detail |
| --------------------------- | -------------------------------------------------------------------------- |
| `name` | `bare-os-protocol` |
| `version` | `0.1.0` |
| `main` / `exports["."]` | `./index.js` |
| `exports["./constants.js"]` | `./constants.js` (for `bare-os-protocol/constants.js` imports) |
| `exports["./messages"]` | `./lib/messages.js` |
| `scripts.test` | `brittle-bare test.js` |
| `scripts.test:node` | `brittle-node test.js` |
| `engines.bare` | `>=2.0.0` |
| `dependencies` | `b4a`, `compact-encoding`, `hypercore-crypto` |
| `devDependencies` | `brittle` |
| `imports` | Conditional `fs` / `path` / `events` → `bare-*` under Bare, Node `default` |
### 10.2 [packages/bare-os-protocol/index.js](packages/bare-os-protocol/index.js)
Re-exports from `./constants.js`: `PROTOCOL_NAME`, `TOPIC_STRING`, `BLOCK_SIZE`, `MBR_MAGIC`, `topicKey`, `buildMbr`, `parseMbr`.
Re-exports from `./lib/channel.js`: `setupSeedChannel`.
### 10.3 [packages/bare-os-protocol/constants.js](packages/bare-os-protocol/constants.js)
| Export | Definition |
| ------------------------------------- | --------------------------------------------------------------------------------------------------- |
| `PROTOCOL_NAME` | `'bare-os-v1'` — Protomux channel name |
| `TOPIC_STRING` | `'bare-os-v1'` — input to swarm topic hash |
| `BLOCK_SIZE` | `512` — MBR size |
| `MBR_MAGIC` | `b4a.from('BIOS')` — first 4 bytes of MBR |
| `topicKey(b4aMod?)` | `crypto.hash(b4aMod.from(TOPIC_STRING))` — 32-byte Hyperswarm topic |
| `buildMbr(primaryKey, failoverKeys?)` | Allocates 512 bytes, writes magic at 0, primary 32-byte key at offset 8, optional keys at 40 and 72 |
| `parseMbr(mbr)` | Validates length ≥ 104, magic `BIOS`, returns `{ keys: Uint8Array[] }` (non-zero 32-byte slots) |
### 10.4 [packages/bare-os-protocol/lib/messages.js](packages/bare-os-protocol/lib/messages.js)
`compact-encoding` schemas (for tests and documentation parity with SwarmDisk):
- `msgRead` — placeholder object (`encoding: c.uint32`, `onmessage: null`).
- `msgDataEncoding` — `{ index: uint32, data: buffer }`.
- `msgSearchReqEncoding` — `{ id: uint32, query: string }`.
- `msgSearchResEncoding` — `{ id: uint32, matches: string[] }`.
- `msgRpcReqEncoding` — `{ id, module, method, args: string[] }`.
- `msgRpcResEncoding` — `{ id, success: bool, result, error: string }`.
### 10.5 [packages/bare-os-protocol/lib/channel.js](packages/bare-os-protocol/lib/channel.js)
**`setupSeedChannel(mux, localRAM, replicateDrive)`**
- Creates Protomux channel with `protocol: PROTOCOL_NAME`.
- **Message 0**: `uint32` read index → if `localRAM.get(index)`, send on message 1.
- **Message 1**: `msgDataEncoding` (inbound only on client).
- **Message 2**: `c.buffer` — after `chan.open()`, sends 250-byte bitfield with bit 0 set (gossip stub).
- **Message 3**: search request → responds on message 4 with empty `matches`.
- **Message 4**: search response encoding.
- **Message 5**: RPC request → responds on message 6 with failure `"RPC not implemented on seeder"`.
- **Message 6**: RPC response encoding.
- Calls `replicateDrive(mux.stream)` (typically `drive.replicate(stream)`).
### 10.6 [packages/bare-os-protocol/test.js](packages/bare-os-protocol/test.js)
Brittle tests:
1. `topicKey` length 32 and stable.
2. `topicKey` ≠ hash of `'other-topic'`.
3. `buildMbr` + `parseMbr` roundtrip one key.
4. `msgDataEncoding` encode/decode.
5. `msgSearchReqEncoding` encode/decode.
6. `TOPIC_STRING === 'bare-os-v1'`.
---
## 11. Package: `bare-os-seeder`
### 11.1 [packages/bare-os-seeder/package.json](packages/bare-os-seeder/package.json)
| Item | Detail |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `main` | `./index.js` |
| `scripts.start` / `dev` | `bare index.js` (Pear/Bare entry) |
| `dependencies` | `bare-os-protocol` (workspace `*`), `b4a`, `compact-encoding`, `corestore`, `hyperdrive`, `hyperswarm`, `protomux`, `safety-catch` |
| `engines.bare` | `>=2.0.0` |
| `pear.name` | `bare-os-seeder` |
| `pear.stage.ignore` | `.git`, `test`, `coverage`, `.DS_Store`, `node_modules/.bin`, `node_modules/.package-lock.json` |
| `imports` | `fs`, `fs/promises`, `path`, `url` — Bare vs `node:` shims |
### 11.2 [packages/bare-os-seeder/index.js](packages/bare-os-seeder/index.js)
**Paths**
- `repoRoot` = two levels up from this file → monorepo root.
- `corestorePath()` = `BARE_OS_SEED_STORE` or `repoRoot/data/corestore-seeder`.
- `kernelRoot` = `BARE_OS_KERNEL_ROOT` (resolved) or `repoRoot/kernel`.
**`stageKernelTree(drive, kernelRoot)`**
- Recursive `walk(rel)` using `fs/promises` `readdir` + `readFile`.
- **Drive path mapping**:
- `init.js` → `/boot/init.js`
- `bin/<name>` → `/bin/<name>` (POSIX slashes)
- `etc/...` → `/etc/...`
- Any other file → `/<subRel>` with slashes normalized
- `drive.put(drivePath, b4a.from(raw))` for each file.
**`main()`**
1. Optional `console.clear()`.
2. `Corestore(corestorePath())`, `Hyperdrive(store)`, `ready()`.
3. `stageKernelTree`.
4. `localRAM = Map`, `buildMbr(drive.key)`, `localRAM.set(0, mbr)`.
5. `Hyperswarm`, on `connection`: `Protomux(socket)`, `setupSeedChannel(mux, localRAM, stream => drive.replicate(stream))`.
6. `swarm.join(topicKey())`, `swarm.join(drive.discoveryKey)`, `flush()`.
7. Logs topic prefix hex, full drive key hex, Corestore path.
**Entry**: `main().catch(safetyCatch)`.
---
## 12. Package: `bare-os-booter`
### 12.1 [packages/bare-os-booter/package.json](packages/bare-os-booter/package.json)
| Item | Detail |
| ----------------------- | ---------------------------------------------- |
| `main` | `./index.js` |
| `scripts.start` / `dev` | `bare index.js` |
| `scripts.test` | `brittle-node test.js` |
| `dependencies` | Same hyperstack as seeder + `bare-os-protocol` |
| `devDependencies` | `brittle` |
| `imports` | `path`, `url` Bare/Node conditional |
### 12.2 [packages/bare-os-booter/index.js](packages/bare-os-booter/index.js)
**Note:** Imports are ordered as: external modules, `repoRoot` from `__dirname`, then `./lib/...` (SwarmDisk, kernel-runner).
**`bootStorePath()`** — `BARE_OS_BOOT_STORE` or `repoRoot/data/corestore-booter`.
**`createReadLine()`**
- If `BARE_OS_SKIP_REPL === '1'`: returns `async () => null`.
- Else tries `node:readline` `createInterface` + `question` per prompt.
- On failure: warns and returns `async () => null`.
**`executeKernel(disk, store, swarm, initSource)`** (`store`/`swarm` unused but kept for signature symmetry / future use)
- Builds `ctx`: `disk`, `drive`, `personalDrive`, `console`, `b4a`, `topic: topicKey()`, `readLine`, `execLine` → `runBinCommand(disk.drive, parts, ctx)`.
- Sets `disk.os` with stub `searchLocal` ([]) and `execRpc` (`''`).
- `runKernelFromSource(b4a.toString(initSource), ctx)`.
**`bootFromPeers(disk, store, swarm)`**
- `disk.read(0)` → `parseMbr`.
- For each key: `Hyperdrive(store, driveKey)`, `ready()`, replicate on all `disk.peers`’ mux streams, join drive discovery, `findingPeers` + `swarm.flush`, poll up to 30×200ms for `/boot/init.js`.
- On success: `initPersonalDrive`, `executeKernel`.
**`bootLocal(disk, store, swarm)`**
- `Corestore(BARE_OS_LOCAL_SEED || repoRoot/data/corestore-seeder)`, `Hyperdrive(seedStore)` (same store as seeder’s default layout).
- Poll for `/boot/init.js`, then `initPersonalDrive`, `executeKernel`.
**`main()`**
- `Corestore(bootStorePath())`, `Hyperswarm`, `SwarmDisk`, join `topicKey()`.
- Wait loop: 500ms steps until `disk.peers.size > 0` or `BARE_OS_PEER_WAIT_MS` (default 8000).
- If peers: `bootFromPeers`; else `bootLocal`.
- **`finally`**: close `personalDrive`, `disk.drive`, `swarm.destroy()`, `store.close()` (each in try/catch).
**Entry**: `main().catch(safetyCatch)`.
### 12.3 [packages/bare-os-booter/lib/swarm-disk.js](packages/bare-os-booter/lib/swarm-disk.js)
**`SwarmDisk` class**
**State:** `localRAM`, `peers` (Set of `{ chan, mux, socket, id }`), `pendingReads`, `pendingSearches`, `pendingRpc`, counters, `drive`, `personalDrive`, `os`.
**`initPersonalDrive(store, swarm, Hyperdrive)`**
- `store.namespace('bare-os-personal-v1')`, new `Hyperdrive(localStore)`, ensure writable, log id prefix, `swarm.join(personalDrive.discoveryKey)`.
**`addPeer(mux, socket)`**
- Builds `context` with `onread` / `ondata` / `ongossip` / `onsearchreq` / `onsearchres` / `onrpcreq` / `onrpcres` wired to Protomux messages **0–6** (same order as seeder + hyper-os style): read request, data, gossip buffer, search req/res, RPC req/res.
- `chan.open()`, track peer, handshake-based `peer.id`, remove peer on `mux.stream` `close`.
- If `this.drive` / `this.personalDrive` set, `replicate(mux.stream, { live: true, download: true })` for system drive.
**`read(index)`**
- If `localRAM.has(index)`, return cached.
- Else broadcast message 0 to all peers, single consumer callback from message 1, 10s timeout.
**`search(query)`**
- Fan-out message 3 to all peers with shared id (actually each peer gets same id from counter — **one id per `search()` call**, registered in `pendingSearches`; concurrent searches could collide in id space; documented as current behavior).
### 12.4 [packages/bare-os-booter/lib/kernel-runner.js](packages/bare-os-booter/lib/kernel-runner.js)
- **`AsyncFunction`** = `Object.getPrototypeOf(async function () {}).constructor`.
**`runKernelFromSource(source, ctx)`**
- `new AsyncFunction('ctx', source + guard + 'return start(ctx)')` where guard checks `typeof start === 'function'`.
**`runBinCommand(drive, argv, ctx)`**
- Loads `/bin/<argv[0]>`, builds `AsyncFunction('ctx','argv', ...)` requiring `run` function, invokes `run(ctx, argv)`.
- Unknown command: `ctx.console.log('unknown command: ...')`.
### 12.5 [packages/bare-os-booter/test.js](packages/bare-os-booter/test.js)
- **`testCorestoreDir(name)`** — unique directory under `__dirname/.test-data/` (must exist parent via `mkdirSync`).
Tests:
1. **`runKernelFromSource`** with inline `start` pushing to `ctx.calls`.
2. **`runBinCommand`** with `/bin/hello` script pushing joined argv to `ctx.out`; `store.close()`, `rmSync` dir.
3. **Hyperdrive** put/get `/boot/init.js` on Corestore; `drive.close()`, `rmSync`.
Uses Node builtins: `node:fs`, `node:path`, `node:url`.
---
## 13. End-to-end data flow
```mermaid
sequenceDiagram
participant Seeder as bare_os_seeder
participant Swarm as Hyperswarm
participant Booter as bare_os_booter
participant Sys as System_Hyperdrive
participant Pers as Personal_Hyperdrive
Seeder->>Sys: stage kernel files and MBR key in RAM map
Seeder->>Swarm: join topic plus drive discovery
Booter->>Swarm: join topic
Booter->>Seeder: Protomux bare_os_v1
Booter->>Seeder: read block 0 MBR
Booter->>Sys: open with key replicate
Booter->>Pers: new writable drive namespace
Booter->>Booter: AsyncFunction start ctx from init.js
```
---
## 14. Environment variables (complete list)
| Variable | Used by | Meaning |
| ---------------------- | ------- | --------------------------------------------------------------------- |
| `BARE_OS_KERNEL_ROOT` | Seeder | Absolute path to kernel tree (default: `repo/kernel`) |
| `BARE_OS_SEED_STORE` | Seeder | Corestore directory (default: `repo/data/corestore-seeder`) |
| `BARE_OS_BOOT_STORE` | Booter | Corestore for boot side (default: `repo/data/corestore-booter`) |
| `BARE_OS_LOCAL_SEED` | Booter | Corestore path for local boot (default: `repo/data/corestore-seeder`) |
| `BARE_OS_PEER_WAIT_MS` | Booter | Max ms to wait for ≥1 peer before local boot (default `8000`) |
| `BARE_OS_SKIP_REPL` | Booter | If `1`, readline returns null — non-interactive exit |
---
## 15. What is intentionally out of scope in this repo
- **No** `node_modules` documentation (upstream packages).
- **No** line-by-line `package-lock.json` (machine-generated).
- **No** committed `data/` or `.test-data/` binary stores (runtime artifacts).
---
## 16. Version and tooling summary
| Tool | Role |
| ----------------- | ------------------------------------------------------------ |
| Node.js 20+ | Install, `brittle-node` tests, `node index.js` dev runs |
| Bare ≥2 (engines) | Intended runtime for `pear run` / `bare index.js` |
| Prettier | Format/lint at root |
| Brittle | Test runner (`brittle-bare` protocol, `brittle-node` booter) |
| GitHub Actions | `npm ci` + `npm test` on `main` |
This file was generated to satisfy **full** source documentation for the **bare-operating-system** first-party tree as of the commit that added `DOCUMENTATION.md`.
README.md
+4 -4
View File
@@ -4,10 +4,12 @@ Distributed system image in **Hyperdrive**, discovered over **Hyperswarm**, with
This project is experimental research software, not a production OS.
**Full codebase reference** (every first-party file, protocols, env vars, tests): [DOCUMENTATION.md](DOCUMENTATION.md).
## Repository layout
| Path | Role |
|------|------|
| ------------------------------------------------------ | -------------------------------------------------------------------------------- |
| [kernel/](kernel/) | Source tree mirrored into the system drive (`/boot/init.js`, `/bin/*`, `/etc/*`) |
| [packages/bare-os-protocol](packages/bare-os-protocol) | `bare-os-v1` topic, MBR layout, Protomux seed channel helper |
| [packages/bare-os-seeder](packages/bare-os-seeder) | Pear app: stage kernel → Hyperdrive, serve MBR block 0, replicate |
@@ -43,7 +45,6 @@ Runs workspace tests (`brittle-node` / `brittle-bare` where configured).
```
Optional:
- `BARE_OS_KERNEL_ROOT` — absolute path to a kernel tree (defaults to repo `kernel/`).
- `BARE_OS_SEED_STORE` — Corestore directory (default `./data/corestore-seeder`).
@@ -55,7 +56,6 @@ Runs workspace tests (`brittle-node` / `brittle-bare` where configured).
```
Optional:
- `BARE_OS_PEER_WAIT_MS` — ms to wait for peers (default `8000`).
- `BARE_OS_LOCAL_SEED` — Corestore path for **local boot** when no peers (default `./data/corestore-seeder`).
- `BARE_OS_SKIP_REPL=1` — non-interactive kernel (CI / automation).
@@ -71,7 +71,7 @@ Distribution uses Pear’s Hyperdrive staging. After you run **`pear stage`** (a
Replace the placeholders below with the keys from your own staging output:
| App | Placeholder |
|-----|-------------|
| ------ | -------------------------- |
| Seeder | `pear://<YOUR_SEEDER_KEY>` |
| Booter | `pear://<YOUR_BOOTER_KEY>` |