Updates to HDMS

This commit is contained in:
Raven Scott
2026-04-05 21:52:22 -04:00
parent cd2f9209a0
commit 028725c113
13 changed files with 320 additions and 42 deletions
+23 -5
View File
@@ -11,7 +11,7 @@ This chapter covers **who the session is** (guest vs unlocked), **where keys liv
1. **Boot** — The booter applies **guest** defaults: `USER=guest`, `HOME=/home/guest`, no Ed25519 material in the environment. You can already read the **system** image and write **guest-scoped** paths on the **personal** drive.
2. **`login`** — You provide a passphrase. If the account exists, the booter decrypts **`/.bare/account`** and derives session keys; if **`login --new`**, it mints a new **Ed25519** keypair and writes a **v2** blob (see below).
3. **Unlocked**`HOME` moves under **`/home/<pubkey-prefix>`**, `BARE_OS_PUBLIC_KEY` is set, and **HDMS** may **attach** extra Hyperdrives registered on the personal drive.
4. **`hdms mount …`** — Writable or read-only drives appear under **`/mnt/<label>`**; the registry JSON on the personal drive is the source of truth.
4. **`hdms create` / `hdms add`** — Writable or read-only drives appear under **`/mnt/<label>`**; the registry JSON on the personal drive is the source of truth.
5. **`logout`** — Sensitive material is zeroed; you return to **guest**. **`logout --save`** or **`savevault`** can snapshot selected paths into **`/.bare/vault/`** as encrypted records.
**Pairing hints (operators):** when the host sets **`BARE_OS_AUTOPASS_INVITE_URL`**, the guest exposes a JSON hint at **`/proc/bare_os/hdms_hints.json`**. The guest does **not** open network URLs automatically; hosts or operators consume the hint out-of-band. **`/proc/bare_os/hdms_health.json`** (and flat alias) summarizes **non-secret** mount/registry metadata (counts, labels) for dashboards—no key material.
@@ -71,8 +71,14 @@ The personal drive still persists: guest data is **not** anonymous to the drive
- Registry JSON on the **personal** drive: **`/.bare/hdms/registry.json`**
- **Writable** drives: new Corestore namespace + Hyperdrive, label, replicate to swarm
- **Read-only** drives: open by key string
- **invite / pair** — uses **Autopass** (static ESM import for Pear tracing)
- **Read-only** drives: open by key string (`hdms add <label> <z32-key>`)
- **invite / pair** — uses **Autopass** (static ESM import for Pear tracing) for blind pairing; optionally embeds an HDMS **drive offer** so the peer can auto-register a read-only mount
**Invite with a drive label:** **`hdms invite [--read-only] <label>`** (the label must be an existing **writable** HDMS mount). The booter stores **`{ label, key }`** (z32 public key) in an Autopass record (**`bare-os-hdms/pending-share`**) before minting the z32 invite string. **`--read-only`** applies to the peers **Autopass writer** role in the shared Autobase, **not** to whether the Hyperdrive at **`/mnt`** is writable. HDMS mounts created from a public key are always **read-only** replicas.
**Pair and auto-mount:** **`hdms pair <invite>`** (invite only — no local mount name argument) waits for **BlindPairing** to reach the inviter (default cap **`BARE_OS_HDMS_PAIR_WAIT_MS`**, 120000 ms — otherwise the shell returns a timeout error instead of hanging). Then it polls for the pending share (default **`BARE_OS_HDMS_PAIR_SHARE_WAIT_MS`**, 25000 ms). If present, the peer runs the same path as **`hdms add`** using the **inviters label** under **`/mnt/<inviter-label>`**. If that label is already taken on the peers machine, the booter picks **`label-2`**, **`label-3`**, … up to **`label-99`** (63-character label cap). The inviter must stay **online** with a **matching Hyperswarm bootstrap** to the peer or pairing never completes.
**Invite without a label:** **`hdms invite`** or **`hdms invite --read-only`** only pairs Autopass; the pending share key is cleared so no stale drive offer remains. The peer will not get an automatic **`/mnt`** mount.
**`assertLoggedIn`** requires **`ctx.identity.state === 'unlocked'`** and active controller—guests can list mounts that are already open but cannot **mutate** registry until login.
@@ -87,13 +93,15 @@ The personal drive still persists: guest data is **not** anonymous to the drive
[alice@bare:/home/a1b2c3d4e5f6] > hdms list
(no mounts)
[alice@bare:/home/a1b2c3d4e5f6] > hdms create-workspace notes
[alice@bare:/home/a1b2c3d4e5f6] > hdms create notes
Created notes key=
[alice@bare:/home/a1b2c3d4e5f6] > ls /mnt
notes
[alice@bare:/home/a1b2c3d4e5f6] > touch /mnt/notes/hello.txt
```
**Sharing that drive with a peer (illustrative):** inviter runs **`hdms invite notes`**, sends the printed z32 token out-of-band; peer runs **`hdms pair <token>`** only — the read-only mount appears at **`/mnt/notes`** (or **`/mnt/notes-2`** if **`notes`** already exists locally).
Exact subcommands and flags: **`man hdms`** after a coreutils build.
```mermaid
@@ -110,6 +118,16 @@ flowchart LR
Extra --> Mnt["/mnt labels"]
```
### Manual QA checklist (HDMS invite with drive)
Use two unlocked sessions with the **same Hyperswarm bootstrap** (or reachable peers).
1. **Inviter:** **`hdms create a`**, then **`hdms invite a`** — copy the z32 invite line.
2. **Peer:** **`hdms pair <invite>`** — expect **`hdms list`** to include **`a`**, **`/mnt/a`** read-only, and a line confirming the HDMS read-only mount (after replication).
3. **Peer (label collision):** create a local mount **`hdms create a`** first, then **`hdms pair <invite>`** from an inviter who shared label **`a`** — expect **`/mnt/a-2`** (or next free suffix) and a log line that the inviters label was in use.
4. **Inviter:** **`hdms invite`** with **no** label — peer **`hdms pair`** should pair Autopass but log that **no** HDMS drive was on the invite.
5. **Stale offer:** after **`hdms invite foo`**, run **`hdms invite`** without a label before the peer pairs — pending share should be cleared (peer should not mount **`foo`** from an old token unless the inviter re-issued an invite with **`foo`**).
---
## Host pairing hints (autopass-style)
+1 -1
View File
@@ -1,7 +1,7 @@
{
"schema": 2,
"profileId": "bare-os-posix-like",
"generatedAt": "2026-04-05T21:35:38.224Z",
"generatedAt": "2026-04-06T01:43:58.480Z",
"note": "Sparse POSIX Issue 7 coverage hints for /bin utilities. Omitted command names are not yet profiled here.",
"commandIndex": [
{
+1 -1
View File
@@ -1,6 +1,6 @@
{
"schema": 1,
"atMs": 1775424938224,
"atMs": 1775439838480,
"commands": [
"arch",
"awk",
File diff suppressed because one or more lines are too long
+2 -1
View File
@@ -6965,6 +6965,7 @@
"bare-url": "^2.4.0",
"compact-encoding": "^2.18.0",
"corestore": "^7.2.1",
"hyperbee": "^2.11.1",
"hypercore-id-encoding": "^1.3.0",
"hyperdrive": "^13.3.2",
"hyperswarm": "^4.17.0",
@@ -7131,7 +7132,7 @@
"version": "0.1.0"
},
"packages/bare-os-protocol": {
"version": "0.9.0",
"version": "0.9.1",
"dependencies": {
"b4a": "^1.6.7",
"compact-encoding": "^2.18.0",
@@ -330,13 +330,43 @@ export async function loadOsFromPeers(disk, store, swarm, splash) {
}
splash.setPhase('Reading MBR from swarm…')
splash.log('Loading block 0 (MBR)')
const mbr = await disk.read(0)
const { keys } = parseMbr(mbr)
disk.mbrKeysHex = keys.map((k) => b4a.toString(k, 'hex'))
const mbrMaxAttempts = 6
const mbrRetryDelayMs = 400
let mbr = /** @type {Uint8Array | null} */ (null)
/** @type {Uint8Array[] | null} */
let mbrKeys = null
let lastMbrErr = ''
for (let attempt = 1; attempt <= mbrMaxAttempts; attempt++) {
try {
splash.log(
attempt === 1
? 'Loading block 0 (MBR)'
: `MBR read retry ${attempt - 1}/${mbrMaxAttempts - 1}`
)
const block = await disk.read(0)
const { keys } = parseMbr(block)
if (!keys.length) throw new Error('MBR has no drive keys')
mbr = block
mbrKeys = keys
lastMbrErr = ''
break
} catch (e) {
lastMbrErr = (e && e.message) || String(e)
splash.log(`MBR read failed (${attempt}/${mbrMaxAttempts}): ${lastMbrErr}`)
if (attempt < mbrMaxAttempts) {
await new Promise((r) => setTimeout(r, mbrRetryDelayMs))
}
}
}
if (!mbr || !mbrKeys) {
throw new Error(
'MBR unavailable after ' + mbrMaxAttempts + ' attempts: ' + lastMbrErr
)
}
disk.mbrKeysHex = mbrKeys.map((k) => b4a.toString(k, 'hex'))
let initSource = null
for (const driveKey of keys) {
for (const driveKey of mbrKeys) {
try {
const hex = b4a.toString(driveKey, 'hex')
splash.setPhase('Opening system Hyperdrive…')
+166 -15
View File
@@ -6,10 +6,29 @@
import Autopass from 'autopass'
import b4a from 'b4a'
import { randomBytes } from 'bare-crypto'
import idEnc from 'hypercore-id-encoding'
import {
HDMS_AUTOPASS_SHARE_KEY,
decodeShareOffer
} from './hdms-share-offer.js'
export const HDMS_REGISTRY_PATH = '/.bare/hdms/registry.json'
/**
* @param {{ get: (k: string) => Promise<{ value?: unknown } | null> }} pass
* @param {number} timeoutMs
*/
async function waitForAutopassShare(pass, timeoutMs) {
const deadline = Date.now() + timeoutMs
while (Date.now() < deadline) {
const payload = decodeShareOffer(await pass.get(HDMS_AUTOPASS_SHARE_KEY))
if (payload) return payload
await new Promise((r) => setTimeout(r, 300))
}
return null
}
/** @param {string} label */
export function assertValidHdmsLabel(label) {
if (!label || typeof label !== 'string') throw new Error('Invalid label')
@@ -21,7 +40,38 @@ export function assertValidHdmsLabel(label) {
}
function randomNsSuffix() {
return b4a.toString(b4a.alloc(8), 'hex')
return b4a.toString(randomBytes(8), 'hex')
}
/** Max time to wait for BlindPairing to complete (inviter must be reachable). */
function parseHdmsPairWaitMs() {
const raw = globalThis.process?.env?.BARE_OS_HDMS_PAIR_WAIT_MS
let ms = 120_000
if (raw != null && String(raw).trim() !== '') {
const n = Number(raw)
if (Number.isFinite(n) && n > 0) ms = Math.min(600_000, n)
}
return ms
}
/**
* Prefer inviter's label; if already mounted locally, try label-2 label-99 (63-char cap).
* @param {Map<string, unknown>} byLabel
* @param {string} base
*/
function pickUniqueHdmsMountLabel(byLabel, base) {
assertValidHdmsLabel(base)
if (!byLabel.has(base)) return base
for (let n = 2; n <= 99; n++) {
const suffix = '-' + n
if (base.length + suffix.length > 63) break
const candidate = base + suffix
assertValidHdmsLabel(candidate)
if (!byLabel.has(candidate)) return candidate
}
throw new Error(
'HDMS mount label in use: ' + base + ' (remove it or ask inviter to use another name)'
)
}
/**
@@ -270,24 +320,27 @@ export class HdmsController {
const id = randomNsSuffix()
const ns = `bare-os-hdms-w-${id}`
const nsStore = this.store.namespace(ns, { writable: true })
const drive = new this.Hyperdrive(nsStore)
await drive.ready()
const keyStr = idEnc.encode(drive.key)
/** @type {HdmsRegistryEntry} */
const entry = {
id,
label,
mode: 'writable',
ns,
key: keyStr
key: ''
}
// Single Hyperdrive open: corestore uses exclusive db cores per namespace; a
// second `new Hyperdrive(ns)` while the first is still open deadlocks on ready().
await this._openEntry(entry)
const opened = this.byLabel.get(label)
if (!opened?.drive?.key) {
throw new Error('HDMS create failed (no drive key)')
}
entry.key = idEnc.encode(opened.drive.key)
this.registry.drives.push(entry)
await saveHdmsRegistry(this.personalDrive, this.registry)
await this._openEntry(entry)
ctx.console.log(`Created ${label} key=${keyStr}`)
ctx.console.log(`Created ${label} key=${entry.key}`)
}
/**
@@ -361,13 +414,49 @@ export class HdmsController {
/**
* @param {Record<string, unknown>} ctx
* @param {boolean} readOnly
* @param {boolean} readOnly Autopass writer role for the peer (not Hyperdrive R/W).
* @param {string} [driveLabel] When set, stash this HDMS drives public key so `hdms pair` can mount it read-only under `/mnt/…`.
*/
async invite(ctx, readOnly) {
async invite(ctx, readOnly, driveLabel) {
this.assertLoggedIn(ctx)
await this._ensureAutopass()
const inv = await this.autopass.createInvite({ readOnly })
const ap = this.autopass
if (!ap) throw new Error('HDMS autopass unavailable')
try {
await ap.remove(HDMS_AUTOPASS_SHARE_KEY)
} catch {
/* no prior offer */
}
if (driveLabel != null && String(driveLabel).trim() !== '') {
assertValidHdmsLabel(driveLabel)
const open = this.byLabel.get(driveLabel)
if (!open) throw new Error('Unknown label: ' + driveLabel)
if (open.entry.mode !== 'writable') {
throw new Error('hdms invite <label> requires a writable HDMS drive')
}
const keyZ32 = idEnc.encode(open.drive.key)
await ap.add(
HDMS_AUTOPASS_SHARE_KEY,
JSON.stringify({ label: driveLabel, key: keyZ32 })
)
if (ap.member) await ap.member.flushed()
} else if (ap.member) {
await ap.member.flushed()
}
const inv = await ap.createInvite({ readOnly })
ctx.console.log(inv)
if (driveLabel != null && String(driveLabel).trim() !== '') {
ctx.console.log(
'Invite includes HDMS drive "' +
driveLabel +
'". Peer: hdms pair <invite> (mounts read-only at /mnt/' +
driveLabel +
' or label-2… if that name exists locally; same swarm/bootstrap).'
)
}
}
/**
@@ -388,7 +477,32 @@ export class HdmsController {
const pairer = Autopass.pair(pairNs, inviteZ32.trim(), {
bootstrap: this.bootstrap
})
const pass = await pairer.finished()
const pairWaitMs = parseHdmsPairWaitMs()
let pairTimeout = 0
const pairTimeoutP = new Promise((_, reject) => {
pairTimeout = setTimeout(() => {
reject(
new Error(
'hdms pair: timed out after ' +
pairWaitMs +
'ms waiting for inviter (inviter must be online with matching HYPERSWARM_BOOTSTRAP; set BARE_OS_HDMS_PAIR_WAIT_MS to adjust)'
)
)
}, pairWaitMs)
})
const finishedP = pairer.finished()
let pass
try {
pass = await Promise.race([finishedP, pairTimeoutP])
} catch (e) {
clearTimeout(pairTimeout)
try {
await pairer.close()
} catch (_) {}
void finishedP.catch(() => {})
throw e
}
clearTimeout(pairTimeout)
await pass.ready()
try {
const wk = pass.writerKey
@@ -396,6 +510,30 @@ export class HdmsController {
'Paired Autopass. writerKey=' +
(wk ? b4a.toString(wk, 'hex').slice(0, 16) + '…' : '?')
)
const offer = await waitForAutopassShare(
pass,
Number(
globalThis.process?.env?.BARE_OS_HDMS_PAIR_SHARE_WAIT_MS ?? 25000
) || 25000
)
if (offer) {
const lbl = pickUniqueHdmsMountLabel(this.byLabel, offer.label)
await this.addReadonly(ctx, lbl, offer.key)
const renamed = lbl !== offer.label
ctx.console.log(
'HDMS read-only mount ready at /mnt/' +
lbl +
(renamed
? ' (inviter label "' + offer.label + '" was in use locally)'
: '') +
' — replicate from swarm; not the same as Autopass R/W.'
)
} else {
ctx.console.log(
'No HDMS drive on this invite. Inviter can run: hdms invite [--read-only] <label>'
)
}
} finally {
try {
await pass.close()
@@ -456,7 +594,13 @@ export async function runHdmsCli(hdms, argv, ctx) {
if (!sub || sub === 'help' || sub === '--help') {
ctx.console.log(
'hdms list | create <label> | add <label> <key> | remove <label> | show <label> | invite [--read-only] | pair <invite>'
'hdms list | create <label> | add <label> <key> | remove <label> | show <label> | invite [--read-only] [<label>] | pair <invite>'
)
ctx.console.log(
'invite [--read-only] <label>: attach that writable drive to the invite (peer gets read-only /mnt mount). --read-only is the peers Autopass role, not the drive.'
)
ctx.console.log(
'pair waits for inviter (BARE_OS_HDMS_PAIR_WAIT_MS default 120s); then for drive offer (BARE_OS_HDMS_PAIR_SHARE_WAIT_MS default 25s).'
)
return
}
@@ -488,11 +632,18 @@ export async function runHdmsCli(hdms, argv, ctx) {
if (sub === 'invite') {
const ro = rest.includes('--read-only')
await hdms.invite(ctx, ro)
const pos = rest.filter((x) => x !== '--read-only')
await hdms.invite(ctx, ro, pos[0])
return
}
if (sub === 'pair') {
if (rest.length > 1) {
ctx.console.error(
'hdms: pair takes only the invite string (local mount name matches inviter label)'
)
return
}
await hdms.pair(ctx, rest[0])
return
}
@@ -0,0 +1,31 @@
/**
* Autopass payload for HDMS drive sharing (invite pair addReadonly).
* Kept separate from hdms-manager.js for small, Node-testable surface.
*/
import b4a from 'b4a'
/** Autopass record key written by hdms invite <label>. */
export const HDMS_AUTOPASS_SHARE_KEY = 'bare-os-hdms/pending-share'
/** @param {{ value?: unknown } | null | undefined} offer */
export function decodeShareOffer(offer) {
if (!offer || offer.value == null) return null
const v = offer.value
const raw =
typeof v === 'string' ? v : b4a.toString(/** @type {Uint8Array} */ (v))
try {
const j = JSON.parse(raw)
if (
j &&
typeof j.key === 'string' &&
typeof j.label === 'string' &&
j.key.length > 0
) {
return { label: j.label, key: j.key }
}
} catch {
/* ignore */
}
return null
}
+18
View File
@@ -16,6 +16,10 @@ import {
createVfs,
BARE_OS_PROC_FILE_TO_ID_HYPERCORE_PACK_HRPC_LIFECYCLE
} from './lib/vfs.js'
import {
decodeShareOffer,
HDMS_AUTOPASS_SHARE_KEY
} from './lib/hdms-share-offer.js'
import { buildBareOsHypercorePackHrpcLifecycleProcJson } from './lib/bare-os-proc-hypercore-pack-hrpc-lifecycle.js'
import { buildBareOsPearCorestoreHrpcProcJson } from './lib/bare-os-proc-pear-corestore-hrpc.js'
import { buildBareOsPearInspectLoggerTlsProcJson } from './lib/bare-os-proc-pear-inspect-logger-tls.js'
@@ -2630,6 +2634,20 @@ test('vfs /mnt lists HDMS mounts and allows writable put', async (t) => {
rmSync(dir, { recursive: true, force: true })
})
test('hdms-share-offer decodeShareOffer parses string and bytes', async (t) => {
t.is(HDMS_AUTOPASS_SHARE_KEY, 'bare-os-hdms/pending-share')
t.is(decodeShareOffer(null), null)
t.is(decodeShareOffer({}), null)
t.is(decodeShareOffer({ value: 'not-json' }), null)
t.is(decodeShareOffer({ value: '{}' }), null)
const json = JSON.stringify({ label: 'vol', key: 'abc123' })
t.alike(decodeShareOffer({ value: json }), { label: 'vol', key: 'abc123' })
t.alike(decodeShareOffer({ value: b4a.from(json, 'utf8') }), {
label: 'vol',
key: 'abc123'
})
})
test('vfs /var in root readdir; /var/log empty lstat; writes map to personal', async (t) => {
const dir = testCorestoreDir('vfsvar')
const store = new Corestore(dir)
+39 -10
View File
@@ -1,26 +1,55 @@
{
"name": "hdms",
"section": 1,
"title": "Hyperswarm distributed map store",
"title": "Hyperdrive management (HDMS)",
"synopsis": [
"hdms [OPTION]... [OPERAND]..."
"hdms help",
"hdms list",
"hdms create LABEL",
"hdms add LABEL KEY",
"hdms remove LABEL",
"hdms show LABEL",
"hdms invite [--read-only] [LABEL]",
"hdms pair INVITE"
],
"description": "When the booter wires ctx.runHdms, hdms manages extra Hyperdrives registered on the personal drive at /.bare/hdms/registry.json. Writable drives use a local Corestore namespace and replicate via Hyperswarm; read-only drives open by z32 public key. Subcommands invite and pair use Autopass for blind pairing. If invite includes LABEL (a writable HDMS mount), the inviter stores {label,key} in Autopass so the peer can auto-run the same path as hdms add after pair (read-only /mnt mount). hdms pair takes only the invite string; the local mount name is the inviter label, or label-2, label-3, ... if that name is already used. hdms pair times out if the inviter never answers (BARE_OS_HDMS_PAIR_WAIT_MS, default 120000 ms). After pair, BARE_OS_HDMS_PAIR_SHARE_WAIT_MS (default 25000) bounds polling for the pending share record. Flag --read-only on invite sets the peer Autopass writer role, not Hyperdrive writability.",
"options": [
{
"flag": "--read-only",
"meaning": "With invite: peer joins Autopass as read-only writer. Does not make the offered HDMS mount writable."
}
],
"description": "Invokes ctx.runHdms when the booter provides HDMS integration; otherwise prints unavailable.",
"options": [],
"keywords": [
"hdms",
"hyperdrive",
"hyperswarm",
"map"
"mount",
"autopass",
"pair",
"invite"
],
"bareOsNotes": "Requires unlocked identity and active HDMS; see handbook ch.5.",
"environment": [
"BARE_OS_HDMS_PAIR_WAIT_MS — max milliseconds to wait for blind pairing to complete before hdms pair fails (default 120000, max 600000)",
"BARE_OS_HDMS_PAIR_SHARE_WAIT_MS — milliseconds to poll Autopass for bare-os-hdms/pending-share after hdms pair (default 25000)",
"HYPERSWARM_BOOTSTRAP — comma-separated bootstrap list for swarm (invite/pair and replication)"
],
"bareOsNotes": "Optional booter capability.",
"examples": [
{
"caption": "when booter wires HDMS",
"code": "hdms ls /mnt"
"caption": "create writable drive and list mounts",
"code": "hdms create myvol\nhdms list"
},
{
"caption": "otherwise",
"code": "# prints unavailable without ctx.runHdms"
"caption": "read-only mount by public key",
"code": "hdms add mirror 8j3nbznnxcp7p668jkik75d83s8j7ew9chq4r34cxe8qfrifzmfo"
},
{
"caption": "invite including drive myvol; peer mounts read-only",
"code": "hdms invite myvol\n# peer:\nhdms pair yryu9hwicdkddw5x81hnw7453mu7mtgaq6s3qguar6ar6h1dgk9pdz3nuea7madc47hcsnikzqegdmzwnb86k67yo1bwnhdd7mtncz9xmo"
},
{
"caption": "peer pairs; mount name matches inviter label (or label-2 if taken)",
"code": "hdms pair <INVITE>"
}
]
}
@@ -1,7 +1,7 @@
{
"schema": 2,
"profileId": "bare-os-posix-like",
"generatedAt": "2026-04-05T21:35:38.224Z",
"generatedAt": "2026-04-06T01:43:58.480Z",
"note": "Sparse POSIX Issue 7 coverage hints for /bin utilities. Omitted command names are not yet profiled here.",
"commandIndex": [
{
@@ -1,6 +1,6 @@
{
"schema": 1,
"atMs": 1775424938224,
"atMs": 1775439838480,
"commands": [
"arch",
"awk",
File diff suppressed because one or more lines are too long