Add storage stack modules and track them in git

The modules workspace previously ignored every path matching `storage-*`, which kept all Hypercore, Hyperbee, Hyperdrive, and Autobase packages out of version control. Narrow .gitignore to test-artifact patterns only so production category trees are committed. Storage packages (23 total): - storage-hypercore (7): replicator, seed-policy, fork-picker, merkle-sync, priority-fetch, bitfield-scheduler, audit-chain - storage-hyperbee (5): batch-write, diff-follow, range-watch, secondary-index, tombstone-gc - storage-hyperdrive (6): entry-catalog, gc-sweep, mirror-sync, mount-bridge, version-snapshot, watch-notify - storage-autobase (5): fork-choice, view-sync, writer-lease, indexer-bus, light-writer Implementation highlights: - Shared attach/gossip helpers in _shared/storage-gossip-base.js - P2P modules use Protomux gossip via p2p-bare; local planners omit swarm - Recent deepen pass: priority queues, batch caps, audit export/import, watch/notify aliases on range-watch, fork/view lease helpers, etc. - Per-package README, docs/api.md, docs/architecture.md, tests, examples Also update modules/README.md doc hub links for the full storage stack (hypercore, hyperbee, hyperdrive, autobase). Co-authored-by: Cursor <[email protected]>
2026-05-21 00:17:51 -04:00
parent b10507b060
commit a11e22badc
214 changed files with 57132 additions and 5 deletions
@@ -0,0 +1,5 @@
+# Changelog
+
+## [0.0.0-scaffold]
+
+- Registry scaffold: file tree, load smoke tests, docs stubs
@@ -0,0 +1,41 @@
+# hyper-p2p-drive-mount-bridge
+
+Maps local drive path prefixes to remote prefixes with gossip sync; session-scoped mounts pair with `hyper-p2p-session-bridge`. Hyperswarm gossip when `topic` is set.
+
+**Category:** Storage (Hyperdrive)
+
+**Composes with:** `hyper-p2p-session-bridge`
+
+**Protocol:** `drive-mount-bridge/v1`
+
+## When to use
+
+Federated drive trees expose a unified path namespace across peers.
+
+## When not to use
+
+Flat drives without mount indirection.
+
+## Quick start
+
+```js
+const { HyperP2PDriveMountBridge } = require('hyper-p2p-drive-mount-bridge')
+const topic = process.argv[2] // 64-char hex or string
+const mod = new HyperP2PDriveMountBridge({ topic })
+await mod.ready()   // joins swarm when topic set
+// ... application logic ...
+await mod.close()
+```
+
+## Docs
+
+- [docs/api.md](docs/api.md) — constructor, methods, events, errors
+- [docs/architecture.md](docs/architecture.md) — wire types, state, composition
+- [../_shared/PRODUCTION.md](../../_shared/PRODUCTION.md) — production checklist
+- [../_shared/DOC_STANDARDS.md](../../_shared/DOC_STANDARDS.md) — documentation standards
+
+## Test
+
+```bash
+npm install && npm test
+```
@@ -0,0 +1,111 @@
+# API: hyper-p2p-drive-mount-bridge
+
+**Protocol:** `drive-mount-bridge/v1`
+
+**Export:** `HyperP2PDriveMountBridge`
+
+## Overview
+
+Maps local drive path prefixes to remote prefixes with gossip sync; session-scoped mounts pair with `hyper-p2p-session-bridge`.
+
+## Constructor
+
+```js
+const mod = new HyperP2PDriveMountBridge(opts)
+```
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `topic` | string \| Buffer \| null | null | Hyperswarm topic; gossip attaches when set |
+| `keyPair` | Ed25519 KeyPair | random | Signing identity for swarm |
+| `drive` | object \| null | null | Attached drive instance (`attach()` also supported) |
+
+## Methods
+
+### `attach(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `mountPath(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `resolvePath(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `unmountPath(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `listMounts(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `getStats(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `ready(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+### `close(…)`
+
+- **Returns:** module-specific (see implementation)
+
+- **Throws:** — (none in method body)
+
+## Events
+
+| Event | Payload |
+|-------|---------|
+| `closed` | no payload |
+
+## getStats()
+
+Returns `{ ...this._stats, protocol }` plus module-specific counters (pending queues, registry sizes, gossip in/out when P2P).
+Local modules report hot-path counters only; P2P modules include gossip traffic when `topic` is set.
+
+## Errors
+
+Stable message substrings: see [`../../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
+
+Validation helpers may throw `ValidationError` (e.g. `peer is required`, `path is required`).
+
+## P2P
+
+When `topic` is set, `ready()` joins Hyperswarm and opens gossip for `drive-mount-bridge/v1`.
+Outbound payloads use `sendGossip` (Protomux peer map); inbound handled in `_onGossip`.
+
+### Gossip message types
+
+- `mount`
+- `unmount`
+
+## Testing
+
+```bash
+npm install && npm test
+```
+
+## Common flows
+
+1. `mountPath(localPrefix, remotePrefix)` — gossip `mount`.
+2. `resolvePath(path)` — translate through longest matching mount.
+3. `unmountPath(localPrefix)` — gossip `unmount`.
+
@@ -0,0 +1,45 @@
+# Architecture: hyper-p2p-drive-mount-bridge
+
+**Category:** Storage (Hyperdrive)
+
+```mermaid
+flowchart LR
+  App[Application] --> Mod[HyperP2PDriveMountBridge]
+  Mod --> Gossip[Hyperswarm gossip]
+  Gossip --> Peer[Remote peers]
+```
+
+## Sequence (P2P)
+
+```mermaid
+sequenceDiagram
+  participant App
+  participant Mod as Module
+  participant SW as Hyperswarm
+  participant Peer
+  App->>Mod: ready(topic)
+  Mod->>SW: join(topic)
+  SW->>Peer: connection
+  Mod->>Peer: gossip payload
+  Peer-->>Mod: onmessage
+  Mod-->>App: emit(event)
+```
+
+## Wire messages
+
+| type | fields | direction | behavior |
+|------|--------|-----------|----------|
+| `mount` | localPrefix, remotePrefix | gossip | add mount |
+| `unmount` | localPrefix | gossip | remove mount |
+
+## State model
+
+- In-memory `Map` / `Set` / array structures for hot path
+- Optional attached HyperDrive via `attach()`
+- `close()` clears ephemeral state and destroys swarm when P2P
+
+## Composition
+
+Composes with: `hyper-p2p-session-bridge`.
+
+Storage gossip helpers: [`../../_shared/storage-gossip-base.js`](../../_shared/storage-gossip-base.js).
@@ -0,0 +1,14 @@
+require('bare-process/global')
+const { HyperP2PDriveMountBridge } = require('../index.js')
+
+async function main () {
+  const topic = process.argv[2] || null
+  const m = new HyperP2PDriveMountBridge({ topic })
+  m.attach({ version: 0 })
+  await m.ready()
+  m.mountPath('/local', '/remote')
+  console.log(m.resolvePath('/local/file'))
+  await m.close()
+  console.log('done')
+}
+main().catch(console.error)
@@ -0,0 +1,96 @@
+require('bare-process/global')
+const EventEmitter = require('bare-events')
+const { assertNonEmpty } = require('../../_shared/lib/errors.js')
+const { assertDrive, attachGossip, sendGossip } = require('../../_shared/storage-gossip-base.js')
+
+const PROTOCOL = 'drive-mount-bridge/v1'
+
+class HyperP2PDriveMountBridge extends EventEmitter {
+  constructor (opts = {}) {
+    super()
+    this.topic = opts.topic || null
+    this.keyPair = opts.keyPair || require('hypercore-crypto').keyPair()
+    this.drive = opts.drive || null
+    this._mounts = new Map()
+    this._stats = { mounted: 0, resolved: 0, gossipIn: 0, gossipOut: 0 }
+    this.swarm = null
+  }
+
+  attach (drive) {
+    assertDrive(drive)
+    this.drive = drive
+    return this
+  }
+
+  mountPath (localPrefix, remotePrefix) {
+    assertNonEmpty(localPrefix, 'localPrefix')
+    assertNonEmpty(remotePrefix, 'remotePrefix')
+    this._mounts.set(localPrefix, { remotePrefix, at: Date.now() })
+    this._stats.mounted++
+    sendGossip(this, { type: 'mount', localPrefix, remotePrefix })
+    this._stats.gossipOut++
+    return { localPrefix, remotePrefix }
+  }
+
+  resolvePath (path) {
+    assertNonEmpty(path, 'path')
+    for (const [local, { remotePrefix }] of this._mounts) {
+      if (path === local || path.startsWith(local + '/')) {
+        const suffix = path === local ? '' : path.slice(local.length)
+        const resolved = remotePrefix + suffix
+        this._stats.resolved++
+        return { local: path, remote: resolved, mount: local }
+      }
+    }
+    return { local: path, remote: path, mount: null }
+  }
+
+  unmountPath (localPrefix) {
+    const ok = this._mounts.delete(localPrefix)
+    if (ok) {
+      sendGossip(this, { type: 'unmount', localPrefix })
+      this._stats.gossipOut++
+    }
+    return ok
+  }
+
+  listMounts () {
+    return [...this._mounts.entries()].map(([local, m]) => ({
+      localPrefix: local,
+      remotePrefix: m.remotePrefix
+    }))
+  }
+
+  _onGossip (d) {
+    if (!d) return
+    this._stats.gossipIn++
+    if (d.type === 'mount') {
+      this._mounts.set(d.localPrefix, { remotePrefix: d.remotePrefix, at: Date.now() })
+    }
+    if (d.type === 'unmount') this._mounts.delete(d.localPrefix)
+  }
+
+  getStats () {
+    return { ...this._stats, mounts: this._mounts.size, protocol: PROTOCOL }
+  }
+
+  async ready () {
+    if (this.swarm || !this.topic) return this
+    await attachGossip(this, {
+      keyPair: this.keyPair,
+      topic: this.topic,
+      protocol: PROTOCOL,
+      onmessage: (d) => this._onGossip(d)
+    })
+    return this
+  }
+
+  async close () {
+    if (this.swarm) await this.swarm.destroy().catch(() => {})
+    this.swarm = null
+    this._mounts.clear()
+    this.emit('closed')
+  }
+}
+
+module.exports = { HyperP2PDriveMountBridge, PROTOCOL }
@@ -0,0 +1,27 @@
+{
+  "name": "hyper-p2p-drive-mount-bridge",
+  "version": "0.3.1",
+  "description": "Mount bridge for drive paths.",
+  "main": "index.js",
+  "type": "commonjs",
+  "license": "Apache-2.0",
+  "scripts": { "test": "brittle-bare test/test.js" },
+  "dependencies": {
+    "bare-events": "^2.8.0",
+    "bare-process": "^4.4.0",
+    "b4a": "^1.6.7",
+    "hypercore-crypto": "^3.0.0",
+    "protomux": "^3.0.0",
+    "compact-encoding": "^2.16.0"
+  },
+  "peerDependencies": {
+    "hyperswarm": "^4.0.0",
+    "bare": ">=1.0.0",
+    "hyperdrive": "^11.0.0"
+  },
+  "devDependencies": { "brittle": "^3.0.0" },
+  "imports": {
+    "process": { "bare": "bare-process", "default": "process" },
+    "events": { "bare": "bare-events", "default": "events" }
+  }
+}
@@ -0,0 +1,37 @@
+require('bare-process/global')
+const test = require('brittle')
+const { HyperP2PDriveMountBridge, PROTOCOL } = require('../index.js')
+
+test('exports', (t) => {
+  t.ok(HyperP2PDriveMountBridge)
+  t.is(PROTOCOL, 'drive-mount-bridge/v1')
+})
+
+test('mountPath resolvePath', async (t) => {
+  const m = new HyperP2PDriveMountBridge()
+  m.attach({ get: async () => null })
+  m.mountPath('/local', '/remote')
+  const r = m.resolvePath('/local/sub/file')
+  t.is(r.remote, '/remote/sub/file')
+  await m.close()
+})
+
+test('unmountPath', async (t) => {
+  const m = new HyperP2PDriveMountBridge()
+  m.mountPath('/a', '/b')
+  t.ok(m.unmountPath('/a'))
+  await m.close()
+})
+
+test('validation', async (t) => {
+  const m = new HyperP2PDriveMountBridge()
+  try { m.mountPath(null, '/r') } catch (e) { t.ok(e) }
+  await m.close()
+})
+
+test('getStats', async (t) => {
+  const m = new HyperP2PDriveMountBridge()
+  m.mountPath('/l', '/r')
+  t.is(m.getStats().mounts, 1)
+  await m.close()
+})