Updates

2026-05-20 21:02:45 -04:00
parent 1f3f4b24a2
commit 14d0980b4f
734 changed files with 682 additions and 746 deletions
@@ -0,0 +1,7 @@
+# Measurement & rate control
+
+1 modules. See [`../MODULE_CATEGORIES.md`](../../MODULE_CATEGORIES.md).
+
+| Module |
+|--------|
+| [hyper-p2p-bucket-rate-limit](hyper-p2p-bucket-rate-limit/) |
@@ -0,0 +1,2 @@
+node_modules/
+*-storage/
@@ -0,0 +1,23 @@
+# Changelog
+
+<!-- legacy: v0.2.0 -->
+
+- Gossip bucket sync on configure, consume, and periodic `sync()`; `getBucket(peerId)`.
+
+<!-- legacy: v0.1.0 -->
+
+- Initial release.
+<!-- legacy: v0.2.1 -->
+
+- Production docs, input validation, third test, integration notes.
+<!-- legacy: v0.3.0 -->
+
+- Wave 6: presence-tier API tables, architecture wire section, validation test.
+
+<!-- legacy: v0.3.1 -->
+
+- Wave 7: correct protocol in docs, getStats(), wire tables, category README.
+## [0.3.2] - 2026-05-21
+
+### Changed
+- Exhaustive documentation pass (api, architecture, README) per DOC_STANDARDS.md.
@@ -0,0 +1,43 @@
+# hyper-p2p-bucket-rate-limit
+
+Production measurement & rate control module: Hyperswarm discovery + Protomux when `topic` is set.
+
+**Category:** Measurement & rate control
+
+**Composes with:** `hyper-p2p-congestion-signal`, `hyper-p2p-bandwidth-broker`
+
+**Protocol:** `bucket-rate-limit/v1`
+
+## When to use
+
+Multi-peer apps that need measurement & rate control over a shared Hyperswarm topic.
+
+## When not to use
+
+Single-process tools with no P2P topic (use local APIs only or skip `ready()`).
+
+## Quick start
+
+```js
+const { HyperP2PBucketRateLimit } = require('hyper-p2p-bucket-rate-limit')
+const topic = process.argv[2] // 64-char hex or string
+const mod = new HyperP2PBucketRateLimit({ topic, enableBackgroundTimers: false })
+await mod.ready()   // joins swarm when topic set
+// ... application logic ...
+await mod.close()
+```
+
+## Docs
+
+- [docs/api.md](docs/api.md) — constructor, methods, events, errors
+- [docs/architecture.md](docs/architecture.md) — wire types, state, composition
+- [../_shared/PRODUCTION.md](../../_shared/PRODUCTION.md) — production checklist
+- [../_shared/DOC_STANDARDS.md](../../_shared/DOC_STANDARDS.md) — documentation standards
+
+- Integration: [`../../real_tests/integration/`](../../../real_tests/integration/) — `bucket-rate-limit-two-node.js`
+
+## Test
+
+```bash
+npm install && npm test
+```
@@ -0,0 +1,98 @@
+# API: hyper-p2p-bucket-rate-limit
+
+**Protocol:** `bucket-rate-limit/v1`
+
+**Export:** `HyperP2PBucketRateLimit`
+
+## Overview
+
+Production measurement & rate control module: Hyperswarm discovery + Protomux when `topic` is set.
+
+## Constructor
+
+```js
+const mod = new HyperP2PBucketRateLimit(opts)
+```
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `topic` | varies | null | topic |
+| `keyPair` | KeyPair | random Ed25519 | keyPair |
+| `rate` | number | 10 | rate |
+| `burst` | number | 20 | burst |
+| `syncIntervalMs` | number | 10000 (ms) | Background sync interval |
+| `enableBackgroundTimers` | boolean | `false` | Periodic timers (off in tests) |
+
+## Methods
+
+### `configure({ rate, burst })`
+
+- **Returns:** `value`
+- **Throws:** — (none documented in method body)
+
+### `snapshot(—)`
+
+- **Returns:** `value`
+- **Throws:** — (none documented in method body)
+
+### `getBucket(peerId)`
+
+- **Returns:** `value`
+- **Throws:** — (none documented in method body)
+
+### `tryConsume(peerId, cost = 1)`
+
+- **Returns:** `value`
+- **Throws:** — (none documented in method body)
+
+### `sync(—)`
+
+- **Returns:** `value`
+- **Throws:** — (none documented in method body)
+
+### `ready(—)`
+
+- **Returns:** `Promise`
+- **Throws:** — (none documented in method body)
+
+### `getStats(—)`
+
+- **Returns:** `object`
+- **Throws:** — (none documented in method body)
+
+### `close(—)`
+
+- **Returns:** `Promise<void>`
+- **Throws:** — (none documented in method body)
+
+## Events
+
+| Event | Payload |
+|-------|---------|
+| `closed` | no payload |
+| `configure` | snap |
+| `consume` | tokens |
+| `reject` | payload object |
+| `remote-configure` | rate, burst |
+| `sync` | snap |
+
+## getStats()
+
+Returns `{ ...this._stats }` — typically `ops`, `errors`, and module-specific counters (`created`, `relays`, `open`, `peers`, etc.).
+Library-only modules may include `mode: 'local'`.
+
+## Errors
+
+Stable message substrings: see [`../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
+
+## P2P
+
+When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `bucket-rate-limit/v1`.
+
+## Testing
+
+```bash
+npm install && npm test
+```
+
+Integration: [`../../real_tests/integration/bucket-rate-limit-two-node.js`](../../../real_tests/integration/bucket-rate-limit-two-node.js)
@@ -0,0 +1,45 @@
+# Architecture: hyper-p2p-bucket-rate-limit
+
+**Category:** Measurement & rate control
+
+```mermaid
+flowchart LR
+  App[Application] --> Mod[HyperP2PBucketRateLimit]
+  Mod --> Mux[Protomux bucket-rate-limit/v1]
+  Mux --> Swarm[Hyperswarm]
+```
+
+## Sequence (P2P)
+
+```mermaid
+sequenceDiagram
+  participant App
+  participant Mod as Module
+  participant SW as Hyperswarm
+  participant Peer
+  App->>Mod: ready(topic)
+  Mod->>SW: join(topic)
+  SW->>Peer: connection
+  Mod->>Peer: gossip / Protomux
+  Peer-->>Mod: onmessage
+  Mod-->>App: emit(event)
+```
+
+## Wire messages
+
+| type | fields | direction | behavior |
+|------|--------|-----------|----------|
+| `bucket` | peerId, tokens, updatedAt | gossip | Handled in onmessage / gossipSend |
+| `configure` | buckets, burst, rate, type | gossip | Handled in onmessage / gossipSend |
+| `sync` | buckets, type | gossip | Handled in onmessage / gossipSend |
+
+## State model
+
+- In-memory `Map` / `Set` structures for hot path
+- Optional Hyperbee/Hypercore persistence when `storageDir` or `memoryOnly` is configured
+- `close()` tears down swarm, timers, and clears ephemeral state
+
+## Composition
+
+Composes with: `hyper-p2p-congestion-signal`, `hyper-p2p-bandwidth-broker`.
+
@@ -0,0 +1,11 @@
+require('bare-process/global')
+const { HyperP2PBucketRateLimit } = require('../index.js')
+
+async function main () {
+  const rl = new HyperP2PBucketRateLimit({ rate: 5, burst: 10 })
+  console.log('consume', rl.tryConsume('peer-1', 3))
+  console.log('consume', rl.tryConsume('peer-1', 8))
+  await rl.close()
+  console.log('done')
+}
+main().catch(console.error)
@@ -0,0 +1,145 @@
+require('bare-process/global')
+const EventEmitter = require('bare-events')
+const { setInterval, clearInterval } = require('bare-timers')
+const b4a = require('b4a')
+const { initModuleSwarm, gossipSend } = require('../../_shared/p2p-bare.js')
+
+const PROTOCOL = 'bucket-rate-limit/v1'
+
+class HyperP2PBucketRateLimit extends EventEmitter {
+  constructor (opts = {}) {
+    super()
+    this._stats = { ops: 0, errors: 0 }
+
+    this.topic = opts.topic || null
+    this.keyPair = opts.keyPair || require('hypercore-crypto').keyPair()
+    this._buckets = new Map()
+    this.rate = opts.rate ?? 10
+    this.burst = opts.burst ?? 20
+    this.syncIntervalMs = opts.syncIntervalMs ?? 10000
+    this.enableBackgroundTimers = opts.enableBackgroundTimers === true
+    this._syncTimer = null
+    this.swarm = null
+    this._peerMsgs = null
+  }
+
+  configure ({ rate, burst }) {
+    if (rate != null) this.rate = rate
+    if (burst != null) this.burst = burst
+    const snap = this.snapshot()
+    this.emit('configure', snap)
+    this._gossip({ type: 'configure', ...snap })
+    return snap
+  }
+
+  snapshot () {
+    return { rate: this.rate, burst: this.burst, buckets: this._bucketSnapshot() }
+  }
+
+  _bucketSnapshot () {
+    const out = {}
+    for (const [peerId, b] of this._buckets) {
+      out[peerId] = { tokens: b.tokens, updatedAt: b.updatedAt }
+    }
+    return out
+  }
+
+  _gossip (data) {
+    if (this._peerMsgs) gossipSend(this, data)
+  }
+
+  _bucket (peerId) {
+    const key = typeof peerId === 'string' ? peerId : b4a.toString(peerId, 'hex')
+    let b = this._buckets.get(key)
+    if (!b) {
+      b = { tokens: this.burst, updatedAt: Date.now() }
+      this._buckets.set(key, b)
+    }
+    const now = Date.now()
+    const elapsed = (now - b.updatedAt) / 1000
+    b.tokens = Math.min(this.burst, b.tokens + elapsed * this.rate)
+    b.updatedAt = now
+    return b
+  }
+
+  getBucket (peerId) {
+    const key = typeof peerId === 'string' ? peerId : b4a.toString(peerId, 'hex')
+    const b = this._bucket(key)
+    return { peerId: key, tokens: b.tokens, rate: this.rate, burst: this.burst, updatedAt: b.updatedAt }
+  }
+
+  tryConsume (peerId, cost = 1) {
+    const b = this._bucket(peerId)
+    if (b.tokens < cost) {
+      this.emit('reject', { peerId, cost })
+      return false
+    }
+    b.tokens -= cost
+    this.emit('consume', { peerId, cost, tokens: b.tokens })
+    this._gossip({ type: 'bucket', peerId: typeof peerId === 'string' ? peerId : b4a.toString(peerId, 'hex'), tokens: b.tokens, updatedAt: b.updatedAt })
+    return true
+  }
+
+  _applyRemote (data) {
+    if (!data) return
+    if (data.type === 'configure') {
+      if (data.rate != null) this.rate = data.rate
+      if (data.burst != null) this.burst = data.burst
+      this.emit('remote-configure', { rate: this.rate, burst: this.burst })
+    } else if (data.type === 'sync' && data.buckets) {
+      for (const [peerId, remote] of Object.entries(data.buckets)) {
+        const cur = this._buckets.get(peerId)
+        if (!cur || remote.updatedAt > cur.updatedAt) {
+          this._buckets.set(peerId, { tokens: remote.tokens, updatedAt: remote.updatedAt })
+        }
+      }
+      this.emit('sync', { peers: Object.keys(data.buckets).length })
+    } else if (data.type === 'bucket' && data.peerId) {
+      const cur = this._buckets.get(data.peerId)
+      if (!cur || data.updatedAt >= cur.updatedAt) {
+        this._buckets.set(data.peerId, { tokens: data.tokens, updatedAt: data.updatedAt })
+      }
+    }
+  }
+
+  sync () {
+    const snap = { type: 'sync', ...this.snapshot() }
+    this._gossip(snap)
+    this.emit('sync', snap)
+    return snap
+  }
+
+  _ensureSyncTimer () {
+    if (!this.enableBackgroundTimers || this._syncTimer) return
+    this._syncTimer = setInterval(() => this.sync(), this.syncIntervalMs)
+  }
+
+  async ready () {
+    if (this.swarm || !this.topic) return this
+    await initModuleSwarm(this, {
+      keyPair: this.keyPair,
+      topic: this.topic,
+      protocol: PROTOCOL,
+      onmessage: (data) => this._applyRemote(data)
+    })
+    this._ensureSyncTimer()
+    return this
+  }
+
+
+  getStats () {
+    return { ...this._stats }
+  }
+
+  async close () {
+    if (this._syncTimer) {
+      clearInterval(this._syncTimer)
+      this._syncTimer = null
+    }
+    if (this.swarm) await this.swarm.destroy().catch(() => {})
+    this.swarm = null
+    this.emit('closed')
+  }
+}
+
+module.exports = { HyperP2PBucketRateLimit, PROTOCOL }
@@ -0,0 +1,25 @@
+{
+  "name": "hyper-p2p-bucket-rate-limit",
+  "version": "0.3.1",
+  "description": "Token-bucket rate limiting per peer for Bare/Pear P2P.",
+  "main": "index.js",
+  "type": "commonjs",
+  "license": "Apache-2.0",
+  "scripts": { "test": "brittle-bare test/test.js" },
+  "dependencies": {
+    "bare-events": "^2.8.0",
+    "bare-process": "^4.4.0",
+    "bare-timers": "^2.0.0",
+    "b4a": "^1.6.7",
+    "hypercore-crypto": "^3.0.0",
+    "protomux": "^3.0.0",
+    "compact-encoding": "^2.0.0"
+  },
+  "peerDependencies": { "hyperswarm": "^4.0.0", "bare": ">=1.0.0" },
+  "devDependencies": { "brittle": "^3.0.0" },
+  "imports": {
+    "process": { "bare": "bare-process", "default": "process" },
+    "events": { "bare": "bare-events", "default": "events" },
+    "timers": { "bare": "bare-timers", "default": "timers" }
+  }
+}
@@ -0,0 +1,51 @@
+require('bare-process/global')
+const test = require('brittle')
+const { HyperP2PBucketRateLimit } = require('../index.js')
+
+test('bucket-rate-limit: consume within burst', async (t) => {
+  const rl = new HyperP2PBucketRateLimit({ rate: 1, burst: 2 })
+  t.ok(rl.tryConsume('peer-a', 1))
+  t.ok(rl.tryConsume('peer-a', 1))
+  t.not(rl.tryConsume('peer-a', 1))
+  await rl.close()
+})
+
+test('bucket-rate-limit: configure', async (t) => {
+  const rl = new HyperP2PBucketRateLimit()
+  rl.configure({ rate: 100, burst: 100 })
+  t.ok(rl.tryConsume('p', 50))
+  await rl.close()
+})
+
+test('bucket-rate-limit: getBucket', async (t) => {
+  const rl = new HyperP2PBucketRateLimit({ rate: 2, burst: 3 })
+  rl.tryConsume('peer-z', 1)
+  const b = rl.getBucket('peer-z')
+  t.is(b.peerId, 'peer-z')
+  t.ok(b.tokens < 3)
+  await rl.close()
+})
+test('hyper-p2p-bucket-rate-limit: close without leak', async (t) => {
+  const m = new HyperP2PBucketRateLimit()
+  await m.close()
+  t.pass()
+})
+test('hyper-p2p-bucket-rate-limit: validation rejects invalid input', async (t) => {
+  const m = new HyperP2PBucketRateLimit()
+  try {
+    if (typeof m.addNeighbor === 'function') m.addNeighbor(null)
+    else if (typeof m.buildCircuit === 'function') m.buildCircuit([])
+    else if (typeof m.grant === 'function') m.grant(null, -1)
+    else if (typeof m.enqueue === 'function') m.enqueue('bad', null)
+    else if (typeof m.reportSample === 'function') m.reportSample(null, -1, -1)
+    else if (typeof m.fanout === 'function') m.fanout(null, 0)
+    else if (typeof m.probe === 'function') m.probe(null)
+    else if (typeof m.resolve === 'function') m.resolve(null)
+    else if (typeof m.acquire === 'function') m.acquire(null)
+    else throw new Error('no validation hook')
+    t.fail('expected throw')
+  } catch (err) {
+    t.ok(err instanceof Error)
+  }
+  await m.close()
+})