Next-depth pass: network-stack, CRDTs, gossip, scheduling, experimental

Expand introspection helpers and getStats(protocol) across six categories
after the storage/trust giant pass.

network-stack (10): listPeerIds, hasPeer, bestPeer, listCircuitIds,
queueDepth, listPendingIds, and related helpers; protocol in getStats.

state-crdts: crdt-map has/size/entries; conflict-set size/isEmpty;
reactive-state size/keys; richer getStats on map and conflict-set.

messaging-gossip: gossip-mesh seenCount/clearSeen; dedup size/clear;
event-bus getStats merges metrics/topics/peers.

scheduling-queues: topic-lease listShards/isHeld; peer-scheduler listJobs;
activity-queue pending/claimed stats.

measurement-rate-control: bucket-rate listPeerIds/peerCount.

experimental (16): protocol-rich getStats; mycelium listPeerIds/totalCredits;
pheromone listPathIds.

Docs: API Methods for link-probe, crdt-map, gossip-mesh; expanded category
READMEs for state-crdts, messaging-gossip, scheduling, measurement.

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Raven Scott
2026-05-21 00:32:11 -04:00
co-authored by Cursor
parent 7cb1612e71
commit 70119888cd
37 changed files with 463 additions and 168 deletions
@@ -88,7 +88,7 @@ class HyperP2PContradictionGraph extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, claims: this._claims.length, protocol: PROTOCOL }
} }
async close () { async close () {
@@ -67,7 +67,12 @@ class HyperP2PEntropyBeacon extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
contributions: this._contributions,
poolHash: this.poolHash(),
protocol: PROTOCOL
}
} }
async close () { async close () {
+6 -1
View File
@@ -88,7 +88,12 @@ class HyperP2PGravityWell extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
wells: this._wells.size,
sink: this._sink.length,
protocol: PROTOCOL
}
} }
async close () { async close () {
+7 -1
View File
@@ -109,7 +109,13 @@ class HyperP2PMirrorRealm extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
realmA: this._realmA.size,
realmB: this._realmB.size,
forked: this._forked,
protocol: PROTOCOL
}
} }
async close () { async close () {
+17 -1
View File
@@ -27,6 +27,16 @@ class HyperP2PMyceliumPool extends EventEmitter {
return this._balances.get(this._pid(peerId)) || 0 return this._balances.get(this._pid(peerId)) || 0
} }
listPeerIds () { return [...this._balances.keys()] }
totalCredits () {
let t = 0
for (const b of this._balances.values()) t += b
return t
}
loanCount () { return this._loans.length }
donate (peerId, credits) { donate (peerId, credits) {
if (credits == null || credits < 0) throw new Error('credits must be non-negative') if (credits == null || credits < 0) throw new Error('credits must be non-negative')
const id = this._pid(peerId) const id = this._pid(peerId)
@@ -91,7 +101,13 @@ class HyperP2PMyceliumPool extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
peers: this._balances.size,
loans: this._loans.length,
totalCredits: this.totalCredits(),
protocol: PROTOCOL
}
} }
async close () { async close () {
@@ -83,7 +83,7 @@ class HyperP2PParadoxClock extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, stamps: this._stamps.size, protocol: PROTOCOL }
} }
async close () { async close () {
@@ -77,6 +77,10 @@ class HyperP2PPheromoneTrail extends EventEmitter {
return { trails: [...this._trails.values()] } return { trails: [...this._trails.values()] }
} }
listPathIds () { return [...this._trails.keys()] }
trailCount () { return this._trails.size }
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
await initModuleSwarm(this, { await initModuleSwarm(this, {
@@ -92,7 +96,7 @@ class HyperP2PPheromoneTrail extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, trails: this._trails.size, protocol: PROTOCOL }
} }
async close () { async close () {
@@ -80,7 +80,12 @@ class HyperP2PSilenceProtocol extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
seen: this._seen.size,
absent: this._absent.size,
protocol: PROTOCOL
}
} }
async close () { async close () {
+5 -1
View File
@@ -106,7 +106,11 @@ class HyperP2PTimeCapsule extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
capsules: this._capsules.size,
protocol: PROTOCOL
}
} }
async close () { async close () {
+1 -1
View File
@@ -70,7 +70,7 @@ class HyperP2PWhisperMesh extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, seen: this._seen.size, protocol: PROTOCOL }
} }
async close () { async close () {
+24 -6
View File
@@ -1,10 +1,28 @@
# Measurement & rate control # Measurement & rate control
**Path:** `modules/measurement-rate-control/` · **Modules:** 4 (4 production, 0 scaffold) **Path:** `modules/measurement-rate-control/` · **Modules:** 4 (all production)
See [MODULE_CATEGORIES.md](../MODULE_CATEGORIES.md#measurement-rate-control). Token buckets, histogram gossip, percentile sketches, and SLA budgets. Hub: [`../../docs/measurement-rate-control/README.md`](../../docs/measurement-rate-control/README.md).
- [hyper-p2p-bucket-rate-limit](./hyper-p2p-bucket-rate-limit/) — production ## Modules
- [hyper-p2p-histogram-gossip](./hyper-p2p-histogram-gossip/) — production
- [hyper-p2p-percentile-sketch](./hyper-p2p-percentile-sketch/) — production | Module | Protocol | Summary |
- [hyper-p2p-sla-budget](./hyper-p2p-sla-budget/) — production |--------|----------|---------|
| [hyper-p2p-bucket-rate-limit](./hyper-p2p-bucket-rate-limit/) | `bucket-rate-limit/v1` | Per-peer token bucket (`listPeerIds`, `peerCount`) |
| [hyper-p2p-histogram-gossip](./hyper-p2p-histogram-gossip/) | `histogram-gossip/v1` | Merged latency histograms |
| [hyper-p2p-percentile-sketch](./hyper-p2p-percentile-sketch/) | `percentile-sketch/v1` | Streaming quantiles |
| [hyper-p2p-sla-budget](./hyper-p2p-sla-budget/) | `sla-budget/v1` | Service-level byte/error budgets |
## Quick start
```js
const { HyperP2PBucketRateLimit } = require('hyper-p2p-bucket-rate-limit')
const limit = new HyperP2PBucketRateLimit({ rate: 5, burst: 10 })
limit.tryConsume('peer-a')
```
## Test
```bash
cd hyper-p2p-bucket-rate-limit && npm test
```
@@ -62,6 +62,10 @@ class HyperP2PBucketRateLimit extends EventEmitter {
return b return b
} }
listPeerIds () { return [...this._buckets.keys()] }
peerCount () { return this._buckets.size }
getBucket (peerId) { getBucket (peerId) {
const key = typeof peerId === 'string' ? peerId : b4a.toString(peerId, 'hex') const key = typeof peerId === 'string' ? peerId : b4a.toString(peerId, 'hex')
const b = this._bucket(key) const b = this._bucket(key)
@@ -128,7 +132,7 @@ class HyperP2PBucketRateLimit extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, buckets: this._buckets.size, rate: this.rate, burst: this.burst, protocol: PROTOCOL }
} }
async close () { async close () {
+32 -5
View File
@@ -1,9 +1,36 @@
# Messaging & gossip # Messaging & gossip
**Path:** `modules/messaging-gossip/` · **Modules:** 3 (3 production, 0 scaffold) **Path:** `modules/messaging-gossip/` · **Modules:** 3 (all production)
See [MODULE_CATEGORIES.md](../MODULE_CATEGORIES.md#messaging-gossip). Epidemic messaging, deduplication, and distributed event sourcing. Hub: [`../../docs/messaging/README.md`](../../docs/messaging/README.md).
- [hyper-p2p-dedup-filter](./hyper-p2p-dedup-filter/) — production ## When to use
- [hyper-p2p-distributed-event-bus](./hyper-p2p-distributed-event-bus/) — production
- [hyper-p2p-gossip-mesh](./hyper-p2p-gossip-mesh/) — production - Fan-out app events without a broker (`gossip-mesh`).
- Cross-peer dedup before expensive handlers (`dedup-filter`).
- Durable topic logs with vector clocks (`distributed-event-bus`).
## Modules
| Module | Protocol | Summary |
|--------|----------|---------|
| [hyper-p2p-gossip-mesh](./hyper-p2p-gossip-mesh/) | `gossip-mesh/v1` | TTL/fanout epidemic pub/sub (`seenCount`, `clearSeen`) |
| [hyper-p2p-dedup-filter](./hyper-p2p-dedup-filter/) | `hyper-p2p-dedup-filter/v1` | Bounded seen-id set (`size`, `clear`, `compact`) |
| [hyper-p2p-distributed-event-bus](./hyper-p2p-distributed-event-bus/) | `hyper-p2p-distributed-event-bus/v1` | Hyperbee event log, signing, replay |
## Quick start
```js
const { HyperP2PGossipMesh } = require('hyper-p2p-gossip-mesh')
const { HyperP2PDedupFilter } = require('hyper-p2p-dedup-filter')
const dedup = new HyperP2PDedupFilter()
const mesh = new HyperP2PGossipMesh({ dedupFilter: dedup })
mesh.subscribe((m) => m.type === 'ping')
mesh.publish({ type: 'ping', at: Date.now() })
```
## Test
```bash
cd hyper-p2p-gossip-mesh && npm test
```
@@ -53,6 +53,14 @@ class HyperP2PDedupFilter extends EventEmitter {
return true return true
} }
size () { return this._seen.size }
clear () {
const n = this._seen.size
this._seen.clear()
return n
}
compact () { compact () {
const arr = Array.from(this._seen) const arr = Array.from(this._seen)
const keep = arr.slice(-Math.floor(this.maxIds * 0.75)) const keep = arr.slice(-Math.floor(this.maxIds * 0.75))
@@ -63,7 +71,7 @@ class HyperP2PDedupFilter extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, seen: this._seen.size, maxIds: this.maxIds, protocol: PROTOCOL }
} }
async close () { async close () {
@@ -425,7 +425,15 @@ class HyperP2PDistributedEventBus extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
...this._metrics,
topics: this.eventLog.size,
peers: this.peers.size,
subscriptions: this.subscriptions.size,
seen: this.seenEvents.size,
protocol: EVENT_BUS_PROTOCOL
}
} }
async close () { async close () {
@@ -6,7 +6,7 @@
## Overview ## Overview
Production messaging & gossip module: Hyperswarm discovery + Protomux when `topic` is set. Epidemic pub/sub with TTL + fanout. Optional `dedupFilter` integration. Subscribers filter via `subscribe(fn)`.
## Constructor ## Constructor
@@ -16,70 +16,57 @@ const mod = new HyperP2PGossipMesh(opts)
| Option | Type | Default | Description | | Option | Type | Default | Description |
|--------|------|---------|-------------| |--------|------|---------|-------------|
| `topic` | varies | null | topic | | `topic` | varies | `null` | Hyperswarm topic |
| `keyPair` | KeyPair | random Ed25519 | keyPair | | `keyPair` | KeyPair | random | Discovery identity |
| `defaultTtl` | number | 8 | defaultTtl | | `defaultTtl` | number | `8` | Hop budget per message |
| `defaultFanout` | number | 3 | defaultFanout | | `defaultFanout` | number | `3` | Peers to forward to per hop |
| `dedupFilter` | varies | null | dedupFilter | | `dedupFilter` | HyperP2PDedupFilter \| null | `null` | Shared dedup module |
## Methods ## Methods
### `subscribe(filter)` ### `subscribe(filter) → unsubscribe`
- **Returns:** `value` `filter` is `(msg) => boolean` or omitted (accept all). Returns restore function.
- **Throws:** — (none documented in method body)
### `publish(msg, opts = {})` ### `publish(msg, opts?) → string | false`
- **Returns:** `value` Hashes message to id; returns id or `false` if deduped. `opts.ttl`, `opts.fanout` override defaults.
- **Throws:** — (none documented in method body)
### `ready(—)` ### `seenCount() → number`
- **Returns:** `Promise` Local seen-id set size.
- **Throws:** — (none documented in method body)
### `getStats(—)` ### `hasSeen(id) → boolean`
- **Returns:** `object` ### `clearSeen() → number`
- **Throws:** — (none documented in method body)
### `close(—)` Clears seen set; returns prior size.
- **Returns:** `Promise<void>` ### `getStats() → object`
- **Throws:** — (none documented in method body)
`{ ops, errors, seen, protocol }`.
### `async ready()` / `async close()`
Protomux `gossip-mesh/v1`; receives `gossip` envelopes.
## Events ## Events
| Event | Payload | | Event | Payload |
|-------|---------| |-------|---------|
| `closed` | no payload | | `closed` | no payload |
| `message` | envelope | | `message` | `{ id, msg, ttl, fanout }` |
## getStats() ## getStats()
Returns `{ ...this._stats }` — typically `ops`, `errors`, and module-specific counters (`created`, `relays`, `open`, `peers`, etc.). `{ ops, errors, seen, protocol: 'gossip-mesh/v1' }`.
Library-only modules may include `mode: 'local'`.
## Errors
Stable message substrings: see [`../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
## P2P ## P2P
When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `gossip-mesh/v1`. Outbound `gossip` with envelope; inbound decrements TTL and fanouts to up to `fanout` peers.
## Testing ## Testing
```bash ```bash
npm install && npm test npm install && npm test
``` ```
Integration: [`../../real_tests/integration/gossip-mesh-two-node.js`](../../../real_tests/integration/gossip-mesh-two-node.js)
## Common flows
1. `ready(topic)` — join swarm and open `gossip-mesh/v1` channel.
2. `publish(payload, { ttl })` — epidemic fanout with TTL decay per hop.
3. Listen for `message` events — deduplicate at app layer with `hyper-p2p-dedup-filter` when needed.
4. `close()` — destroy swarm and clear peer message map.
@@ -21,6 +21,16 @@ class HyperP2PGossipMesh extends EventEmitter {
this._peerMsgs = null this._peerMsgs = null
} }
seenCount () { return this._seen.size }
hasSeen (id) { return this._seen.has(id) }
clearSeen () {
const n = this._seen.size
this._seen.clear()
return n
}
subscribe (filter) { subscribe (filter) {
this._filter = typeof filter === 'function' ? filter : () => true this._filter = typeof filter === 'function' ? filter : () => true
return () => { this._filter = () => true } return () => { this._filter = () => true }
@@ -76,7 +86,7 @@ class HyperP2PGossipMesh extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, seen: this._seen.size, protocol: PROTOCOL }
} }
async close () { async close () {
@@ -33,6 +33,17 @@ class HyperP2PAnycastSelector extends EventEmitter {
if (this._peerMsgs) gossipSend(this, { type: 'latency', peerId, ms }) if (this._peerMsgs) gossipSend(this, { type: 'latency', peerId, ms })
} }
listTags () { return [...this._tags.keys()] }
peersForTag (tag) {
const s = this._tags.get(tag)
return s ? [...s] : []
}
hasCapability (tag, peerId) {
return this._tags.get(tag)?.has(peerId) ?? false
}
resolve (tag) { resolve (tag) {
if (!tag) throw new Error('tag required') if (!tag) throw new Error('tag required')
const peers = this._tags.get(tag) const peers = this._tags.get(tag)
@@ -48,7 +59,9 @@ class HyperP2PAnycastSelector extends EventEmitter {
return best return best
} }
getStats () { return { ...this._stats, tags: this._tags.size } } getStats () {
return { ...this._stats, tags: this._tags.size, latencyPeers: this._latency.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
@@ -40,6 +40,10 @@ class HyperP2PBandwidthBroker extends EventEmitter {
balance (peerId) { return this._balances.get(peerId) || 0 } balance (peerId) { return this._balances.get(peerId) || 0 }
listPeerIds () { return [...this._balances.keys()] }
hasBalance (peerId) { return this.balance(peerId) > 0 }
request (peerId, bytes) { request (peerId, bytes) {
if (!peerId || bytes < 0) throw new Error('invalid request') if (!peerId || bytes < 0) throw new Error('invalid request')
if (this._peerMsgs) gossipSend(this, { type: 'request', peerId, bytes }) if (this._peerMsgs) gossipSend(this, { type: 'request', peerId, bytes })
@@ -47,7 +51,9 @@ class HyperP2PBandwidthBroker extends EventEmitter {
return this.balance(peerId) >= bytes return this.balance(peerId) >= bytes
} }
getStats () { return { ...this._stats, peers: this._balances.size } } getStats () {
return { ...this._stats, peers: this._balances.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
+11 -1
View File
@@ -81,7 +81,17 @@ class HyperP2PCircuitLoom extends EventEmitter {
return c ? { ...c, hops: [...c.hops] } : null return c ? { ...c, hops: [...c.hops] } : null
} }
getStats () { return { ...this._stats, open: [...this._circuits.values()].filter(c => c.state === 'open').length } } listCircuitIds () { return [...this._circuits.keys()] }
hasCircuit (circuitId) { return this._circuits.has(circuitId) }
openCount () {
return [...this._circuits.values()].filter((c) => c.state === 'open').length
}
getStats () {
return { ...this._stats, open: this.openCount(), total: this._circuits.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
@@ -26,6 +26,12 @@ class HyperP2PCongestionSignal extends EventEmitter {
return s return s
} }
listPeerIds () { return [...this._samples.keys()] }
getSample (peerId) { return this._samples.get(peerId) || null }
listSamples () { return [...this._samples.values()] }
getHint (peerId) { getHint (peerId) {
const s = this._samples.get(peerId) const s = this._samples.get(peerId)
if (!s) return { sendRateBps: this.baseRateBps } if (!s) return { sendRateBps: this.baseRateBps }
@@ -38,7 +44,9 @@ class HyperP2PCongestionSignal extends EventEmitter {
return h.sendRateBps < this.baseRateBps * 0.5 return h.sendRateBps < this.baseRateBps * 0.5
} }
getStats () { return { ...this._stats, peers: this._samples.size } } getStats () {
return { ...this._stats, peers: this._samples.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
@@ -39,6 +39,15 @@ class HyperP2PConnectionPool extends EventEmitter {
return lane return lane
} }
hasPeer (peerId) { return this._lanes.has(peerId) }
listPeerIds () { return [...this._lanes.keys()] }
isOpen (peerId) {
const lane = this._lanes.get(peerId)
return lane ? lane.state === 'open' : false
}
release (peerId) { release (peerId) {
if (!peerId) throw new Error('peerId required') if (!peerId) throw new Error('peerId required')
const lane = this._lanes.get(peerId) const lane = this._lanes.get(peerId)
@@ -70,7 +79,9 @@ class HyperP2PConnectionPool extends EventEmitter {
} }
} }
getStats () { return this.getPoolStats() } getStats () {
return { ...this.getPoolStats(), protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
+12 -1
View File
@@ -37,6 +37,16 @@ class HyperP2PFlowShaper extends EventEmitter {
return item return item
} }
queueDepth (priority) {
if (priority) return this._queues[priority]?.length ?? 0
return PRIOS.reduce((n, p) => n + this._queues[p].length, 0)
}
peek (priority = 'control') {
const q = this._queues[priority]
return q?.length ? q[0] : null
}
drain () { drain () {
const out = [] const out = []
for (const p of PRIOS) { for (const p of PRIOS) {
@@ -59,7 +69,8 @@ class HyperP2PFlowShaper extends EventEmitter {
getStats () { getStats () {
return { return {
...this._stats, ...this._stats,
queued: PRIOS.reduce((n, p) => n + this._queues[p].length, 0) queued: this.queueDepth(),
protocol: PROTOCOL
} }
} }
+33 -30
View File
@@ -16,68 +16,71 @@ const mod = new HyperP2PLinkProbe(opts)
| Option | Type | Default | Description | | Option | Type | Default | Description |
|--------|------|---------|-------------| |--------|------|---------|-------------|
| `topic` | varies | null | topic | | `topic` | Buffer \| string \| null | `null` | Hyperswarm topic; omit for local-only |
| `keyPair` | KeyPair | random Ed25519 | keyPair | | `keyPair` | KeyPair | random Ed25519 | Discovery identity |
| `enableBackgroundTimers` | boolean | `false` | Periodic timers (off in tests) | | `enableBackgroundTimers` | boolean | `false` | Periodic timers (off in tests) |
## Methods ## Methods
### `probe(peerId)` ### `probe(peerId) → { peerId, sent }`
- **Returns:** `value` Sends a ping gossip frame and records `lastSent`. **Throws:** `Error: peerId required`.
- **Throws:**
- `Error: peerId required`
### `pong(peerId, sent)` ### `pong(peerId, sent) → record`
- **Returns:** `value` Computes RTT from `sent` timestamp, updates jitter vs previous RTT, emits `pong`.
- **Throws:** — (none documented in method body)
### `getMetrics(peerId)` ### `getMetrics(peerId) → record | null`
- **Returns:** `value` Per-peer `{ peerId, rttMs, jitterMs, probes, lastSent }`.
- **Throws:** — (none documented in method body)
### `publishMatrix(—)` ### `listPeerIds() → string[]`
- **Returns:** `value` All peers with metric rows.
- **Throws:** — (none documented in method body)
### `getStats(—)` ### `listMetrics() → record[]`
- **Returns:** `object` Shallow copy of all metric rows.
- **Throws:** — (none documented in method body)
### `ready(—)` ### `hasPeer(peerId) → boolean`
- **Returns:** `Promise` Whether metrics exist for `peerId`.
- **Throws:** — (none documented in method body)
### `close(—)` ### `bestPeer() → string | null`
- **Returns:** `Promise<void>` Peer id with lowest RTT (ignores peers without RTT yet).
- **Throws:** — (none documented in method body)
### `publishMatrix() → record[]`
Gossip full matrix snapshot; returns local metrics array.
### `getStats() → object`
`{ probes, pongs, entries, protocol }`.
### `async ready()` / `async close()`
Join swarm when `topic` set; `close` clears metrics and destroys swarm.
## Events ## Events
| Event | Payload | | Event | Payload |
|-------|---------| |-------|---------|
| `closed` | no payload | | `closed` | no payload |
| `ping` | payload object | | `ping` | `{ peerId, sent }` |
| `pong` | payload object | | `pong` | `{ peerId, rttMs }` |
## getStats() ## getStats()
Returns `{ ...this._stats }` — typically `ops`, `errors`, and module-specific counters (`created`, `relays`, `open`, `peers`, etc.). Returns `{ probes, pongs, entries, protocol: 'link-probe/v1' }`.
Library-only modules may include `mode: 'local'`.
## Errors ## Errors
Stable message substrings: see [`../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md). Stable message substrings: see [`../../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
## P2P ## P2P
When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `link-probe/v1`. When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `link-probe/v1`. Handles `ping`, `pong`, and `matrix` gossip.
## Testing ## Testing
+18 -1
View File
@@ -43,13 +43,30 @@ class HyperP2PLinkProbe extends EventEmitter {
getMetrics (peerId) { return this._metrics.get(peerId) || null } getMetrics (peerId) { return this._metrics.get(peerId) || null }
listPeerIds () { return [...this._metrics.keys()] }
listMetrics () { return [...this._metrics.values()] }
hasPeer (peerId) { return this._metrics.has(peerId) }
bestPeer () {
let best = null
for (const rec of this._metrics.values()) {
if (rec.rttMs == null) continue
if (!best || rec.rttMs < best.rttMs) best = rec
}
return best ? best.peerId : null
}
publishMatrix () { publishMatrix () {
const matrix = [...this._metrics.values()] const matrix = [...this._metrics.values()]
if (this._peerMsgs) gossipSend(this, { type: 'matrix', matrix }) if (this._peerMsgs) gossipSend(this, { type: 'matrix', matrix })
return matrix return matrix
} }
getStats () { return { ...this._stats, entries: this._metrics.size } } getStats () {
return { ...this._stats, entries: this._metrics.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
@@ -53,6 +53,10 @@ class HyperP2PMultipathFanout extends EventEmitter {
return false return false
} }
listPendingIds () { return [...this._pending.keys()] }
hasPending (msgId) { return this._pending.has(msgId) }
reassemble (msgId) { reassemble (msgId) {
const p = this._pending.get(msgId) const p = this._pending.get(msgId)
if (!p || p.shards.size !== p.total) return null if (!p || p.shards.size !== p.total) return null
@@ -61,7 +65,9 @@ class HyperP2PMultipathFanout extends EventEmitter {
return b4a.concat(parts) return b4a.concat(parts)
} }
getStats () { return { ...this._stats, pending: this._pending.size } } getStats () {
return { ...this._stats, pending: this._pending.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
@@ -46,6 +46,10 @@ class HyperP2POverlayTopology extends EventEmitter {
return [...this._neighbors.values()] return [...this._neighbors.values()]
} }
hasNeighbor (peerId) { return this._neighbors.has(peerId) }
neighborCount () { return this._neighbors.size }
suggestReplacement (failedPeer) { suggestReplacement (failedPeer) {
if (!failedPeer) throw new Error('failedPeer required') if (!failedPeer) throw new Error('failedPeer required')
let best = null let best = null
@@ -57,7 +61,9 @@ class HyperP2POverlayTopology extends EventEmitter {
return best return best
} }
getStats () { return { ...this._stats, degree: this._neighbors.size } } getStats () {
return { ...this._stats, degree: this._neighbors.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
@@ -49,7 +49,13 @@ class HyperP2PProtocolHandshake extends EventEmitter {
getAgreed (peerId) { return this._agreed.get(peerId) || null } getAgreed (peerId) { return this._agreed.get(peerId) || null }
getStats () { return { ...this._stats, pending: this._offers.size } } listPendingOffers () { return [...this._offers.values()] }
hasAgreed (peerId) { return this._agreed.has(peerId) }
getStats () {
return { ...this._stats, pending: this._offers.size, agreed: this._agreed.size, protocol: PROTOCOL }
}
async ready () { async ready () {
if (this.swarm || !this.topic) return this if (this.swarm || !this.topic) return this
+21 -7
View File
@@ -1,11 +1,25 @@
# Scheduling & queues # Scheduling & queues
**Path:** `modules/scheduling-queues/` · **Modules:** 5 (5 production, 0 scaffold) **Path:** `modules/scheduling-queues/` · **Modules:** 5 (all production)
See [MODULE_CATEGORIES.md](../MODULE_CATEGORIES.md#scheduling-queues). Leader leases, peer cron, activity queues, and deadline ordering. Hub: [`../../docs/scheduling-queues/README.md`](../../docs/scheduling-queues/README.md).
- [hyper-p2p-activity-queue](./hyper-p2p-activity-queue/) — production ## Modules
- [hyper-p2p-cron-gossip](./hyper-p2p-cron-gossip/) — production
- [hyper-p2p-deadline-queue](./hyper-p2p-deadline-queue/) — production | Module | Protocol | Summary |
- [hyper-p2p-peer-scheduler](./hyper-p2p-peer-scheduler/) — production |--------|----------|---------|
- [hyper-p2p-topic-lease](./hyper-p2p-topic-lease/) — production | [hyper-p2p-topic-lease](./hyper-p2p-topic-lease/) | `topic-lease/v1` | Shard leader election (`listShards`, `isHeld`) |
| [hyper-p2p-peer-scheduler](./hyper-p2p-peer-scheduler/) | `peer-scheduler/v1` | Cron-like jobs with lease-aware leader (`listJobs`) |
| [hyper-p2p-cron-gossip](./hyper-p2p-cron-gossip/) | `cron-gossip/v1` | Gossiped schedule registry |
| [hyper-p2p-activity-queue](./hyper-p2p-activity-queue/) | `activity-queue/v1` | Priority queue + vector-clock ordering |
| [hyper-p2p-deadline-queue](./hyper-p2p-deadline-queue/) | `deadline-queue/v1` | Time-ordered deadline heap |
## Composition
`peer-scheduler` accepts `topicLease` — only the lease holder runs `tick` for a shard.
## Test
```bash
cd hyper-p2p-topic-lease && npm test
```
@@ -176,7 +176,14 @@ class HyperP2PActivityQueue extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
pending: this.getQueueDepth(),
total: this._queue.length,
claimed: this._claimed.size,
deadLetter: this._deadLetter.length,
protocol: PROTOCOL
}
} }
async close () { async close () {
@@ -42,6 +42,12 @@ class HyperP2PPeerScheduler extends EventEmitter {
return id return id
} }
listJobs () { return [...this._jobs.values()] }
hasJob (jobId) { return this._jobs.has(jobId) }
jobCount () { return this._jobs.size }
cancel (jobId) { cancel (jobId) {
const ok = this._jobs.delete(jobId) const ok = this._jobs.delete(jobId)
if (ok) this.emit('cancelled', { jobId }) if (ok) this.emit('cancelled', { jobId })
@@ -94,7 +100,7 @@ class HyperP2PPeerScheduler extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, jobs: this._jobs.size, protocol: PROTOCOL }
} }
async close () { async close () {
@@ -64,6 +64,18 @@ class HyperP2PTopicLease extends EventEmitter {
return this.holder(topicShard) return this.holder(topicShard)
} }
listShards () { return [...this._leases.keys()] }
isHeld (topicShard) {
const h = this.holder(topicShard)
return h != null
}
leaseCount () {
const now = Date.now()
return [...this._leases.values()].filter((l) => l.expiresAt > now).length
}
_expireSweep () { _expireSweep () {
const now = Date.now() const now = Date.now()
for (const [shard, lease] of this._leases) { for (const [shard, lease] of this._leases) {
@@ -100,7 +112,7 @@ class HyperP2PTopicLease extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, leases: this.leaseCount(), shards: this._leases.size, protocol: PROTOCOL }
} }
async close () { async close () {
+40 -11
View File
@@ -1,16 +1,45 @@
# State & CRDTs # State & CRDTs
**Path:** `modules/state-crdts/` · **Modules:** 10 (10 production, 0 scaffold) **Path:** `modules/state-crdts/` · **Modules:** 10 (all production)
Collaborative replicated types and reactive key/value state for Bare/Pear apps. Doc hub: [`../../docs/state-crdts/README.md`](../../docs/state-crdts/README.md).
See [MODULE_CATEGORIES.md](../MODULE_CATEGORIES.md#state-crdts). See [MODULE_CATEGORIES.md](../MODULE_CATEGORIES.md#state-crdts).
- [hyper-p2p-conflict-set](./hyper-p2p-conflict-set/) — production ## When to use
- [hyper-p2p-crdt-grow-only-set](./hyper-p2p-crdt-grow-only-set/) — production
- [hyper-p2p-crdt-lww-register](./hyper-p2p-crdt-lww-register/) — production - Multi-writer maps, counters, text, or sets without a central server.
- [hyper-p2p-crdt-map](./hyper-p2p-crdt-map/) — production - Causal or reactive UI state synced over Hyperswarm.
- [hyper-p2p-crdt-or-map](./hyper-p2p-crdt-or-map/) — production - Pair with [`hyper-p2p-gossip-mesh`](../messaging-gossip/hyper-p2p-gossip-mesh/) or [`applications-collab`](../applications-collab/).
- [hyper-p2p-crdt-pn-counter](./hyper-p2p-crdt-pn-counter/) — production
- [hyper-p2p-crdt-rga-text](./hyper-p2p-crdt-rga-text/) — production ## Quick start
- [hyper-p2p-crdt-two-phase-set](./hyper-p2p-crdt-two-phase-set/) — production
- [hyper-p2p-crdt-version-vector](./hyper-p2p-crdt-version-vector/) — production ```js
- [hyper-p2p-reactive-state](./hyper-p2p-reactive-state/) — production const { HyperP2PCrdtMap } = require('hyper-p2p-crdt-map')
const map = new HyperP2PCrdtMap({ topic: 'collab-demo' })
await map.ready()
map.set('title', 'Hello')
console.log(map.get('title'), map.size())
```
## Modules
| Module | Protocol | Role |
|--------|----------|------|
| [hyper-p2p-crdt-map](./hyper-p2p-crdt-map/) | `crdt-map/v1` | LWW map (`has`, `size`, `entries`) |
| [hyper-p2p-crdt-lww-register](./hyper-p2p-crdt-lww-register/) | `crdt-lww-register/v1` | Per-key LWW register |
| [hyper-p2p-crdt-or-map](./hyper-p2p-crdt-or-map/) | `crdt-or-map/v1` | Observed-remove map |
| [hyper-p2p-crdt-pn-counter](./hyper-p2p-crdt-pn-counter/) | `crdt-pn-counter/v1` | PN-counter |
| [hyper-p2p-crdt-grow-only-set](./hyper-p2p-crdt-grow-only-set/) | `crdt-grow-only-set/v1` | G-set |
| [hyper-p2p-crdt-two-phase-set](./hyper-p2p-crdt-two-phase-set/) | `crdt-two-phase-set/v1` | 2P-set add/remove |
| [hyper-p2p-crdt-version-vector](./hyper-p2p-crdt-version-vector/) | `crdt-version-vector/v1` | Version vectors |
| [hyper-p2p-crdt-rga-text](./hyper-p2p-crdt-rga-text/) | `crdt-rga-text/v1` | RGA collaborative text |
| [hyper-p2p-conflict-set](./hyper-p2p-conflict-set/) | `conflict-set/v1` | Dot-tracked set (`size`, `isEmpty`) |
| [hyper-p2p-reactive-state](./hyper-p2p-reactive-state/) | `reactive-state/v1` | Hyperbee-backed reactive KV |
## Test
```bash
cd hyper-p2p-crdt-map && npm install && npm test
../../real_tests/run-all.sh --tier=production
```
+5 -1
View File
@@ -67,6 +67,10 @@ class HyperP2PConflictSet extends EventEmitter {
return out return out
} }
size () { return this.values().length }
isEmpty () { return this.size() === 0 }
merge (remote) { merge (remote) {
if (!remote || !remote.elements) return false if (!remote || !remote.elements) return false
let changed = false let changed = false
@@ -111,7 +115,7 @@ class HyperP2PConflictSet extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, live: this.size(), tracked: this._elements.size, protocol: PROTOCOL }
} }
async close () { async close () {
+36 -40
View File
@@ -6,7 +6,7 @@
## Overview ## Overview
Production state & crdts module: Hyperswarm discovery + Protomux when `topic` is set. Last-writer-wins map CRDT: per-key cells with timestamp + peer tie-break. Gossip `set` on local writes; `merge` for snapshots.
## Constructor ## Constructor
@@ -16,81 +16,77 @@ const mod = new HyperP2PCrdtMap(opts)
| Option | Type | Default | Description | | Option | Type | Default | Description |
|--------|------|---------|-------------| |--------|------|---------|-------------|
| `topic` | varies | null | topic | | `topic` | Buffer \| string \| null | `null` | Hyperswarm topic |
| `keyPair` | KeyPair | random Ed25519 | keyPair | | `keyPair` | KeyPair | random Ed25519 | Writer identity (`peerId` hex) |
## Methods ## Methods
### `set(key, value)` ### `set(key, value) → boolean`
- **Returns:** `value` Applies LWW cell `{ key, value, ts, peerId }`. Returns `true` if local write wins. Gossip on success.
- **Throws:** — (none documented in method body)
### `get(key)` ### `get(key) → value | undefined`
- **Returns:** `value` Current winning value (not tombstone metadata).
- **Throws:** — (none documented in method body)
### `delete(key)` ### `delete(key) → boolean`
- **Returns:** `value` `set(key, null)` tombstone.
- **Throws:** — (none documented in method body)
### `keys()` ### `keys() → string[]`
- **Returns:** `value` All cell keys.
- **Throws:** — (none documented in method body)
### `merge(remote)` ### `has(key) → boolean`
- **Returns:** `value` Whether a cell exists (including tombstoned).
- **Throws:** — (none documented in method body)
### `toJSON(—)` ### `size() → number`
- **Returns:** `value` Cell count.
- **Throws:** — (none documented in method body)
### `ready(—)` ### `entries() → [key, value][]`
- **Returns:** `Promise` Live values only.
- **Throws:** — (none documented in method body)
### `getStats(—)` ### `values() → any[]`
- **Returns:** `object` All winning values.
- **Throws:** — (none documented in method body)
### `close(—)` ### `merge(remote) → number`
- **Returns:** `Promise<void>` `remote.cells` array; returns count of updated keys.
- **Throws:** — (none documented in method body)
### `toJSON() → { cells }`
Snapshot for persistence.
### `getStats() → object`
`{ ops, errors, cells, protocol }`.
### `async ready()` / `async close()`
Protomux `crdt-map/v1` when `topic` set.
## Events ## Events
| Event | Payload | | Event | Payload |
|-------|---------| |-------|---------|
| `closed` | no payload | | `closed` | no payload |
| `merge` | updated | | `merge` | `{ updated }` |
| `set` | cell | | `set` | cell |
## getStats() ## getStats()
Returns `{ ...this._stats }` — typically `ops`, `errors`, and module-specific counters (`created`, `relays`, `open`, `peers`, etc.). `{ ops, errors, cells, protocol: 'crdt-map/v1' }`.
Library-only modules may include `mode: 'local'`.
## Errors
Stable message substrings: see [`../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
## P2P ## P2P
When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `crdt-map/v1`. Gossip type `set` with `{ cell }`.
## Testing ## Testing
```bash ```bash
npm install && npm test npm install && npm test
``` ```
Integration: [`../../real_tests/integration/crdt-map-two-node.js`](../../../real_tests/integration/crdt-map-two-node.js)
+13 -1
View File
@@ -50,6 +50,18 @@ class HyperP2PCrdtMap extends EventEmitter {
return [...this._cells.keys()] return [...this._cells.keys()]
} }
has (key) { return this._cells.has(key) }
size () { return this._cells.size }
entries () {
return [...this._cells.entries()].map(([k, c]) => [k, c.value])
}
values () {
return [...this._cells.values()].map((c) => c.value)
}
merge (remote) { merge (remote) {
if (!remote || !remote.cells) return 0 if (!remote || !remote.cells) return 0
let n = 0 let n = 0
@@ -83,7 +95,7 @@ class HyperP2PCrdtMap extends EventEmitter {
getStats () { getStats () {
return { ...this._stats } return { ...this._stats, cells: this._cells.size, protocol: PROTOCOL }
} }
async close () { async close () {
+12 -1
View File
@@ -386,9 +386,20 @@ class HyperP2PReactiveState extends EventEmitter {
})) }))
} }
size () { return this.state.size }
has (key) { return this.state.has(key) }
keys () { return [...this.state.keys()] }
getStats () { getStats () {
return { ...this._stats } return {
...this._stats,
keys: this.state.size,
peers: this.peers.size,
joined: this._joined,
protocol: 'reactive-state/v1'
}
} }
async close () { async close () {