Updates

2026-05-20 23:36:32 -04:00
parent a020270cb1
commit be94546cd3
218 changed files with 9189 additions and 3078 deletions
@@ -1,37 +1,31 @@
 # hyper-p2p-cron-gossip

-Cron scheduling with `cron-fire` gossip over Hyperswarm.
+Distributed **cron scheduling** with local `tick()` and remote `cron-fire` gossip.

-**Category:** scheduling-queues
-
-**Protocol:** `cron-gossip/v1`
+**Category:** Scheduling & queues · **Protocol:** `cron-gossip/v1`

 ## When to use

-Coordinating periodic jobs across peers and observing remote fires.
+Cluster-wide periodic tasks, `@every` intervals, or classic five-field UTC cron strings shared on a topic.

 ## When not to use

-One-shot deadline tasks (use hyper-p2p-deadline-queue).
+Sub-second precision timers, single-process `setInterval` only, or durable job queues (use deadline/activity queues).

 ## Quick start

 ```js
 const { HyperP2PCronGossip } = require('hyper-p2p-cron-gossip')
-const c = new HyperP2PCronGossip({ topic: process.argv[2] })
-await c.ready()
-c.schedule('@every 30s', 'heartbeat')
-console.log(c.tick(Date.now()))
-await c.close()
+const cron = new HyperP2PCronGossip({ topic: process.argv[2] })
+await cron.ready()
+cron.schedule('@every 5s', 'heartbeat')
+setInterval(() => cron.tick(), 1000)
 ```

 ## Docs

- [docs/api.md](docs/api.md)
- [docs/architecture.md](docs/architecture.md)
+[docs/api.md](docs/api.md) · [docs/architecture.md](docs/architecture.md)

 ## Test

-```bash
-npm install && npm test
-```
+`npm install && npm test` · `bare examples/basic.js [topic]`
@@ -1,20 +1,98 @@
 # API: hyper-p2p-cron-gossip

-**Export:** `HyperP2PCronGossip`
+**Protocol:** `cron-gossip/v1`
+
+**Export:** `{ HyperP2PCronGossip, PROTOCOL }`
+
+## Overview
+
+`HyperP2PCronGossip` stores jobs in a local `Map` keyed by `jobId`. `schedule(expr, jobId)` registers an expression. `tick(now)` evaluates due jobs, emits `fire`, and gossips `cron-fire` to peers. Remote fires update `lastRun` and emit `remote-fire`. Expressions support `@every Nms|s|m|h` or five-field UTC cron (`minute hour day month weekday`).
+
+## Constructor
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `topic` | `string` \| `Buffer` \| `null` | `null` | Hyperswarm topic |
+| `keyPair` | `KeyPair` | `hypercore-crypto.keyPair()` | Swarm identity; `peerHex` derived for gossip |

 ## Methods

-### `schedule(expr, jobId)` / `tick(now)` / `unschedule(jobId)`
+### `schedule(expr, jobId)`

-Supports `@every Nms|s|m|h` or 5-field `min hour day month weekday` (UTC).
+- **Returns:** `{ jobId, expr, lastRun, at }` job record
+- **Throws:** `ValidationError` if `jobId` or `expr` empty
+- **Emits:** `schedule`

-Gossip: `cron-fire` with `{ jobId, expr, peer, at }`.
+### `unschedule(jobId)`

-### `list()` / `get(jobId)` / `getStats()` / `ready()` / `close()`
+- **Returns:** `boolean` — whether job existed
+
+### `tick(now = Date.now())`
+
+Evaluates all jobs; fires due ones.
+
+- **Returns:** `string[]` — fired `jobId` list
+- **Emits:** `fire` per local job
+
+### `list()` / `get(jobId)`
+
+Snapshots of registered jobs.
+
+### `async ready()` / `async close()`
+
+`close()` clears jobs, destroys swarm, emits `closed`.
+
+### `getStats()`
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `scheduled` | `number` | `schedule` calls |
+| `fired` | `number` | Local tick fires |
+| `gossipIn` / `gossipOut` | `number` | Wire counters |
+| `jobs` | `number` | Map size |
+| `protocol` | `string` | `cron-gossip/v1` |

 ## Events

-| Event | Payload |
-|-------|---------|
-| `fire` | `{ jobId, at }` |
-| `remote-fire` | `{ jobId, peer, at }` |
+| Event | Payload | When |
+|-------|---------|------|
+| `schedule` | job object | `schedule()` |
+| `fire` | `{ jobId, at }` | Local `tick` fired job |
+| `remote-fire` | `{ jobId, peer, at }` | Inbound `cron-fire` |
+| `closed` | — | `close()` |
+
+## getStats()
+
+Shallow copy with live `jobs` count.
+
+## Wire
+
+| type | fields | direction | behavior |
+|------|--------|-----------|----------|
+| `cron-fire` | `jobId`, `expr`, `peer`, `at` | bidirectional | Update/create job `lastRun`; emit `remote-fire` |
+
+Gossip only sends when `_peerMsgs` is populated (after `ready()` + connections).
+
+## Cron expression helpers
+
+| Form | Example | Semantics |
+|------|---------|-----------|
+| `@every` | `@every 30s` | Interval since `lastRun` |
+| Five-field | `0 * * * *` | UTC minute/hour/day/month/weekday match |
+
+## Errors
+
+| Message | Source |
+|---------|--------|
+| `jobId is required` / `expr is required` | `schedule` |
+| `topic is required for createSwarm` | swarm |
+
+## Testing
+
+```bash
+cd modules/scheduling-queues/hyper-p2p-cron-gossip
+npm install && npm test
+bare examples/basic.js
+```
+
+Drive `tick` with injected `now` for deterministic tests.
@@ -1,18 +1,25 @@
 # Architecture: hyper-p2p-cron-gossip

-**Category:** scheduling-queues
+**Protocol:** `cron-gossip/v1`

 ```mermaid
-flowchart LR
-  App[Application] --> Mod[HyperP2PCronGossip]
-  Mod --> Mux[Protomux cron-gossip/v1]
-  Mux --> Swarm[Hyperswarm]
+flowchart TB
+  App --> CG[HyperP2PCronGossip]
+  CG --> Jobs["_jobs Map"]
+  CG --> Tick[tick dueCron]
+  Tick --> Fire[cron-fire gossip]
 ```

 ## Wire messages

-| type | fields | behavior |
-|------|--------|----------|
-| `cron-fire` | `jobId`, `expr`, `peer`, `at` | Update `lastRun`, emit `remote-fire` |
+| type | fields | direction | behavior |
+|------|--------|-----------|----------|
+| `cron-fire` | `jobId`, `expr`, `peer`, `at` | bidirectional | Sync `lastRun`, learn unknown jobs |

-Uses `../../_shared/p2p-bare.js`.
+## State model
+
+Job: `{ jobId, expr, lastRun, at }`.
+
+## Composition
+
+Pair with `hyper-p2p-leader-lease` so only the leader schedules cluster-wide jobs; use `hyper-p2p-deadline-queue` for per-task deadlines after fire.
@@ -2,11 +2,12 @@ require('bare-process/global')
 const { HyperP2PCronGossip } = require('../index.js')

 async function main () {
-  const c = new HyperP2PCronGossip()
-  c.schedule('@every 100ms', 'ping')
-  const fired = c.tick(Date.now() + 150)
-  console.log('[cron-gossip]', fired, c.getStats())
-  await c.close()
+  const cron = new HyperP2PCronGossip({ topic: process.argv[2] || null })
+  await cron.ready()
+  cron.schedule('@every 1s', 'demo')
+  const fired = cron.tick(Date.now() + 2000)
+  console.log('fired:', fired, cron.getStats())
+  await cron.close()
 }

 main().catch(console.error)
@@ -1,35 +1,30 @@
 # hyper-p2p-deadline-queue

-Priority queue ordered by deadline with `drain(now)` for due tasks.
+In-memory **deadline-sorted task queue** (local process; no Hyperswarm wire).

-**Category:** scheduling-queues
-
-**Protocol:** `deadline-queue/v1`
+**Category:** Scheduling & queues · **Protocol:** `deadline-queue/v1` (identifier only)

 ## When to use

-Scheduling work items that must run before a timestamp (ms).
+Run callbacks or payloads when `deadline` timestamps elapse; priority by soonest due.

 ## When not to use

-Recurring cron-style jobs (use hyper-p2p-cron-gossip).
+Cross-peer scheduling (use `hyper-p2p-cron-gossip`). Durable queues across restarts.

 ## Quick start

 ```js
 const { HyperP2PDeadlineQueue } = require('hyper-p2p-deadline-queue')
 const q = new HyperP2PDeadlineQueue()
-q.enqueue('t1', { run: true }, Date.now() + 1000)
-console.log(q.drain(Date.now() + 2000))
+q.enqueue('t1', { run: true }, Date.now() + 5000)
+q.drain(Date.now() + 6000)
 ```

 ## Docs

- [docs/api.md](docs/api.md)
- [docs/architecture.md](docs/architecture.md)
+[docs/api.md](docs/api.md) · [docs/architecture.md](docs/architecture.md)

 ## Test

-```bash
-npm install && npm test
-```
+`npm install && npm test` · `bare examples/basic.js`
@@ -1,11 +1,106 @@
 # API: hyper-p2p-deadline-queue

-**Export:** `HyperP2PDeadlineQueue`
+**Protocol:** `deadline-queue/v1` (metadata constant; no network transport)
+
+**Export:** `{ HyperP2PDeadlineQueue, PROTOCOL }`
+
+## Overview
+
+`HyperP2PDeadlineQueue` is a local `EventEmitter` priority queue ordered by `deadline` ms timestamp. `enqueue` replaces duplicate ids. `drain(now)` extracts all entries with `deadline <= now`. No P2P: `ready()` resolves immediately; use alongside cron or consensus modules for distributed triggers.
+
+## Constructor
+
+```js
+const q = new HyperP2PDeadlineQueue()
+```
+
+No options.

 ## Methods

-### `enqueue(id, task, deadline)` / `drain(now)` / `peek()`
+### `enqueue(id, task, deadline)`

-`drain` returns all entries with `deadline <= now`.
+- **Parameters:**
+  - `id` — unique string id
+  - `task` — arbitrary payload (must not be null)
+  - `deadline` — finite ms timestamp
+- **Returns:** entry `{ id, task, deadline, at }`
+- **Throws:**
+  - `ValidationError: id is required`
+  - `Error: task required`
+  - `Error: deadline must be a number (ms timestamp)`
+- **Emits:** `enqueue`

-### `cancel(id)` / `get(id)` / `pending()` / `list()` / `getStats()` / `close()`
+### `peek()`
+
+- **Returns:** shallow copy of earliest entry or `null`
+
+### `drain(now = Date.now())`
+
+- **Returns:** `entry[]` due items (full objects)
+- **Emits:** `drain` with `{ now, items: id[] }` when non-empty
+
+### `cancel(id)`
+
+- **Returns:** `boolean`
+- **Emits:** `cancel`
+
+### `get(id)` / `pending()` / `list()`
+
+Query helpers; `list()` returns `{ id, deadline }` only.
+
+### `async ready()`
+
+No-op resolve for API symmetry.
+
+### `async close()`
+
+Clears queue, emits `closed`.
+
+### `getStats()`
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `enqueued` | `number` | Successful enqueues |
+| `drained` | `number` | Entries removed by `drain` |
+| `pending` | `number` | Queue length |
+| `protocol` | `string` | `deadline-queue/v1` |
+
+## Events
+
+| Event | Payload | When |
+|-------|---------|------|
+| `enqueue` | entry | New/replaced task |
+| `drain` | `{ now, items }` | One or more tasks due |
+| `cancel` | `{ id }` | Removed by id |
+| `closed` | — | `close()` |
+
+## getStats()
+
+Shallow copy; `pending` is live length.
+
+## Wire
+
+No wire messages. `PROTOCOL` is exported for module registry consistency only.
+
+## Errors
+
+| Message | Source |
+|---------|--------|
+| `id is required` | `enqueue` / `cancel` |
+| `task required` | `enqueue` |
+| `deadline must be a number (ms timestamp)` | invalid deadline |
+
+## Testing
+
+```bash
+cd modules/scheduling-queues/hyper-p2p-deadline-queue
+npm install && npm test
+bare examples/basic.js
+```
+
+Pass explicit `now` to `drain` for deterministic ordering tests.
+
+## Composition
+
+Run a timer loop: `setInterval(() => { const due = q.drain(); due.forEach(runTask) }, 100)` after enqueuing work from `hyper-p2p-cron-gossip` `fire` handlers.
@@ -1,5 +1,27 @@
 # Architecture: hyper-p2p-deadline-queue

-**Category:** scheduling-queues
+**Protocol:** `deadline-queue/v1` (local only)

-Sorted array by deadline plus `_byId` index for cancel/replace. `peek` returns earliest entry without removal.
+```mermaid
+flowchart LR
+  App --> DQ[HyperP2PDeadlineQueue]
+  DQ --> Q["_queue sorted by deadline"]
+  DQ --> Map["_byId Map"]
+```
+
+## Wire messages
+
+| type | fields | direction | behavior |
+|------|--------|-----------|----------|
+| — | — | — | No network layer |
+
+## State model
+
+| Structure | Role |
+|-----------|------|
+| `_queue` | Array sorted ascending by `deadline` |
+| `_byId` | Fast lookup / cancel |
+
+## Composition
+
+Downstream executor for `hyper-p2p-cron-gossip` and `hyper-p2p-task-orchestrator`; not a distributed queue by itself.
@@ -3,9 +3,13 @@ const { HyperP2PDeadlineQueue } = require('../index.js')

 async function main () {
  const q = new HyperP2PDeadlineQueue()
-  const now = Date.now()
-  q.enqueue('job', { msg: 'hi' }, now + 500)
-  console.log('[deadline-queue]', q.peek(), q.drain(now + 600))
+  await q.ready()
+  const t0 = Date.now()
+  q.enqueue('a', { msg: 'soon' }, t0 + 100)
+  q.enqueue('b', { msg: 'later' }, t0 + 5000)
+  console.log('peek:', q.peek())
+  const due = q.drain(t0 + 200)
+  console.log('due:', due.map((e) => e.id), q.getStats())
  await q.close()
 }