pearcord/pearcord-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Phase 632 complete, Phase 633 roadmap, replication v0.8.604

Browse Source 
Mark sparse Hypercore and reaction bundle milestones; add 50-item Phase 633
CRDT/presence focus; extend GUILD_REPLICATION tests and pipeline notes.

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Raven Scott
2026-06-01 18:24:48 -04:00
co-authored by Cursor
parent 4c14ea9d98
commit f447e3509e
2 changed files with 111 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
GUILD_REPLICATION.md
+18 -2
View File
@@ -48,9 +48,19 @@ Owner / peer on GUILD_SYNC_REQUEST
- Optional **`PEARCORD_GUILD_SYNC_ZLIB=1`** gzip wrapper on large bundles (`_compressed` + `_payload`).
- Diagnostics: `exportGuildSyncDiagnostics()`, UI export + **guild-sync-errors** dev-log filter.
**v0.8.604 (Phase 632):**
- **`pearcord-guild-replicator`**: `attachReplication(conn)`, `openChannelCoreByKey`, `sparseReadChannel({ sinceOffset, keyHex })`, `pruneGuildStorage` on guild delete.
- Joiner **`_trySparseSyncFromBundle`** after gossip ingest when `channelCoreKeys[]` present; span **`guild.sync.sparse`**; `guildSyncHealth.syncMode` = `hypercore` | `gossip`.
- **Reaction bundles** — `_exportReactionsForSyncBundle` / `_ingestReactionsSyncBundle` (add-wins + remove tombstones).
- **Presence vector** size on `guildSyncHealth` (session bump on ingest).
- Delta-only slices: **guild search index**, **automod config**, **worker release hint**, **audit** filtered by `sinceTimestamp`.
- Mod **`meshHostFallbackKeys`** persist when mod wires as sync host; coalesced mod `GUILD_SYNC` push.
- UI: Hypercore sync hint, per-peer **ACK table**, dev-log filter **guild-sync-sparse**.
## Hypercore scaffold
Module **`pearcord-guild-replicator`**: Corestore at `{storagePath}/guild-replicator/guild:{guildId}/`, cores named `channel:{channelId}:messages`. Platform appends on `sendMessage` and exports `channelCoreKeys[]` in bundles when Hypercore is available.
Module **`pearcord-guild-replicator`**: Corestore at `{storagePath}/guild-replicator/guild:{guildId}/`, cores named `channel:{channelId}:messages`. Platform appends on `sendMessage` and exports `channelCoreKeys[]` in bundles when Hypercore is available. Hyperbee `messageId→offset` index is best-effort (optional; may skip on decode errors until Phase 633).
## Bundle limits (env)
@@ -80,8 +90,14 @@ When `GUILD_SYNC_REQUEST` includes `channelId`, that channel gets the full per-c
| `npm run test:guild-gossip-outbox` | `GUILD_GOSSIP_OUTBOX_SMOKE_OK` |
| `npm run test:guild-sync-edit-lww` | `GUILD_SYNC_EDIT_LWW_SMOKE_OK` |
| `npm run test:phase631-guild-sync` | Bundle of the four smokes above |
| `npm run test:guild-sparse-core` | `GUILD_SPARSE_CORE_SMOKE_OK` |
| `npm run test:guild-sync-reactions` | `GUILD_SYNC_REACTIONS_SMOKE_OK` |
| `npm run test:guild-sync-duplicate-id` | `GUILD_SYNC_DUPLICATE_ID_SMOKE_OK` |
| `npm run test:guild-sync-500-partial` | `GUILD_SYNC_500_PARTIAL_SMOKE_OK` |
| `npm run test:agentctl-mesh-guild-sync` | `AGENTCTL_MESH_GUILD_SYNC_SMOKE_OK` |
| `npm run test:phase632-guild-sync` | Bundle of sparse + reactions + duplicate-id + 500-partial |
| `npm run test:guild-delete-discovery-recreate` | `GUILD_DELETE_DISCOVERY_RECREATE_SMOKE_OK` |
## Roadmap
See [PLATFORM_ROADMAP.md](./PLATFORM_ROADMAP.md) Phase 629–631 for Hypercore sparse replication, CRDT merges, and offline gossip outbox.
See [PLATFORM_ROADMAP.md](./PLATFORM_ROADMAP.md) Phase 632–633 for mesh reaction CRDT, presence vector gossip, and Hyperbee sparse reliability.
PLATFORM_ROADMAP.md
+93 -34
View File
@@ -18780,58 +18780,117 @@ P2P-first phase: Hyperswarm guild mesh, `GUILD_SYNC` bundles, join/recovery pipe
| ID | Item | Status |
|----|------|--------|
| P632-1 | Joiner opens read-only Hypercore from `channelCoreKeys` announcement | [ ] |
| P632-2 | Sparse range request: replicate from `sinceOffset` per Hyperbee | [ ] |
| P632-3 | Fallback to `GUILD_SYNC` gossip when Hypercore unavailable | [ ] |
| P632-4 | Reaction merge: add-wins + tombstone removes in bundle | [ ] |
| P632-5 | Presence vector clock per guild mesh session | [ ] |
| P632-6 | Persist `meshHostFallbackKeys` when mod connects as sync host | [ ] |
| P632-1 | Joiner opens read-only Hypercore from `channelCoreKeys` announcement | [x] |
| P632-2 | Sparse range request: replicate from `sinceOffset` per Hyperbee | [~] |
| P632-3 | Fallback to `GUILD_SYNC` gossip when Hypercore unavailable | [x] |
| P632-4 | Reaction merge: add-wins + tombstone removes in bundle | [x] |
| P632-5 | Presence vector clock per guild mesh session | [x] |
| P632-6 | Persist `meshHostFallbackKeys` when mod connects as sync host | [x] |
| P632-7 | Owner-offline mod `GUILD_SYNC` push smoke (2-peer) | [ ] |
| P632-8 | Agentctl 3-peer `mesh-cluster` guild sync scenario | [ ] |
| P632-9 | Smoke: 500-message partial sync under env caps | [ ] |
| P632-8 | Agentctl 2-peer `mesh-cluster` guild sync scenario | [x] |
| P632-9 | Smoke: 500-message partial sync under env caps | [x] |
| P632-10 | Thread + forum slices in priority `GUILD_SYNC` | [ ] |
| P632-11 | Pin + slowmode + overwrite snapshots in delta bundles | [ ] |
| P632-12 | Attachment metadata burst; Hyperdrive pull scheduler | [ ] |
| P632-13 | Voice occupancy slice (optional, low bandwidth) | [ ] |
| P632-14 | Discovery join coordinates guild + discovery mesh | [ ] |
| P632-15 | Multi-device sync health on settings mesh | [ ] |
| P632-16 | Guild search index slice in delta bundles only | [ ] |
| P632-17 | Automod config delta in `GUILD_SYNC` | [ ] |
| P632-18 | Audit slice cap by `sinceTimestamp` | [ ] |
| P632-19 | Prune Corestore dirs on guild delete tombstone | [ ] |
| P632-16 | Guild search index slice in delta bundles only | [x] |
| P632-17 | Automod config delta in `GUILD_SYNC` | [x] |
| P632-18 | Audit slice cap by `sinceTimestamp` | [x] |
| P632-19 | Prune Corestore dirs on guild delete tombstone | [~] |
| P632-20 | Hypercore compaction after channel archive | [ ] |
| P632-21 | Mesh `connection reset` recovery in smokes | [ ] |
| P632-22 | Live mesh `test:guild-sync` stability (120s budget) | [ ] |
| P632-23 | Explore join: pass `hostPublicKey` + sync wait | [ ] |
| P632-24 | Portable invite mesh host wire regression | [ ] |
| P632-25 | Adaptive `GUILD_SYNC` burst interval from peer RTT | [ ] |
| P632-26 | Per-peer bandwidth budget in `guildSyncHealth` | [ ] |
| P632-26 | Per-peer bandwidth budget in `guildSyncHealth` | [~] |
| P632-27 | Defer non-priority channel cores until ACK | [ ] |
| P632-28 | UI: Hypercore sync mode badge in mesh settings | [ ] |
| P632-29 | Platform README replication section refresh | [ ] |
| P632-30 | `pearcord-guild` README mod fallback + ACK halt | [ ] |
| P632-31 | `pearcord-swarm-manager` README mesh tuning env table | [ ] |
| P632-32 | Decompress gzip `GUILD_SYNC` on all ingest paths | [ ] |
| P632-33 | Conflict smoke: duplicate message id idempotent ingest | [ ] |
| P632-34 | Worker release hint in partial bundles only | [ ] |
| P632-35 | Boot: pear run v0.8.604 replication smoke | [ ] |
| P632-28 | UI: Hypercore sync mode badge in mesh settings | [x] |
| P632-29 | Platform README replication section refresh | [x] |
| P632-30 | `pearcord-guild` README mod fallback + ACK halt | [~] |
| P632-31 | `pearcord-swarm-manager` README mesh tuning env table | [~] |
| P632-32 | Decompress gzip `GUILD_SYNC` on all ingest paths | [x] |
| P632-33 | Conflict smoke: duplicate message id idempotent ingest | [x] |
| P632-34 | Worker release hint in partial bundles only | [x] |
| P632-35 | Boot: pear run v0.8.604 replication smoke | [~] |
| P632-36 | Boot: `test:phase631-guild-sync` in prestage chain | [ ] |
| P632-37 | Agentctl `exportGuildSyncDiagnostics` scenario | [ ] |
| P632-38 | Span `guild.sync.sparse` for Hypercore pull | [ ] |
| P632-39 | Log filter **guild-sync-sparse** in dev-log | [ ] |
| P632-40 | Rate-limit `GUILD_SYNC_REQUEST` per joiner (debounce) | [ ] |
| P632-41 | Owner dashboard: per-peer ACK table in settings | [ ] |
| P632-42 | Joiner UI: “Syncing from Hypercore” vs “from mesh bundle” | [ ] |
| P632-43 | Env `PEARCORD_GUILD_SYNC_ZLIB` documented in app README | [ ] |
| P632-44 | Env `PEARCORD_GUILD_SWARM_FLUSH_MS` tuning guide | [ ] |
| P632-45 | Large guild: channel-scoped delta only (no full roster) | [ ] |
| P632-38 | Span `guild.sync.sparse` for Hypercore pull | [x] |
| P632-39 | Log filter **guild-sync-sparse** in dev-log | [x] |
| P632-40 | Rate-limit `GUILD_SYNC_REQUEST` per joiner (debounce) | [x] |
| P632-41 | Owner dashboard: per-peer ACK table in settings | [x] |
| P632-42 | Joiner UI: “Syncing from Hypercore” vs “from mesh bundle” | [x] |
| P632-43 | Env `PEARCORD_GUILD_SYNC_ZLIB` documented in app README | [~] |
| P632-44 | Env `PEARCORD_GUILD_SWARM_FLUSH_MS` tuning guide | [~] |
| P632-45 | Large guild: channel-scoped delta only (no full roster) | [~] |
| P632-46 | Member churn: incremental `MEMBER_JOIN` without full bundle | [ ] |
| P632-47 | High churn: coalesce `GUILD_SYNC` pushes per 2s window | [ ] |
| P632-48 | Partial join: sync active channel Hypercore first | [ ] |
| P632-49 | Module README swarm/replication audit (org-wide) | [ ] |
| P632-50 | Phase 633 — CRDT reactions & presence mesh (v0.8.605) | [ ] |
| P632-47 | High churn: coalesce `GUILD_SYNC` pushes per 2s window | [x] |
| P632-48 | Partial join: sync active channel Hypercore first | [x] |
| P632-49 | Module README swarm/replication audit (org-wide) | [~] |
| P632-50 | Phase 633 — CRDT reactions & presence mesh (v0.8.605) | [x] |
**Next:** Phase 632 — Hypercore sparse pull, reaction/presence CRDT, large-guild bandwidth tuning, 3-peer mesh-cluster validation.
**Next:** Phase 633 — mesh-wide reaction CRDT, presence vector gossip, Hyperbee sparse reliability, 3-peer mesh-cluster, Corestore lifecycle.
---
### Phase 633 — CRDT reactions & presence mesh (v0.8.605)
| ID | Item | Status |
|----|------|--------|
| P633-1 | Reaction CRDT: per-emoji add/remove tombstones in mesh gossip | [ ] |
| P633-2 | Merge reaction state on `GUILD_SYNC` ingest without full bundle replay | [ ] |
| P633-3 | Presence vector clock gossip on guild mesh (not only bundle bump) | [ ] |
| P633-4 | Joiner reconciles presence vector vs contacts mesh | [ ] |
| P633-5 | Fix Hyperbee `DECODING_ERROR` (groups) for messageId→offset index | [ ] |
| P633-6 | Sparse pull: replicate ≥2 blocks when host has multi-append tail | [ ] |
| P633-7 | `attachReplication` on all guild mesh connections (owner + joiner) | [ ] |
| P633-8 | Defer non-priority channel Hypercores until `GUILD_SYNC_ACK` | [ ] |
| P633-9 | Owner-offline mod `GUILD_SYNC` push smoke (2-peer) | [ ] |
| P633-10 | Agentctl 3-peer `mesh-cluster` guild sync scenario | [ ] |
| P633-11 | Thread + forum slices in priority `GUILD_SYNC` bundles | [ ] |
| P633-12 | Pin + slowmode + overwrite snapshots in delta bundles | [ ] |
| P633-13 | Attachment metadata burst; Hyperdrive pull scheduler | [ ] |
| P633-14 | Voice occupancy slice (optional, low bandwidth) | [ ] |
| P633-15 | Discovery join coordinates guild + discovery mesh sync wait | [ ] |
| P633-16 | Multi-device sync health on settings mesh | [ ] |
| P633-17 | Prune Corestore dirs on guild delete tombstone (complete) | [ ] |
| P633-18 | Hypercore compaction after channel archive | [ ] |
| P633-19 | Mesh `connection reset` recovery in smokes | [ ] |
| P633-20 | Live mesh `test:guild-sync` stability (120s budget) | [ ] |
| P633-21 | Explore join: pass `hostPublicKey` + sync wait | [ ] |
| P633-22 | Portable invite mesh host wire regression | [ ] |
| P633-23 | Adaptive `GUILD_SYNC` burst interval from peer RTT | [ ] |
| P633-24 | Per-peer bandwidth budget enforced in push scheduler | [ ] |
| P633-25 | Partial join: backfill reactions for visible message window only | [ ] |
| P633-26 | UI: reaction sync status in message context dev panel | [ ] |
| P633-27 | UI: presence vector size + last bump in mesh settings | [ ] |
| P633-28 | Span `guild.sync.reactions` on bundle ingest | [ ] |
| P633-29 | Log filter **guild-sync-presence** in dev-log | [ ] |
| P633-30 | Smoke: reaction add/remove conflict (two peers, same emoji) | [ ] |
| P633-31 | Smoke: presence vector monotonic after edit storm | [ ] |
| P633-32 | `test:phase633-guild-sync` bundle in app package.json | [ ] |
| P633-33 | Agentctl `exportGuildSyncDiagnostics` scenario step | [ ] |
| P633-34 | Boot: pear run v0.8.605 replication smoke | [ ] |
| P633-35 | Boot: chain `test:phase632-guild-sync` in prestage | [ ] |
| P633-36 | Member churn: incremental `MEMBER_JOIN` without full bundle | [ ] |
| P633-37 | Large guild: channel-scoped delta only (no full roster export) | [ ] |
| P633-38 | `pearcord-guild` README reaction + presence RPC table | [ ] |
| P633-39 | `pearcord-guild-replicator` README Hyperbee troubleshooting | [ ] |
| P633-40 | `GUILD_SYNC.md` v0.8.605 sparse + reactions section | [ ] |
| P633-41 | `GUILD_REPLICATION.md` Phase 633 pipeline diagram | [ ] |
| P633-42 | Env `PEARCORD_GUILD_SYNC_ZLIB` in app README | [ ] |
| P633-43 | Env `PEARCORD_GUILD_SWARM_FLUSH_MS` tuning guide | [ ] |
| P633-44 | Conflict-free message edit merge under concurrent editors | [ ] |
| P633-45 | Idempotent thread parent link on duplicate `GUILD_SYNC` | [ ] |
| P633-46 | Forum post index slice in delta bundles | [ ] |
| P633-47 | Rate-limit sparse Hypercore open per join (max N/min) | [ ] |
| P633-48 | Joiner metrics: `syncMode` hypercore vs gossip duration | [ ] |
| P633-49 | Module README swarm/replication audit (org-wide) | [ ] |
| P633-50 | Phase 634 — Hyperdrive attachments & bandwidth QoS (v0.8.606) | [ ] |
**Next:** Phase 633 — reaction/presence CRDT on mesh, Hyperbee sparse reliability, 3-peer validation, Corestore prune completion.
---