pearcord/pearcord-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Phase 651 swarm replication and Phase 652 plan (v0.8.623)

Browse Source 
Add SWARM_REPLICATION playbook and mark Phase 651 core items complete.

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Raven Scott
2026-06-01 21:59:47 -04:00
co-authored by Cursor
parent c2f6778027
commit 1288990c76
3 changed files with 142 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
GUILD_REPLICATION.md
+1 -1
View File
@@ -1,6 +1,6 @@
# Guild replication architecture (Pearcord P2P)
Pearcord keeps **no central chat server**. Guild state replicates over **Hyperswarm** topics with **protomux RPC gossip** and local **HyperDB** (JSON/Rocks) persistence. **`pearcord-guild-replicator`** provides Hypercore per-channel logs + Hyperbee indexes; **`pearcord-swarm-manager`** handles bounded flush, wire retry, and offline gossip outbox replay.
Pearcord keeps **no central chat server**. Guild state replicates over **Hyperswarm** topics with **protomux RPC gossip** and local **HyperDB** (JSON/Rocks) persistence. **`pearcord-guild-replicator`** provides Hypercore per-channel logs + Hyperbee indexes; **`pearcord-swarm-manager`** handles bounded flush, wire retry, offline gossip outbox replay, and guild topic pools (Phase 651). See [SWARM_REPLICATION.md](./SWARM_REPLICATION.md).
## Mesh topics
PLATFORM_ROADMAP.md
+86 -26
View File
@@ -19899,13 +19899,13 @@ P2P-first phase: Hyperswarm guild mesh, `GUILD_SYNC` bundles, join/recovery pipe
| ID | Item | Status |
|----|------|--------|
| P651-1 | Live 120s multi-guild guild-sync stability soak | [ ] |
| P651-2 | Agentctl 8-peer mesh message fanout E2E | [ ] |
| P651-3 | E2E two-peer mesh report v0.8.623 | [ ] |
| P651-4 | Boot gate `pear run` v0.8.623 | [ ] |
| P651-5 | Multi-guild swarm topic isolation live verify | [ ] |
| P651-6 | Per-guild bandwidth fair-share cap under 3+ guilds live | [ ] |
| P651-7 | Agentctl 3-peer partition heal roundtrip live | [ ] |
| P651-1 | Live 120s multi-guild guild-sync stability soak | [~] deferred Phase 652 |
| P651-2 | Agentctl 8-peer mesh message fanout E2E | [~] deferred Phase 652 |
| P651-3 | E2E two-peer mesh report v0.8.623 | [~] deferred Phase 652 |
| P651-4 | Boot gate `pear run` v0.8.623 | [x] |
| P651-5 | Multi-guild swarm topic isolation live verify | [~] deferred Phase 652 |
| P651-6 | Per-guild bandwidth fair-share cap under 3+ guilds live | [~] deferred Phase 652 |
| P651-7 | Agentctl 3-peer partition heal roundtrip live | [~] deferred Phase 652 |
| P651-8 | Message edit LWW partition heal live 2-peer | [ ] |
| P651-9 | Reaction tombstone merge live 2-peer | [ ] |
| P651-10 | Member role LWW concurrent edits live | [ ] |
@@ -19931,26 +19931,85 @@ P2P-first phase: Hyperswarm guild mesh, `GUILD_SYNC` bundles, join/recovery pipe
| P651-30 | Live memory profile 3-guild session | [ ] |
| P651-31 | Federation hub mirror ingest live on discovery mesh | [ ] |
| P651-32 | Cross-guild announcement syndication QoS | [ ] |
| P651-33 | Swarm connection pool max per guild topic | [ ] |
| P651-34 | Guild replicator compact under federation load | [ ] |
| P651-35 | `pearcord-swarm-manager` guild topic pool + wire dedupe | [ ] |
| P651-36 | HyperDB cross-guild message index isolation live | [ ] |
| P651-37 | Sparse ACK cursor per-guild isolation verify | [ ] |
| P651-38 | Gossip outbox lane fairness under 3-guild session | [ ] |
| P651-39 | Partition heal must not flush discovery global mesh | [ ] |
| P651-40 | Offline recovery scoped discovery refresh only | [ ] |
| P651-41 | Agentctl `runFederationHubIngestJourney` smoke | [ ] |
| P651-42 | Dev-log filter **swarm-pool** | [ ] |
| P651-43 | Span `guild.mesh.topic-pool` | [ ] |
| P651-44 | `SWARM_REPLICATION.md` playbook | [ ] |
| P651-45 | Pear prestage boot smoke chain v0.8.623 | [ ] |
| P651-46 | Regression `test:phase650-live-federation-mesh` | [ ] |
| P651-47 | `test:phase651-swarm-replication` bundle | [ ] |
| P651-48 | Composer autofocus after hub ingest IPC | [ ] |
| P651-49 | Module README swarm/replication audit (org-wide) | [ ] |
| P651-50 | Phase 652 — Live mesh churn at scale (v0.8.624) | [ ] |
| P651-33 | Swarm connection pool max per guild topic | [x] `GuildTopicPool` + `PEARCORD_GUILD_TOPIC_POOL_MAX` |
| P651-34 | Guild replicator compact under federation load | [~] deferred Phase 652 |
| P651-35 | `pearcord-swarm-manager` guild topic pool + wire dedupe | [x] acquire/release on mesh peer events |
| P651-36 | HyperDB cross-guild message index isolation live | [~] deferred Phase 652 |
| P651-37 | Sparse ACK cursor per-guild isolation verify | [x] smoke + `exportMeshReplicationDiagnostics` |
| P651-38 | Gossip outbox lane fairness under 3-guild session | [x] `laneCounts()` in mesh stats |
| P651-39 | Partition heal must not flush discovery global mesh | [x] `_refreshDiscoveryAfterGuildMeshAction` |
| P651-40 | Offline recovery scoped discovery refresh only | [x] |
| P651-41 | Agentctl `runFederationHubIngestJourney` smoke | [x] |
| P651-42 | Dev-log filter **swarm-pool** | [x] |
| P651-43 | Span `guild.mesh.topic-pool` | [x] |
| P651-44 | `SWARM_REPLICATION.md` playbook | [x] |
| P651-45 | Pear prestage boot smoke chain v0.8.623 | [x] |
| P651-46 | Regression `test:phase650-live-federation-mesh` | [x] chained in bundle |
| P651-47 | `test:phase651-swarm-replication` bundle | [x] |
| P651-48 | Composer autofocus after hub ingest IPC | [~] deferred Phase 652 |
| P651-49 | Module README swarm/replication audit (org-wide) | [x] platform/swarm-manager/agentctl |
| P651-50 | Phase 652 — Live mesh churn at scale (v0.8.624) | [x] |
**Next:** Phase 651 — live multi-peer soaks, swarm topic pool, replication hardening, hub ingest live verify.
**Next:** ~~Phase 651~~ (core complete). Topic pool, scoped discovery on heal/recovery, mesh replication diagnostics, hub ingest smoke. Live 8-peer soaks in Phase 652.
---
### Phase 652 — Live mesh churn at scale (v0.8.624)
| ID | Item | Status |
|----|------|--------|
| P652-1 | Live 120s multi-guild guild-sync stability soak | [ ] |
| P652-2 | Agentctl 8-peer mesh message fanout E2E | [ ] |
| P652-3 | E2E two-peer mesh report v0.8.624 | [ ] |
| P652-4 | Boot gate `pear run` v0.8.624 | [ ] |
| P652-5 | Multi-guild swarm topic isolation live verify | [ ] |
| P652-6 | Per-guild bandwidth fair-share cap under 3+ guilds live | [ ] |
| P652-7 | Agentctl 3-peer partition heal roundtrip live | [ ] |
| P652-8 | Message edit LWW partition heal live 2-peer | [ ] |
| P652-9 | Reaction tombstone merge live 2-peer | [ ] |
| P652-10 | Member role LWW concurrent edits live | [ ] |
| P652-11 | Hypercore sparse pull resume after heal live | [ ] |
| P652-12 | Hyperbee index reconcile after heal live | [ ] |
| P652-13 | Thread archive slice LWW on heal live | [ ] |
| P652-14 | Forum tag palette merge on heal live | [ ] |
| P652-15 | Voice occupancy reconcile on heal live | [ ] |
| P652-16 | Stage roles slice merge on heal live | [ ] |
| P652-17 | Multi-device read receipt bulk sync on rejoin | [ ] |
| P652-18 | Settings mesh DHT reconcile live 3-peer | [ ] |
| P652-19 | Member list virtual scroll performance pass | [ ] |
| P652-20 | Thread bar Discord parity polish | [ ] |
| P652-21 | Agentctl 8-peer churn soak journey | [ ] |
| P652-22 | Push rotation fair-share under 8-peer live | [ ] |
| P652-23 | Sparse ACK live 3-peer defer-until-ACK | [ ] |
| P652-24 | CRDT reaction LWW concurrent toggles live | [ ] |
| P652-25 | Hyperdrive attachment QoS under heal burst | [ ] |
| P652-26 | GUILD_SYNC member churn delta live verify | [ ] |
| P652-27 | Cross-guild DM open while guild mesh syncing | [ ] |
| P652-28 | Large guild 200+ member page live 8-peer | [ ] |
| P652-29 | Discovery join sync wait under multi-guild load | [ ] |
| P652-30 | Live memory profile 3-guild session | [ ] |
| P652-31 | Topic pool cap enforcement on 9th peer join | [ ] |
| P652-32 | Guild replicator compact under federation load | [ ] |
| P652-33 | Per-guild gossip outbox shard (3+ guilds) | [ ] |
| P652-34 | Mesh replication diagnostics IPC export | [ ] |
| P652-35 | Composer autofocus after hub ingest IPC | [ ] |
| P652-36 | Span `guild.mesh.churn.scale` | [ ] |
| P652-37 | Dev-log filter **mesh-churn-scale** | [ ] |
| P652-38 | Agentctl `runPartitionHeal3PeerLiveJourney` | [ ] |
| P652-39 | `MESH_CHURN_SCALE.md` playbook | [ ] |
| P652-40 | Pear prestage boot smoke chain v0.8.624 | [ ] |
| P652-41 | Regression `test:phase651-swarm-replication` | [ ] |
| P652-42 | `test:phase652-live-mesh-churn` bundle | [ ] |
| P652-43 | HyperDB cross-guild message index isolation live | [ ] |
| P652-44 | Federation hub mirror ingest live on discovery mesh | [ ] |
| P652-45 | Cross-guild announcement syndication QoS | [ ] |
| P652-46 | UI server-rail sync dot animation polish | [ ] |
| P652-47 | Explore federation aggregate live update on heal | [ ] |
| P652-48 | Module README swarm/replication audit (org-wide) | [ ] |
| P652-49 | Sparse pull resume after topic pool cap reject | [ ] |
| P652-50 | Phase 653 — Production mesh federation live (v0.8.625) | [ ] |
**Next:** Phase 652 — live multi-peer soaks, topic pool cap live, mesh churn at scale, 3-peer partition heal live.
---
@@ -21396,6 +21455,7 @@ Absorbed into Phase 273 (P273-5–P273-8, P273-11–P273-12). See Phase 274 for
### Changelog
- **2026-06-02** — v0.8.623: Phase 651 — swarm replication hardening (`GuildTopicPool`, scoped discovery on partition heal/offline recovery, `exportMeshReplicationDiagnostics`, gossip `laneCounts`, hub ingest smoke, dev-log **swarm-pool**, `SWARM_REPLICATION.md`, `test:phase651-swarm-replication`). Phase 652 planned (50 items).
- **2026-06-02** — v0.8.622: Phase 650 — live federation mesh scale (`mergeDiscoveryListingLWW`, `refreshDiscoveryExploreScoped`, `discoveryMeshFlushListings`, hub ingest span `discovery.mesh.federation.hub-ingest`, listing TTL chip, Explore scoped refresh, async federation export with hub subs, `runFederationStabilitySoakJourney`, dev-log **federation-hub**, `test:phase650-live-federation-mesh`). Phase 651 planned (50 items).
- **2026-06-02** — v0.8.621: Phase 649 — federation/discovery depth (`refreshDiscoveryListingForGuild`, listing TTL refresh on `loadGuild` focus, `exportFederationSyncDiagnostics`, `view.federationSyncAggregate`, Explore aggregate hint, IPC `export-federation-sync-diagnostics`, dev-log **federation-sync**, span `discovery.mesh.federation`, agentctl `runFederationMeshSoakJourney`, `FEDERATION_MESH.md`, `test:phase649-federation-discovery`). Phase 650 planned (50 items).
- **2026-06-02** — v0.8.620: Phase 648 — multi-guild mesh scale (`_guildMeshBoundedByGuild`, per-guild partition heal cooldown/in-progress, `exportMultiGuildSyncDiagnostics`, `guildSyncHealthRail` + server-rail sync dots, IPC `export-multi-guild-sync-diagnostics`, dev-log **multi-guild-mesh**, span `guild.mesh.multi-scale`, agentctl `runMultiGuildMeshSoakJourney`, `MULTI_GUILD_MESH.md`, `test:phase648-multi-guild-mesh`). Phase 649 planned (50 items).
SWARM_REPLICATION.md
+55
View File
@@ -0,0 +1,55 @@
# Swarm replication hardening
Playbook for Pearcord **guild mesh replication** — topic pools, gossip outbox lanes, scoped discovery refresh after heal/recovery, and sparse ACK isolation (Phase 651, v0.8.623).
## Guild topic pool
| Concept | Detail |
|---------|--------|
| Module | `pearcord-swarm-manager` `GuildTopicPool` |
| Cap | `PEARCORD_GUILD_TOPIC_POOL_MAX` (default **8** peers per guild topic) |
| Span / log | `guild.mesh.topic-pool` (`peer-acquire`, `peer-release`) |
| Dev-log filter | **swarm-pool** |
On guild mesh `peer` join/leave, the platform tracks peers per `guildTopic(guildId)` and logs when the cap is reached (wire dedupe aid for large meshes).
## Gossip outbox lanes
`GossipOutbox.laneCounts()` reports pending entries by lane (message, presence, audit, default). Included in `getMeshStabilityStats()` and `exportMeshReplicationDiagnostics()`.
## Scoped discovery (no global mesh stall)
| Action | Discovery refresh |
|--------|-------------------|
| Partition heal | `refreshDiscoveryListingForGuild(activeGuild)` only — **not** `refreshDiscoveryListingsFromMesh()` |
| Offline recovery (⌘⇧B) | Same scoped path via `_refreshDiscoveryAfterGuildMeshAction` |
| Explore **Refresh listings** | `refreshDiscoveryExploreScoped()` |
## Diagnostics
| API | Description |
|-----|-------------|
| `platform.exportMeshReplicationDiagnostics(guildId)` | Mesh stability + topic pool + sparse ACK guild keys + gossip lanes |
| `platform.getMeshStabilityStats(guildId)` | Includes `guildTopicPool`, `gossipLaneCounts` |
## Agentctl
```bash
cd apps/pearcord
npm run test:agentctl-partition-heal-journey # asserts discoveryScoped on heal
npm run test:agentctl-federation-hub-ingest
npm run test:agentctl-federation-stability-soak
```
## Smoke bundle
```bash
npm run test:phase651-swarm-replication
```
## Related
- [GUILD_REPLICATION.md](./GUILD_REPLICATION.md)
- [PARTITION_HEAL.md](./PARTITION_HEAL.md)
- [FEDERATION_MESH.md](./FEDERATION_MESH.md)
- [MESH_STABILITY.md](./MESH_STABILITY.md)