docs: rewrite Holepunch stack mapping for current Pearcord mesh
Document layers, connection shape, dual-conn rules, helpers, and known gaps versus holepunchto_repos.
This commit is contained in:
Raven Scott
+101
-30
@@ -1,35 +1,106 @@
|
|||||||
# Holepunch stack mapping
|
# Holepunch stack mapping (Pearcord)
|
||||||
|
|
||||||
Repos under `/Users/raven/dev/pearcli/holepunch-repos/holepunchto_repos`.
|
Reference tree: `/Users/raven/dev/pearcli/holepunch-repos/holepunchto_repos`.
|
||||||
|
|
||||||
| Repo | Pearcord usage |
|
Pearcord is a **Pear / Bare** desktop app that uses the Holepunch P2P stack for discovery, encrypted sockets, multiplexing, storage, and (optionally) multi-writer logs. There is **no Pearcord-operated message server**.
|
||||||
|------|----------------|
|
|
||||||
| **hyperdb** | All structured data; local Rocks + P2P bee |
|
|
||||||
| **hyperschema** | Message/guild/event schemas |
|
|
||||||
| **hyperdispatch** | Guild command router codegen |
|
|
||||||
| **autobase** | Ordered guild event log |
|
|
||||||
| **corestore** | Multi-core storage root |
|
|
||||||
| **hyperswarm** | Guild + DM discovery |
|
|
||||||
| **protomux** | Stream multiplexing |
|
|
||||||
| **protomux-rpc** | Sidecar + bot RPC |
|
|
||||||
| **hypercore-crypto** | IDs, topics, signing |
|
|
||||||
| **autopass** | Invite pairing model (`pearcord-invite`) |
|
|
||||||
| **hyperdrive** | Attachments, emoji CDN (P2P) |
|
|
||||||
| **hyperbee** | Under hyperdb bee engine |
|
|
||||||
| **hyperconf** | User/guild settings blobs |
|
|
||||||
| **blind-relay** | Optional relay for NAT |
|
|
||||||
| **pear** / **pear-electron** / **pear-bridge** | Desktop runtime |
|
|
||||||
| **bare** / **bare-module** | Sidecar JS |
|
|
||||||
| **b4a** / **compact-encoding** | Wire format |
|
|
||||||
|
|
||||||
### From new_modules (optional compose)
|
Helpers live in `modules/pearcord-shared/holepunch-stack.js` and are re-exported from `pearcord-shared`.
|
||||||
|
|
||||||
| Module | Use |
|
---
|
||||||
|--------|-----|
|
|
||||||
| `hyper-p2p-presence` | Presence gossip baseline |
|
|
||||||
| `hyper-p2p-reactive-state` | UI-ready replicated state |
|
|
||||||
| `hyper-p2p-distributed-event-bus` | Cross-guild events |
|
|
||||||
| `hyper-p2p-crdt-lww-register` | Profile fields |
|
|
||||||
| `hyper-p2p-autobase-writer-lease` | Single writer lease per channel |
|
|
||||||
|
|
||||||
Pearcord v0.1 implements native modules under `pearcord/modules/`; later phases can `file:` link showcase modules from `new_modules`.
|
## Canonical layering (bottom → top)
|
||||||
|
|
||||||
|
| Layer | Package | Pearcord use |
|
||||||
|
|-------|---------|----------------|
|
||||||
|
| Buffers / codecs / keys | `b4a`, `compact-encoding`, `hypercore-crypto` | Wire frames, 32-byte topic keys, IDs |
|
||||||
|
| DHT | `hyperdht` | Bootstrap + NAT holepunch; local testnet via `hyperdht/testnet` when `PEARCORD_LOCAL_DHT=1` |
|
||||||
|
| Swarm | `hyperswarm` | Topic discovery + Noise secret-stream sockets |
|
||||||
|
| Mux | `protomux` | One muxer per connection (`Protomux.from(conn)` caches on `stream.userData`) |
|
||||||
|
| RPC | `protomux-rpc` | Guild/DM/contacts gossip events + request/response pairs |
|
||||||
|
| Cores | `hypercore`, `corestore` | Append-only logs; multi-core store root |
|
||||||
|
| KV / DB | `hyperbee`, `hyperdb` | Structured local DB (Rocks); HyperDB schema under `pearcord-db/spec` |
|
||||||
|
| Files | `hyperdrive` | Attachments / emoji / voice-note blobs |
|
||||||
|
| Multi-writer (optional) | `autobase` | Scaffold in `pearcord-sync`; production still GUILD_SYNC + gossip until cutover |
|
||||||
|
| Runtime | `pear`, `pear-electron`, `bare-*` | Desktop shell + sidecar |
|
||||||
|
|
||||||
|
Target versions from the holepunchto mirror are exposed as `HOLEPUNCH_TARGET` in `pearcord-shared`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Standard connection shape (Holepunch)
|
||||||
|
|
||||||
|
Matches [filesharing-app-example](https://github.com/holepunchto/filesharing-app-example) and docs.pears.com:
|
||||||
|
|
||||||
|
```js
|
||||||
|
const swarm = new Hyperswarm(defaultHyperswarmOpts(keyPair))
|
||||||
|
// or platform._newHyperswarm(keyPair)
|
||||||
|
|
||||||
|
bindSwarmConnection(swarm, (conn, peerInfo) => {
|
||||||
|
// 1. Protomux / protomux-rpc for control plane
|
||||||
|
const mux = Protomux.from(conn)
|
||||||
|
const rpc = new ProtomuxRPC(mux, { protocol: 'pearcord-gossip-v2', valueEncoding: c.json })
|
||||||
|
// 2. corestore / hyperdrive replicate on the SAME Noise stream when needed
|
||||||
|
// store.replicate(conn)
|
||||||
|
})
|
||||||
|
|
||||||
|
const topic = createTopicKey('pearcord:guild:<id>') // fixed 32 bytes
|
||||||
|
await swarm.join(topic, defaultJoinOpts()).flushed?.()
|
||||||
|
```
|
||||||
|
|
||||||
|
Pearcord specifics:
|
||||||
|
|
||||||
|
- **Topics** — string topics are hashed with `hypercore-crypto.hash` → 32 bytes (`createTopicKey` / `topicToBuffer`).
|
||||||
|
- **Dual connections** — Hyperswarm already swaps sockets with a public-key initiator tie-break. App layer uses the same rule via `shouldKeepNewConnection(existing, conn)` and a short **leave grace** so UI/outbox do not thrash.
|
||||||
|
- **Shared DHT** — optional process-local DHT for multi-instance local tests (`local-hyperswarm-bootstrap.js` + `platform-hyperswarm-runtime-mixin.js`).
|
||||||
|
- **Swarm count** — multiple Hyperswarm instances (guild / contacts / settings / discovery) still share identity key material and optionally the local DHT; full single-swarm merge is a future hardening step.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Guild mesh path
|
||||||
|
|
||||||
|
1. `pearcord-guild` owns a Hyperswarm, joins `guildTopic(guildId)`.
|
||||||
|
2. `connection` → `_registerMeshPeer(conn, { peerInfo })` (Holepunch `(socket, peerInfo)`).
|
||||||
|
3. `attachGossipMesh` → `Protomux.from(conn)` + `protomux-rpc` channel (`pearcord-drive/mux-wire`).
|
||||||
|
4. Optional Hyperdrive / corestore replication on the same connection for attachments.
|
||||||
|
5. Platform outbox (`GossipOutbox`) queues when no delivery; flushes on peer join with wire-ready probes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What “adhere to the stack” means in code
|
||||||
|
|
||||||
|
| Idiom | Helper / location |
|
||||||
|
|-------|-------------------|
|
||||||
|
| 32-byte topics | `createTopicKey` / `topicToBuffer` |
|
||||||
|
| Swarm defaults (`maxPeers`, keyPair) | `defaultHyperswarmOpts` / `_newHyperswarm` |
|
||||||
|
| `server: true, client: true` join | `defaultJoinOpts` / `_defaultSwarmJoinOpts` |
|
||||||
|
| Wire error sinks | `wireSwarmConnection` |
|
||||||
|
| `(conn, peerInfo)` handlers | `bindSwarmConnection` / guild mesh handler |
|
||||||
|
| Dual-connection keep rule | `shouldKeepNewConnection` |
|
||||||
|
| One Protomux per stream | `Protomux.from` + `openRpcChannel(mux, …)` |
|
||||||
|
| Diagnostics snapshot | `holepunchStackSnapshot()` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Known gaps vs full Holepunch apps
|
||||||
|
|
||||||
|
1. **HyperDB** — pearcord may still resolve hyperdb **5.x** while holepunchto is on **6.x**; schema rebuild required for a major bump.
|
||||||
|
2. **Autobase** — `pearcord-sync` probes autobase + corestore; live guild state is still gossip + GUILD_SYNC.
|
||||||
|
3. **Single swarm** — feature meshes are separate Hyperswarm instances (shared keyPair/DHT where possible).
|
||||||
|
4. **corestore.replicate on every guild conn** — attachment path replicates when needed; not every control-plane connection mirrors the filesharing-app “always replicate” pattern.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Smokes
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/pearcord
|
||||||
|
bare scripts/smoke-holepunch-stack-align.cjs
|
||||||
|
bare scripts/smoke-mesh-stability-join-delivery.cjs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||||
|
- [SWARM_REPLICATION.md](./SWARM_REPLICATION.md)
|
||||||
|
- [MESH_STABILITY.md](./MESH_STABILITY.md)
|
||||||
|
- [HYPERDB.md](./HYPERDB.md)
|
||||||
|
|||||||
Reference in New Issue
Block a user