Updates

2026-05-20 21:02:45 -04:00
parent 1f3f4b24a2
commit 14d0980b4f
734 changed files with 682 additions and 746 deletions
@@ -0,0 +1,219 @@
+# API: hyper-p2p-rpc
+
+**Protocol:** `hyper-p2p-rpc/v2` (request/reply and streaming on one Protomux channel)
+
+**Export:** `{ RPCServer, RPCClient, RPC_PROTOCOL, generateId }`
+
+Note: `STREAM_PROTOCOL` (`hyper-p2p-rpc-stream/v2`) is exported as a constant in source but streaming is implemented on the primary `RPC_PROTOCOL` channel via `chunk` / `done` reply fields.
+
+## Overview
+
+`hyper-p2p-rpc` provides typed RPC over an existing duplex socket (typically `@hyperswarm/secret-stream` after Hyperswarm connects). `RPCServer` registers named async handlers on a connection; `RPCClient` issues calls and consumes results. Handlers may return a plain value or an **async iterable**; the server then streams chunks on the same channel before a terminal `done` or `error` frame.
+
+This module does not join Hyperswarm itself—wire it after you have a socket from presence, session-bridge, or your own swarm setup.
+
+## RPCServer
+
+### Constructor
+
+```js
+const server = new RPCServer(opts)
+```
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `signingKeyPair` | `KeyPair` \| `null` | `null` | Reserved for future signed RPC; not used in v0.3.1 handlers |
+| `timeout` | `number` | `30000` | Default RPC timeout in ms for `server.call()` |
+
+Internal state: `services` (`Map`), `connections` (`Set`), `_channels` (`WeakMap` socket → `{ channel, rpcMsg }`).
+
+### `register(name, handler, schema = null)`
+
+Registers a service method.
+
+- **Parameters:**
+  - `name` — `string` method name
+  - `handler` — `async function (params, ctx) => result | AsyncIterable`
+  - `schema` — stored on the entry but **not validated** in current implementation
+- **Returns:** `void`
+- **Throws:**
+  - `TypeError: handler must be function`
+
+`ctx` object passed to handlers:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `socket` | duplex stream | Connection that received the call |
+| `peerKey` | `Buffer` \| `null` | `socket.remotePublicKey` if set |
+
+### `handleConnection(socket)`
+
+Attaches Protomux `hyper-p2p-rpc/v2` to `socket`. Idempotent per socket (second call is a no-op).
+
+- **Returns:** `void`
+- **Throws:** —
+
+Emits `connection` with `socket` when the channel opens.
+
+### `call(socket, method, params = {}, timeoutMs = this.defaultTimeout)`
+
+Server-initiated RPC to a peer on an already-handled socket (adds a temporary reply listener on the same channel).
+
+- **Returns:** `Promise<result>` — resolved with `reply.result`
+- **Throws:**
+  - `Error: Socket not connected to RPC server`
+  - `Error: RPC_TIMEOUT`
+  - `Error: <reply.error>` — remote error string (e.g. `METHOD_NOT_FOUND: <name>`)
+
+### `close()`
+
+Destroys all tracked sockets, clears services, emits `close`.
+
+- **Returns:** `void`
+- **Throws:** —
+
+### `getStats()`
+
+- **Returns:** `{ ops: number, errors: number }` — shallow copy (counters default `0`)
+- **Throws:** —
+
+## RPCClient
+
+### Constructor
+
+```js
+const client = new RPCClient(socket, opts)
+```
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `timeout` | `number` | `30000` | Default timeout for `call()` and `callStream()` |
+
+Requires a duplex `socket` compatible with `Protomux.from(socket)`. Opens `hyper-p2p-rpc/v2` immediately in `_setupControlChannel`.
+
+### `call(method, params = {}, timeoutMs = this.defaultTimeout)`
+
+- **Returns:** `Promise<result>` — `reply.result` from server
+- **Throws:**
+  - `Error: RPC_TIMEOUT`
+  - `Error: <reply.error>` — includes `METHOD_NOT_FOUND: <method>` and handler exception messages
+  - `Error: Connection closed` — if socket closes while pending
+
+### `callStream(method, params = {}, timeoutMs = this.defaultTimeout)`
+
+Invokes a handler that returns an async iterable; resolves to an async iterable of chunks.
+
+- **Returns:** `Promise<AsyncIterable>` — object with `[Symbol.asyncIterator]`
+- **Throws:**
+  - `Error: STREAM_TIMEOUT` — no `{ stream: true }` ack before timeout
+  - `Error: <reply.error>` — server or stream failure
+  - `Error: stream read timeout` — no chunk/done within `timeoutMs` while reading
+  - `Error: Connection closed`
+
+Iterator behavior:
+
+- Yields each `reply.chunk` value
+- Stops when `reply.done === true`
+- Throws `new Error(data.error)` if a stream frame carries `error`
+
+### `close()`
+
+Calls `socket.destroy()`.
+
+- **Returns:** `void`
+- **Throws:** —
+
+## Wire reply shapes (client view)
+
+| Frame | Fields | Meaning |
+|-------|--------|---------|
+| Success | `{ id, result }` | Unary RPC complete |
+| Error | `{ id, error: string }` | Failed RPC or stream |
+| Stream ack | `{ id, stream: true }` | Server will send chunks |
+| Chunk | `{ id, chunk: any }` | One streamed value |
+| Done | `{ id, done: true }` | Stream finished |
+
+Request frame: `{ id, method, params }` where `id` is 32 hex chars from `generateId()` (`crypto.randomBytes(16)`).
+
+## Events
+
+### RPCServer
+
+| Event | Payload | When |
+|-------|---------|------|
+| `connection` | `socket` | RPC Protomux channel `onopen` |
+| `close` | — | `close()` |
+
+### RPCClient
+
+| Event | Payload | When |
+|-------|---------|------|
+| `close` | — | Socket `close`; all pending calls rejected |
+
+## getStats()
+
+| Field | Type | Meaning |
+|-------|------|---------|
+| `ops` | `number` | Reserved counter (default `0`) |
+| `errors` | `number` | Reserved counter (default `0`) |
+
+## Errors
+
+Stable message substrings: see [`../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
+
+| Message | Where |
+|---------|--------|
+| `handler must be function` | `register()` — `TypeError` |
+| `Socket not connected to RPC server` | `RPCServer.call()` |
+| `RPC_TIMEOUT` | `RPCServer.call()`, `RPCClient.call()` |
+| `STREAM_TIMEOUT` | `RPCClient.callStream()` initial ack |
+| `stream read timeout` | `RPCClient.callStream()` iterator `next()` |
+| `Connection closed` | `RPCClient` on socket close |
+| `METHOD_NOT_FOUND: <method>` | Sent in `reply.error`, surfaced as `Error` on client |
+
+Handler exceptions are sent as `{ id, error: err.message || String(err) }` (not rethrown on server).
+
+## P2P
+
+Typical integration:
+
+1. Establish a Hyperswarm (or paired `SecretStream`) connection between two peers.
+2. `server.handleConnection(socketA)` and `new RPCClient(socketB)` on the paired ends.
+3. `client.call('method', params)` or `client.callStream('method', params)`.
+
+`RPC_PROTOCOL` must match on both sides (`core-infrastructure/hyper-p2p-rpc/v2`). Encoding is `compact-encoding` `c.json` for all RPC frames.
+
+Unary server flow: handler return value → `{ id, result }`.
+
+Streaming server flow: async iterable from handler → `{ id, stream: true }`, then `{ id, chunk }` per yield, then `{ id, done: true }` or `{ id, error }`.
+
+Pairing helper for tests: `lib/pair.js` (`pairSecretStreams`, `waitSecretStreamsConnected`).
+
+## Testing
+
+```bash
+cd modules/core-infrastructure/hyper-p2p-rpc
+npm install && npm test
+```
+
+`test/test.js` — registration API and close. For full wire coverage, run streaming and integration tests manually:
+
+```bash
+bare test/streaming-test.js
+bare test/integration-two-node.js
+```
+
+Examples:
+
+```bash
+bare examples/basic.js
+bare examples/streaming.js
+```
+
+## Module exports
+
+```js
+const { RPCServer, RPCClient, RPC_PROTOCOL, generateId } = require('hyper-p2p-rpc')
+```
+
+`generateId()` returns a 32-character hex string (16 random bytes).
@@ -0,0 +1,152 @@
+# Architecture: hyper-p2p-rpc
+
+**Category:** Core infrastructure
+
+**Protocol:** `hyper-p2p-rpc/v2`
+
+## Layer diagram
+
+```mermaid
+flowchart TB
+  App[Application handlers] --> SRV[RPCServer]
+  App2[Application caller] --> CLI[RPCClient]
+  SRV --> MUXs[Protomux.from socket]
+  CLI --> MUXc[Protomux.from socket]
+  MUXs --> CH[Channel hyper-p2p-rpc/v2]
+  MUXc --> CH
+  CH --> JSON[compact-encoding c.json frames]
+  SOCK[SecretStream / swarm socket] --> MUXs
+  SOCK --> MUXc
+```
+
+## Sequence: unary call
+
+```mermaid
+sequenceDiagram
+  participant C as RPCClient
+  participant CH as Protomux RPC channel
+  participant S as RPCServer
+  participant H as Handler
+
+  C->>CH: { id, method, params }
+  CH->>S: onmessage(msg)
+  S->>S: services.get(method)
+  alt method missing
+    S->>CH: { id, error: "METHOD_NOT_FOUND: ..." }
+  else handler ok
+    S->>H: handler(params, ctx)
+    H-->>S: result
+    S->>CH: { id, result }
+  end
+  CH->>C: onmessage(reply)
+  C-->>C: resolve(result) or reject(Error)
+```
+
+## Sequence: streaming call
+
+```mermaid
+sequenceDiagram
+  participant C as RPCClient
+  participant CH as Protomux RPC channel
+  participant S as RPCServer
+  participant H as AsyncIterable handler
+
+  C->>CH: { id, method, params }
+  CH->>S: onmessage
+  S->>H: await handler()
+  H-->>S: AsyncIterable
+  S->>CH: { id, stream: true }
+  CH->>C: stream ack → streamReady()
+  loop for await chunk
+    S->>CH: { id, chunk }
+    CH->>C: onStreamMsg → queue / yield
+  end
+  S->>CH: { id, done: true }
+  CH->>C: iterator returns
+```
+
+Streaming uses the **same** `hyper-p2p-rpc/v2` channel; `STREAM_PROTOCOL` is not opened as a separate channel in v0.3.1.
+
+## Wire messages
+
+Encoding: Protomux message with `encoding: c.json` on protocol `hyper-p2p-rpc/v2`.
+
+### Client → server (request)
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `id` | `string` | yes | 32 hex chars; correlates replies |
+| `method` | `string` | yes | Registered service name |
+| `params` | `object` | no | Defaults to `{}` in handler |
+
+### Server → client (reply)
+
+| Pattern | Fields | When |
+|---------|--------|------|
+| Unary success | `id`, `result` | Handler returned non-iterable value |
+| Unary error | `id`, `error` | Unknown method, handler throw, or stream failure |
+| Stream start | `id`, `stream: true` | Handler returned async iterable |
+| Stream data | `id`, `chunk` | Each `yield` from iterable |
+| Stream end | `id`, `done: true` | Iterable exhausted |
+| Stream error | `id`, `error` | Iterable loop threw |
+
+| `error` value | Cause |
+|---------------|--------|
+| `METHOD_NOT_FOUND: <method>` | No `register()` entry |
+| `<handler message>` | `catch` in `_onRpcMessage` or `_streamOnRpcChannel` |
+
+Messages with non-matching `id` are ignored on the client pending map. Server `call()` adds a one-off reply listener filtered by `id`.
+
+## State model
+
+### RPCServer
+
+| Structure | Contents |
+|-----------|----------|
+| `services` | `Map<name, { handler, schema }>` |
+| `connections` | `Set<socket>` — sockets passed to `handleConnection` |
+| `_channels` | `WeakMap<socket, { channel, rpcMsg }>` — primary RPC message |
+| `_streams` | `Map` — reserved (unused in streaming path) |
+| `defaultTimeout` | ms for `call()` |
+
+### RPCClient
+
+| Structure | Contents |
+|-----------|----------|
+| `pending` | `Map<id, { resolve, reject, timer, streamReady?, onStreamMsg? }>` |
+| `socket` / `mux` / `channel` / `rpcMsg` | Single RPC control plane per client |
+| `defaultTimeout` | ms for `call()` / `callStream()` |
+
+**ID generation:** `generateId()` → `b4a.toString(crypto.randomBytes(16), 'hex')`.
+
+**Connection lifecycle:** `socket.on('close')` rejects all pending with `Connection closed` and emits `close`.
+
+## Handler contracts
+
+| Return type | Server behavior |
+|-------------|-----------------|
+| Plain value / Promise | `{ id, result }` |
+| Object with `[Symbol.asyncIterator]` | Stream ack + chunks + done |
+| Thrown error | `{ id, error: message }` |
+
+`schema` in `register()` is stored for future validation only.
+
+## Composition
+
+| Module | Role with RPC |
+|--------|----------------|
+| `hyper-p2p-presence` | Discovery and metadata; RPC rides the same swarm socket after connect |
+| `hyper-p2p-capabilities` | Authorization before exposing RPC methods |
+| `hyper-p2p-session-bridge` | SecretStream pairing and handoff into `handleConnection` |
+| `hyper-p2p-protocol-handshake` | Version/features before app-level RPC |
+
+Recommended order: Hyperswarm connect → optional handshake → **RPCServer.handleConnection** / **RPCClient** on the encrypted duplex.
+
+See [`../_shared/WAVE6_NETWORK_STACK.md`](../../_shared/WAVE6_NETWORK_STACK.md) — `protocol-handshake` pairs with `rpc` at session layer.
+
+## Design constraints
+
+- One RPC channel per socket per side; server `call()` dynamically adds a reply message type on the same channel (client uses the primary `rpcMsg` listener for all reply shapes).
+- No built-in Hyperswarm topic or `ready()` — transport-agnostic by design.
+- Timeouts are client-side (and server-side for `RPCServer.call`) via `bare-timers`; no keepalive frames.
+- Backpressure: streaming sends chunks as fast as the iterable produces them; no windowing.