116 lines
5.0 KiB
Markdown
116 lines
5.0 KiB
Markdown
# Tech choices
|
||
|
||
Recommendations for collection, streaming, visualization, and reusable libraries.
|
||
|
||
## Transport & identity
|
||
|
||
| Choice | Why |
|
||
|--------|-----|
|
||
| **HyperDHT** | Same peer identity + hole-punching model as PearDock |
|
||
| **protomux-rpc** | Multiplexed request/response + server push events |
|
||
| **compact-encoding JSON** | Fast enough for control + 1s metric batches; shared with template |
|
||
| **Ed25519 seeds** | Agent address = pubkey; admin proof via HMAC from seed |
|
||
|
||
### Streaming metrics
|
||
|
||
MVP uses **protomux-rpc `event` pushes** (`push:metrics`) with per-session subscribe + throttle.
|
||
|
||
If fleet scale demands it later:
|
||
|
||
1. Dedicated protomux channel with binary packs (Float64 arrays)
|
||
2. Hypercore / Hyperbee for durable streams a parent can replicate
|
||
3. Extract a shared **`pearrpc`** package (session middleware + hot-path helpers) from this template
|
||
|
||
For MVP, JSON pushes keep the stack simple and debuggable.
|
||
|
||
## Bare / Pear runtime (no Node builtins)
|
||
|
||
Pear’s native runtime is **Bare**. Do not rely on Node core modules existing at runtime.
|
||
|
||
| Pattern | Use |
|
||
|---------|-----|
|
||
| `package.json` `imports` map | `fs` / `os` / `path` / `crypto` / `events` / `http` / … → `bare-*` under `"bare"` condition (same idea as PearDock / pear-docs node-compat) |
|
||
| `bare-node-runtime/global` | Loaded first in Bare entrypoints (`bin/peardata-server.mjs`, `index.js`) for `process` / `Buffer` / `fetch` |
|
||
| `b4a` | Buffers in `shared/` and wire code (prefer over Node `Buffer`) |
|
||
| Direct `bare-*` | Optional for new Bare-first modules |
|
||
|
||
**Keep bare-safe:** `shared/`, `client/`, Pear `app.js` / `index.js` / `ui/`.
|
||
**Agent (`server/`):** may keep Node-style import names; under Bare they resolve via the imports map. Boot via `bin/peardata-server.mjs`.
|
||
|
||
Do **not** use `if (isBare) require('bare-fs') else require('fs')` branches for packable apps — Bare’s packer walks both sides. Prefer import maps.
|
||
|
||
## Collection
|
||
|
||
| Option | Verdict |
|
||
|--------|---------|
|
||
| **`os` + `/proc` via bare-os / bare-fs (chosen)** | Hybrid Node+Bare; low overhead |
|
||
| `node-os-utils` | Convenient but extra dep / less control |
|
||
| Native bindings (`systeminformation`, etc.) | Higher fidelity; consider Phase 2 for Windows depth |
|
||
| Shell out to `vmstat`/`iostat` | Avoid on hot path |
|
||
|
||
**Linux:** `/proc/meminfo`, `/proc/loadavg`, `/proc/net/dev`, `/proc/diskstats`
|
||
**macOS/Windows:** `os` fallbacks (CPU%, freemem, loadavg where available); I/O/net rates may be zero until platform collectors land.
|
||
|
||
Target overhead: single timer, no child processes per tick, ring buffers only.
|
||
|
||
## Storage
|
||
|
||
| Tier | Implementation |
|
||
|------|----------------|
|
||
| Hot | In-memory ring (`server/services/store.js`) |
|
||
| Warm | **HyperDB** + Corestore (+ optional Hyperswarm) — see [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) |
|
||
| Policy | Data Manager (`retention.json`, default **1y** warm age, auto-prune) — `server/services/retention.js` |
|
||
| HA / multi-writer parents | Autobase with HyperDB view (`extension: false`) |
|
||
| Avoid | SQLite as primary P2P store; copying corestore folders as “backup” |
|
||
|
||
### HyperDB engines
|
||
|
||
| Engine | Sync | Use |
|
||
|--------|------|-----|
|
||
| `HyperDB.bee(core, spec)` | P2P via Hypercore | Agent meta + warm history + linked peers |
|
||
| `HyperDB.rocks(path, spec)` | Local only | Fast local index / desktop cache (`pear-hyperdb` style) |
|
||
|
||
## Visualization (desktop)
|
||
|
||
| Option | Verdict |
|
||
|--------|---------|
|
||
| **Canvas sparklines (chosen MVP)** | No extra deps inside Pear; snappy for 1s updates |
|
||
| **uPlot** | Best next step for interactive charts (tiny, fast) |
|
||
| Chart.js / ECharts | Heavier; fine for secondary views |
|
||
| Grafana via REST | External; use `/api/v3/data` + Prometheus export |
|
||
|
||
## REST
|
||
|
||
`http` module (no Express) — under Bare this is `bare-http1` via import maps. Small attack surface for agent-style GET APIs. Default bind `127.0.0.1:18888` (`PEARDATA_REST_PORT`).
|
||
|
||
## Packaging
|
||
|
||
| Piece | Approach |
|
||
|-------|----------|
|
||
| Agent | Node 20+ or Bare (`bin/peardata-server.mjs`), systemd unit `deploy/peardata.service` |
|
||
| Desktop | Pear (`pear-electron` + `pear-bridge`) with Bare-ready imports |
|
||
| Invites | `pd1.` tokens (PearDock-style) |
|
||
| Installer | `scripts/install.sh` (rolling binaries; journal + Docker socket when detected; see [RELEASE.md](./RELEASE.md)) |
|
||
|
||
## Reusable library extraction (recommended later)
|
||
|
||
From this codebase / PearDock patterns:
|
||
|
||
| Package | Contents |
|
||
|---------|----------|
|
||
| `@pear/rpc-session` or `pearrpc` | PeerSession middleware, ACL, rate limit, audit hooks |
|
||
| `pear-metrics-wire` | Context/chart catalog types + query args |
|
||
| `pear-invite` | `pd1.` encode/decode + capability HMAC |
|
||
|
||
PearData is **AGPL-3.0-only** (same family as PearDock). Prefer composing via public APIs rather than vendoring third-party AGPL trees wholesale unless intentionally unifying under AGPL.
|
||
|
||
## Integration seams
|
||
|
||
| Ecosystem | Seam |
|
||
|-----------|------|
|
||
| PearDock | Container chart plugin reading dock RPC |
|
||
| PearVirt | VM CPU/mem contexts |
|
||
| HoneyPeer | Optional agent directory announcements |
|
||
| BareOS | Bare-compatible collector build |
|
||
| Holesail | Tunnel REST or future agent web UI |
|