Updates
This commit is contained in:
+13
-20
@@ -1,45 +1,38 @@
|
||||
# ── Server identity ──────────────────────────────────────────
|
||||
# ── Agent identity ───────────────────────────────────────────
|
||||
# 32-byte seed as 64 hex chars. Auto-generated into .env on first boot if missing.
|
||||
# Treat SERVER_SEED like a root password (capabilities + admin proofs).
|
||||
# SERVER_SEED=
|
||||
# SERVER_PUBLIC_KEY=
|
||||
|
||||
# ── Roles ────────────────────────────────────────────────────
|
||||
# Baseline role for unknown peers: viewer | operator | admin
|
||||
PEARDATA_DEFAULT_ROLE=viewer
|
||||
|
||||
# Comma-separated peer public keys that always get admin
|
||||
# PEARDATA_ADMIN_KEYS=
|
||||
|
||||
# DEV ONLY — every peer is admin (never enable in production)
|
||||
# PEARDATA_INSECURE_OPEN_ADMIN=1
|
||||
|
||||
# Optional allowlist (empty = all non-revoked peers accepted)
|
||||
# PEARDATA_ALLOWLIST=
|
||||
|
||||
# ── Runtime paths & limits ───────────────────────────────────
|
||||
# Replaces OS home for client identity path:
|
||||
# $PEARDATA_HOME/.config/peardata/identity.json
|
||||
# PEARDATA_HOME=
|
||||
|
||||
# Peer policy + audit.log (default ./data)
|
||||
# PEARDATA_DATA_DIR=./data
|
||||
|
||||
# Demo room ring buffer size
|
||||
# PEARDATA_MAX_MESSAGES=500
|
||||
|
||||
# Per-peer RPC requests per minute
|
||||
# PEARDATA_RATE_LIMIT_RPM=120
|
||||
|
||||
# Client ConnectionManager reconnect attempts
|
||||
# PEARDATA_MAX_RECONNECT=20
|
||||
|
||||
# ── Metrics pipeline ─────────────────────────────────────────
|
||||
# PEARDATA_SAMPLE_MS=1000
|
||||
# PEARDATA_TIER0_POINTS=3600
|
||||
# PEARDATA_TIER1_POINTS=1440
|
||||
# PEARDATA_TIER1_EVERY=60
|
||||
|
||||
# ── Netdata-style REST API ───────────────────────────────────
|
||||
# PEARDATA_REST=1
|
||||
# PEARDATA_REST_HOST=127.0.0.1
|
||||
# PEARDATA_REST_PORT=19999
|
||||
# PEARDATA_REST_CORS=*
|
||||
|
||||
# ── Logging ──────────────────────────────────────────────────
|
||||
# LOG_LEVEL=info
|
||||
# LOG_JSON=1
|
||||
|
||||
# ── Healthcheck / soak ───────────────────────────────────────
|
||||
# Remote dial key (falls back to SERVER_PUBLIC_KEY)
|
||||
# PEARDATA_HEALTH_KEY=
|
||||
# HEALTHCHECK_TIMEOUT_MS=8000
|
||||
# SOAK_DURATION_MS=60000
|
||||
|
||||
@@ -61,6 +61,11 @@ jobs:
|
||||
LICENSE \
|
||||
docs/ARCHITECTURE.md \
|
||||
docs/PROTOCOL.md \
|
||||
docs/DATA-MODEL.md \
|
||||
docs/REST-API.md \
|
||||
docs/ROADMAP.md \
|
||||
docs/TECH-CHOICES.md \
|
||||
docs/STORAGE-HYPERDB.md \
|
||||
docs/GETTING-STARTED.md \
|
||||
docs/SECURITY.md \
|
||||
docs/DESKTOP.md \
|
||||
|
||||
@@ -1,43 +1,45 @@
|
||||
# peardata
|
||||
# PearData
|
||||
|
||||
**Production-oriented boilerplate for Holepunch / HyperDHT P2P apps.**
|
||||
**Decentralized, P2P, Netdata-class real-time monitoring for the Pear / Holepunch ecosystem.**
|
||||
|
||||
Distilled from patterns used in [peardock](https://github.com/snxraven/peardock)-class apps (MIT template — not a copy of peardock’s AGPL sources):
|
||||
Each machine runs a lightweight **PearMonitor agent**. A Pear desktop client dials agents by public key and shows fleet + per-node live dashboards. Agents also expose a **Netdata-compatible REST API** (`/api/v1`, `/api/v2`, `/api/v3`) for scripts, Grafana, and Prometheus.
|
||||
|
||||
Built from the [pear-app-template](https://github.com/snxraven) HyperDHT + protomux-rpc patterns used by PearDock-class apps (MIT template — not a copy of PearDock’s AGPL sources).
|
||||
|
||||
| Plane | Stack |
|
||||
|-------|--------|
|
||||
| Transport | **HyperDHT** secret streams (Noise) |
|
||||
| RPC | **protomux-rpc** + compact-encoding JSON |
|
||||
| Identity | Ed25519 keypairs (server seed + client identity file) |
|
||||
| AuthZ | Roles (`viewer` / `operator` / `admin`) + HMAC capabilities + admin seed proof |
|
||||
| Desktop | **Pear** (`pear-electron` + `pear-bridge` + `<pear-ctrl>` titlebar) |
|
||||
| Server | Node 20+ (systemd unit included) |
|
||||
|
||||
The demo product is **PearData**: a multi-peer room with messages, presence, and invite minting. Swap `server/handlers/demo.js` + `server/services/room.js` for your domain.
|
||||
| Metrics | 1s collector → tiered buffers → push + query |
|
||||
| AuthZ | Roles (`viewer` / `operator` / `admin`) + `pd1.` invites + admin seed |
|
||||
| HTTP | Netdata-style REST on `127.0.0.1:19999` |
|
||||
| Desktop | **Pear** (`pear-electron` + `pear-bridge` + `<pear-ctrl>`) |
|
||||
|
||||
---
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
cd pear_app_template # or your clone path
|
||||
cd peardata
|
||||
npm install
|
||||
|
||||
# Terminal A — server (prints public key)
|
||||
# Terminal A — agent (prints public key + REST URL)
|
||||
npm run start:server
|
||||
|
||||
# Terminal B — mint an operator invite (optional)
|
||||
# Terminal B — REST smoke test
|
||||
curl -s http://127.0.0.1:19999/api/v3/info | jq
|
||||
curl -s 'http://127.0.0.1:19999/api/v3/data?chart=system.cpu&after=-30&points=30' | jq
|
||||
|
||||
# Terminal C — mint an operator invite (optional)
|
||||
npm run mint-invite -- operator
|
||||
|
||||
# Terminal C — Pear desktop UI
|
||||
# Terminal D — Pear desktop
|
||||
npm start
|
||||
# or: pear run -d .
|
||||
```
|
||||
|
||||
In the UI, paste the **server public key** (viewer) or a **`pd1.` invite** (elevated role).
|
||||
Paste `SERVER_SEED` from `.env` into “Admin seed” for full admin without an invite.
|
||||
|
||||
Drag the **titlebar** to move the window; resize from the edges (`pear.gui.resizable`).
|
||||
In the UI, paste the **agent public key** (viewer) or a **`pd1.` invite**.
|
||||
Paste `SERVER_SEED` for admin without an invite.
|
||||
|
||||
```bash
|
||||
npm test
|
||||
@@ -49,26 +51,20 @@ npm run healthcheck
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
pear_app_template/
|
||||
├── app.js # Desktop UI logic
|
||||
├── index.html / index.js # Pear shell (titlebar + pear-ctrl + drag)
|
||||
├── ui/styles.css # Titlebar drag regions + layout
|
||||
├── shared/ # Protocol, encodings, schema, crypto-auth
|
||||
├── server/ # HyperDHT listener + RPC middleware + demo domain
|
||||
│ ├── server.js
|
||||
│ ├── core/ # keys, acl, audit, peer-policy, registry
|
||||
│ ├── rpc/ # PeerSession + register
|
||||
│ ├── handlers/ # Domain RPCs (replace demo.js)
|
||||
│ ├── services/ # Domain state (replace room.js)
|
||||
│ └── utils/
|
||||
├── client/ # Connection, manager, identity
|
||||
├── bin/ # Server binary entry
|
||||
├── scripts/ # healthcheck, soak, mint-invite, rename, release
|
||||
├── test/ # brittle unit + integration
|
||||
peardata/
|
||||
├── app.js / index.html / ui/ # Pear fleet dashboard
|
||||
├── shared/ # Protocol, metrics catalog, schema, crypto
|
||||
├── server/ # PearMonitor agent
|
||||
│ ├── server.js # DHT + pipeline + REST boot
|
||||
│ ├── pipeline.js # collector → store → anomaly → push
|
||||
│ ├── handlers/monitor.js # Domain RPCs
|
||||
│ ├── services/ # collector, store, anomaly, jobs, …
|
||||
│ ├── rest/ # Netdata-compatible HTTP API
|
||||
│ └── core/ rpc/ utils/ # Auth, ACL, sessions (template)
|
||||
├── client/ # Multi-peer connection manager
|
||||
├── docs/ # Full documentation set
|
||||
├── deploy/ # systemd unit
|
||||
├── .github/workflows/ # GitHub CI + release
|
||||
└── .gitea/workflows/ # Gitea CI + rolling release skeleton
|
||||
├── deploy/peardata.service # systemd
|
||||
└── test/
|
||||
```
|
||||
|
||||
---
|
||||
@@ -77,57 +73,23 @@ pear_app_template/
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `npm start` / `npm run dev` | Pear desktop UI |
|
||||
| `npm run start:server` | HyperDHT server |
|
||||
| `npm start` | Pear desktop UI |
|
||||
| `npm run start:server` | PearMonitor agent (P2P + REST) |
|
||||
| `npm test` | brittle unit + integration |
|
||||
| `npm run mint-invite -- [role] [ttlMs]` | Offline `pd1.` invite |
|
||||
| `npm run mint-invite -- [role]` | Offline `pd1.` invite |
|
||||
| `npm run healthcheck` | Liveness / remote ping |
|
||||
| `npm run soak` | Load exercise (needs env keys) |
|
||||
| `npm run rename -- <slug> <Product>` | Rebrand the tree |
|
||||
| `bash scripts/release.sh` | Source tarball + checksum |
|
||||
|
||||
Full env reference: [docs/CONFIGURATION.md](./docs/CONFIGURATION.md).
|
||||
| `npm run soak` | Load exercise |
|
||||
|
||||
---
|
||||
|
||||
## Rebrand for a new app
|
||||
|
||||
```bash
|
||||
npm run rename -- my-app MyApp
|
||||
# → package name, protocol id, env prefixes, invite prefix, product strings
|
||||
```
|
||||
|
||||
Then implement your domain:
|
||||
|
||||
1. Extend `shared/protocol.js` (`MethodRoles`, `Pushes`, `Methods`)
|
||||
2. Validate args in `shared/schema.js`
|
||||
3. Add handlers under `server/handlers/`
|
||||
4. Register them in `server/rpc/register.js`
|
||||
5. Call from `client/` + UI
|
||||
6. Update `docs/PROTOCOL.md` and tests
|
||||
|
||||
See [docs/EXTENDING.md](./docs/EXTENDING.md).
|
||||
|
||||
---
|
||||
|
||||
## Auth model (secure defaults)
|
||||
## Auth model
|
||||
|
||||
| Mode | How | Role |
|
||||
|------|-----|------|
|
||||
| Viewer | Dial public key only | `viewer` (read) |
|
||||
| Capability | `pd1.` invite or raw HMAC token | grant role |
|
||||
| Viewer | Dial public key only | `viewer` (read + subscribe) |
|
||||
| Capability | `pd1.` invite or HMAC token | granted role |
|
||||
| Admin seed | HMAC proof from `SERVER_SEED` | `admin` |
|
||||
| Allowlist | `PEARDATA_ADMIN_KEYS` | admin for listed peers |
|
||||
| Dev escape | `PEARDATA_INSECURE_OPEN_ADMIN=1` | everyone admin |
|
||||
|
||||
See [docs/SECURITY.md](./docs/SECURITY.md).
|
||||
|
||||
---
|
||||
|
||||
## CI
|
||||
|
||||
- **GitHub**: `.github/workflows/ci.yml` (Node 20/22 matrix + docs presence), `release.yml` on `v*` tags
|
||||
- **Gitea**: `.gitea/workflows/ci.yml`, `release-rolling.yml` (every push to `main` → `rolling` release; needs `RELEASE_TOKEN`)
|
||||
|
||||
---
|
||||
|
||||
@@ -135,20 +97,19 @@ See [docs/SECURITY.md](./docs/SECURITY.md).
|
||||
|
||||
| Doc | Contents |
|
||||
|-----|----------|
|
||||
| [Getting started](./docs/GETTING-STARTED.md) | Install, run, connect, systemd, troubleshooting |
|
||||
| [Desktop](./docs/DESKTOP.md) | Pear shell, `pear-ctrl`, drag/resize, identity |
|
||||
| [Architecture](./docs/ARCHITECTURE.md) | Planes, boot, session pipeline, module map |
|
||||
| [Protocol](./docs/PROTOCOL.md) | Methods, pushes, errors, versioning |
|
||||
| [Security](./docs/SECURITY.md) | Threat model, secrets, hardening checklist |
|
||||
| [Configuration](./docs/CONFIGURATION.md) | Full environment + scripts reference |
|
||||
| [Testing](./docs/TESTING.md) | brittle suite, soak, manual checks |
|
||||
| [CI](./docs/CI.md) | Pipelines and required docs |
|
||||
| [Release](./docs/RELEASE.md) | Version, tag, tarball, rollback |
|
||||
| [Extending](./docs/EXTENDING.md) | Grow past the demo room |
|
||||
| [Getting started](./docs/GETTING-STARTED.md) | Install, agent, REST, desktop, systemd |
|
||||
| [Roadmap](./docs/ROADMAP.md) | MVP → advanced phases |
|
||||
| [Architecture](./docs/ARCHITECTURE.md) | PearDock mapping, planes, modules |
|
||||
| [Protocol](./docs/PROTOCOL.md) | RPC methods & pushes |
|
||||
| [Data model](./docs/DATA-MODEL.md) | Metrics, anomalies, health |
|
||||
| [REST API](./docs/REST-API.md) | `/api/v1\|v2\|v3` |
|
||||
| [Tech choices](./docs/TECH-CHOICES.md) | Collector, charts, libraries |
|
||||
| [Security](./docs/SECURITY.md) | Threat model & hardening |
|
||||
| [Configuration](./docs/CONFIGURATION.md) | Environment reference |
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
MIT — use this as a starting point for proprietary or open apps.
|
||||
(Peardock itself is AGPL; this template does **not** copy peardock source verbatim and is intentionally MIT.)
|
||||
MIT — Pear ecosystem tooling starter.
|
||||
(Peardock itself is AGPL; this project does **not** copy peardock source verbatim.)
|
||||
|
||||
@@ -112,7 +112,7 @@ function drawChart(canvasId, values, color = '#5b8cff') {
|
||||
}
|
||||
|
||||
function renderPeers() {
|
||||
const list = manager.list?.() || []
|
||||
const list = manager.list() || []
|
||||
els.peerList.innerHTML = ''
|
||||
if (!list.length) {
|
||||
const li = document.createElement('li')
|
||||
@@ -123,11 +123,12 @@ function renderPeers() {
|
||||
}
|
||||
for (const p of list) {
|
||||
const li = document.createElement('li')
|
||||
const active = manager.activeId === p.id
|
||||
const id = p.publicKeyHex || p.id
|
||||
const active = manager.active?.publicKeyHex === id
|
||||
li.className = active ? 'active' : ''
|
||||
li.innerHTML = `<span>${escapeHtml(p.id)}</span><span class="muted">${p.connected ? 'live' : '…'}</span>`
|
||||
li.innerHTML = `<span>${escapeHtml(String(id).slice(0, 12))}…</span><span class="muted">${p.connected ? 'live' : '…'}</span>`
|
||||
li.addEventListener('click', () => {
|
||||
manager.setActive?.(p.id)
|
||||
manager.setActive(id)
|
||||
renderPeers()
|
||||
})
|
||||
els.peerList.appendChild(li)
|
||||
@@ -189,18 +190,6 @@ async function refreshMeta() {
|
||||
renderPeers()
|
||||
}
|
||||
|
||||
function parseConnectInput(raw) {
|
||||
const input = raw.trim()
|
||||
if (input.startsWith('pd1.')) {
|
||||
const inv = decodeInvite(input)
|
||||
return {
|
||||
publicKeyHex: inv.publicKeyHex,
|
||||
capability: inv.capability,
|
||||
}
|
||||
}
|
||||
return { publicKeyHex: input.toLowerCase(), capability: null }
|
||||
}
|
||||
|
||||
els.btnConnect.addEventListener('click', async () => {
|
||||
const raw = els.connectInput.value.trim()
|
||||
const adminSeed = els.adminSeed.value.trim() || null
|
||||
@@ -210,9 +199,9 @@ els.btnConnect.addEventListener('click', async () => {
|
||||
}
|
||||
els.btnConnect.disabled = true
|
||||
try {
|
||||
const { publicKeyHex, capability } = parseConnectInput(raw)
|
||||
log(`Dialing ${publicKeyHex.slice(0, 16)}…`)
|
||||
await manager.connect(publicKeyHex, { capability, adminSeed })
|
||||
log(`Dialing…`)
|
||||
const conn = await manager.connect(raw, { adminSeed })
|
||||
log(`Connected ${conn.publicKeyHex.slice(0, 16)}…`)
|
||||
setOnline(true)
|
||||
await manager.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
||||
await manager.request(Methods.subscribeAnomalies, {})
|
||||
@@ -250,7 +239,7 @@ els.btnConnect.addEventListener('click', async () => {
|
||||
})
|
||||
|
||||
els.btnDisconnect.addEventListener('click', async () => {
|
||||
await manager.disconnectAll?.()
|
||||
await manager.disconnect()
|
||||
setOnline(false)
|
||||
renderPeers()
|
||||
log('Disconnected')
|
||||
@@ -267,7 +256,7 @@ els.btnInvite.addEventListener('click', async () => {
|
||||
}
|
||||
})
|
||||
|
||||
manager.on?.('push', (ev) => {
|
||||
manager.on('push', (ev) => {
|
||||
if (ev.type === Pushes.metrics) onSamples(ev.data?.samples)
|
||||
if (ev.type === Pushes.anomaly) {
|
||||
prependAnomaly(ev.data)
|
||||
@@ -278,17 +267,15 @@ manager.on?.('push', (ev) => {
|
||||
}
|
||||
})
|
||||
|
||||
// Compatibility if manager emits per-connection
|
||||
manager.on?.('connection', (conn) => {
|
||||
conn.on?.(Pushes.metrics, (data) => onSamples(data?.samples))
|
||||
conn.on?.(Pushes.anomaly, (data) => prependAnomaly(data))
|
||||
conn.on?.(Pushes.health, (data) => {
|
||||
els.statHealth.textContent = data?.status || '—'
|
||||
})
|
||||
conn.on?.('disconnected', () => {
|
||||
setOnline(false)
|
||||
manager.on('connected', () => {
|
||||
setOnline(true)
|
||||
renderPeers()
|
||||
})
|
||||
|
||||
manager.on('disconnected', () => {
|
||||
if (!manager.active?.connected) setOnline(false)
|
||||
renderPeers()
|
||||
log('Agent disconnected')
|
||||
})
|
||||
})
|
||||
|
||||
setOnline(false)
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@ export class ConnectionManager extends EventEmitter {
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {string} input - public key, pa1 invite, or capability+key object fields
|
||||
* @param {string} input - public key, pd1 invite, or capability+key object fields
|
||||
* @param {{ adminSeed?: string, alias?: string, autoReconnect?: boolean }} [opts]
|
||||
*/
|
||||
async connect(input, opts = {}) {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
[Unit]
|
||||
Description=Pear App P2P server (HyperDHT + protomux-rpc)
|
||||
Description=PearData PearMonitor agent (HyperDHT + REST metrics)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
|
||||
+123
-151
@@ -1,193 +1,165 @@
|
||||
# Architecture
|
||||
|
||||
PearData is a **decentralized, P2P clone of the Netdata real-time monitoring experience**, built on the same HyperDHT + protomux-rpc patterns as PearDock-class apps (via the pear-app template).
|
||||
|
||||
## Design goals
|
||||
|
||||
1. **No central control plane** — peers dial a public key, not a SaaS tenant.
|
||||
2. **Cryptographic identity** — HyperDHT Noise streams authenticate both ends.
|
||||
3. **Clear AuthZ** — roles, rate limits, audit, optional allowlist / revoke.
|
||||
4. **Shared wire contract** — `shared/*` is the single source of truth for client + server.
|
||||
5. **Replaceable domain** — demo room is a thin layer over the session stack.
|
||||
6. **Pear-native desktop** — `pear-electron` shell with `<pear-ctrl>` window chrome.
|
||||
1. **No central control plane** — dial an agent by Ed25519 public key.
|
||||
2. **Instant live truth** — ~1s metric push to connected desktops.
|
||||
3. **Dual API** — P2P RPC for the Pear client; Netdata-style REST for scripts/Grafana.
|
||||
4. **Clear AuthZ** — viewer (pubkey) vs operator/admin (`pd1.` invite / seed proof).
|
||||
5. **Low agent overhead** — Node/`os` + `/proc` collectors, ring buffers, hot-path RPCs.
|
||||
6. **Ecosystem-ready** — shared wire contract in `shared/` for PearDock / PearVirt adapters later.
|
||||
|
||||
## Mapping to PearDock / template components
|
||||
|
||||
| PearDock-class concept | PearData component |
|
||||
|------------------------|--------------------|
|
||||
| HyperDHT secret stream | `server/server.js` + `client/connection.js` |
|
||||
| protomux-rpc methods | `shared/protocol.js` + `server/handlers/monitor.js` |
|
||||
| Capability invites (`pd1.`) | `shared/crypto-auth.js` |
|
||||
| Roles viewer/operator/admin | `shared/protocol.js` `MethodRoles` + `server/core/acl.js` |
|
||||
| Peer policy / revoke | `server/core/peer-policy.js` |
|
||||
| Connection manager / multi-peer | `client/manager.js` |
|
||||
| Job tray | `server/services/jobs.js` + desktop actions |
|
||||
| Domain service | **Metrics pipeline** (`collector` → `store` → `anomaly` → pushes) |
|
||||
| Optional HTTP surface | `server/rest/*` (Netdata v1/v2/v3) |
|
||||
|
||||
## System context
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
UI[Pear desktop / scripts] -->|HyperDHT Noise| SRV[Node server]
|
||||
SRV --> STATE[Room / your domain]
|
||||
UI -.->|bootstrap / punch| NET[HyperDHT network]
|
||||
SRV -.-> NET
|
||||
```
|
||||
|
||||
## Layered stack
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Presentation
|
||||
HTML[index.html + app.js + ui/styles.css]
|
||||
PEAR[index.js pear-electron + bridge]
|
||||
end
|
||||
subgraph ClientCore
|
||||
MGR[client/manager.js]
|
||||
CON[client/connection.js]
|
||||
ID[client/identity.js]
|
||||
end
|
||||
subgraph Wire
|
||||
PROT[shared/protocol.js]
|
||||
ENC[shared/encodings.js]
|
||||
AUTH[shared/crypto-auth.js]
|
||||
SCH[shared/schema.js]
|
||||
end
|
||||
subgraph ServerCore
|
||||
BOOT[server/server.js]
|
||||
SESS[server/rpc/session.js]
|
||||
ACL[server/core/acl.js]
|
||||
HAND[server/handlers/*]
|
||||
SVC[server/services/*]
|
||||
end
|
||||
PEAR --> HTML
|
||||
HTML --> MGR --> CON
|
||||
CON --> PROT
|
||||
CON --> AUTH
|
||||
SESS --> PROT
|
||||
SESS --> ACL
|
||||
SESS --> SCH
|
||||
HAND --> SVC
|
||||
BOOT --> SESS
|
||||
CON <-->|secret stream| SESS
|
||||
UI[PearData desktop] -->|HyperDHT Noise + RPC| AG[PearMonitor agent]
|
||||
SCRIPTS[curl / Grafana / Prometheus scrapers] -->|HTTP REST :19999| AG
|
||||
AG --> COL[Collector 1s]
|
||||
COL --> STORE[Tiered ring buffers]
|
||||
COL --> ANO[Anomaly engine]
|
||||
AG -.->|bootstrap / punch| DHT[HyperDHT]
|
||||
UI -.-> DHT
|
||||
```
|
||||
|
||||
## Process model
|
||||
|
||||
| Process | Entry | Responsibility |
|
||||
|---------|-------|----------------|
|
||||
| **Server** | `server/server.js` or `bin/peardata-server.mjs` | HyperDHT listen, RPC, domain state |
|
||||
| **Desktop** | `index.js` → Pear Runtime | Window + HTML UI; dials servers as a client |
|
||||
| **Scripts** | `scripts/*` | mint-invite, healthcheck, soak (use client stack) |
|
||||
| **Agent** | `server/server.js` / `bin/peardata-server.mjs` | Collect, store, P2P RPC, optional REST |
|
||||
| **Desktop** | `index.js` → Pear Runtime | Fleet UI, multi-peer dial, live charts |
|
||||
| **Scripts** | `scripts/*` | mint-invite, healthcheck, soak |
|
||||
|
||||
Server and desktop are **independent**. You can run many clients against one server, or headless scripts with no UI.
|
||||
Many desktops may dial one agent; one desktop may dial many agents.
|
||||
|
||||
## Server boot
|
||||
## Planes
|
||||
|
||||
1. `loadOrCreateKeyPair()` → persist `SERVER_SEED` / `SERVER_PUBLIC_KEY` in `.env`
|
||||
2. `initAuthKeys()` → HKDF MAC key for capabilities
|
||||
3. `loadPeerPolicy()` → roles / revocations / spent JTIs from `PEARDATA_DATA_DIR`
|
||||
4. `dht.createServer().listen(keyPair)`
|
||||
5. On connection → revoke check → `PeerSession` → `registerAllHandlers` → peer registry
|
||||
6. Banner logs public key + secure/insecure mode
|
||||
7. `graceful-goodbye` / SIGINT / SIGTERM drain peers and destroy DHT
|
||||
### 1. Control / metadata (RPC)
|
||||
|
||||
## Session middleware (every non-hot RPC)
|
||||
Handshake, node info, chart/context catalog, alert config, invites, jobs, ACL.
|
||||
|
||||
### 2. High-frequency metrics
|
||||
|
||||
- **Ingest:** collector emits sample batches every `PEARDATA_SAMPLE_MS` (default 1000).
|
||||
- **Store:** tier0 (1s) + tier1 (downsampled averages).
|
||||
- **Push:** `push:metrics` to subscribed peers (protomux-rpc events).
|
||||
- **Pull:** `queryData` / REST `/api/v3/data` for history windows.
|
||||
|
||||
### 3. Anomaly / health
|
||||
|
||||
Threshold engine evaluates each batch; transitions emit `push:anomaly` / `push:alert`; health snapshot on an interval via `push:health`.
|
||||
|
||||
### 4. Local REST (optional)
|
||||
|
||||
Netdata-compatible HTTP on `127.0.0.1:19999` by default — for local tooling without P2P. Disable with `PEARDATA_REST=0`.
|
||||
|
||||
## Layered stack
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
IN[method + args] --> RL{Rate limit}
|
||||
RL -->|deny| E1[RATE_LIMIT_EXCEEDED]
|
||||
RL -->|ok| ACL{roleAllows MethodRoles}
|
||||
ACL -->|deny| E2[PERMISSION_DENIED + audit]
|
||||
ACL -->|ok| VAL{validateMethodArgs}
|
||||
VAL -->|fail| E3[INVALID_ARGS]
|
||||
VAL -->|ok| H[Handler]
|
||||
H --> OK[Result + optional audit]
|
||||
flowchart TB
|
||||
subgraph Presentation
|
||||
HTML[index.html + app.js]
|
||||
PEAR[index.js pear-electron]
|
||||
end
|
||||
subgraph ClientCore
|
||||
MGR[client/manager.js]
|
||||
CON[client/connection.js]
|
||||
end
|
||||
subgraph Wire
|
||||
PROT[shared/protocol.js]
|
||||
MET[shared/metrics.js]
|
||||
SCH[shared/schema.js]
|
||||
AUTH[shared/crypto-auth.js]
|
||||
end
|
||||
subgraph Agent
|
||||
BOOT[server/server.js]
|
||||
PIPE[server/pipeline.js]
|
||||
COL[collector]
|
||||
STORE[store]
|
||||
ANO[anomaly]
|
||||
HAND[handlers/monitor.js]
|
||||
REST[rest/http-server.js]
|
||||
end
|
||||
PEAR --> HTML --> MGR --> CON
|
||||
CON <-->|Noise + protomux-rpc| HAND
|
||||
BOOT --> PIPE --> COL --> STORE
|
||||
COL --> ANO
|
||||
REST --> STORE
|
||||
HAND --> STORE
|
||||
```
|
||||
|
||||
**Hot path** (`session.respond(method, handler, { hot: true })` or stream method names):
|
||||
## Agent boot sequence
|
||||
|
||||
- Still rate-limited and ACL-checked
|
||||
- Skips full schema validation / success audit (for high-frequency streams)
|
||||
1. Load/create `SERVER_SEED` / public key → `initAuthKeys`
|
||||
2. Load peer policy from `PEARDATA_DATA_DIR`
|
||||
3. `startPipeline()` — collector + store + anomaly fan-out
|
||||
4. `startRestServer()` — unless disabled
|
||||
5. HyperDHT `createServer().listen(keyPair)`
|
||||
6. On connection → revoke check → `PeerSession` → register monitor handlers
|
||||
7. Banner prints pubkey + REST URL
|
||||
|
||||
## Client connection states
|
||||
## Session middleware
|
||||
|
||||
```
|
||||
idle → dialing → handshaking → ready
|
||||
↘ closed → (manager reconnect timer)
|
||||
```
|
||||
|
||||
| State | Meaning |
|
||||
|-------|---------|
|
||||
| `idle` | Constructed, not dialing |
|
||||
| `dialing` | `dht.connect(serverPk)` in flight |
|
||||
| `handshaking` | Stream open; `handshake` RPC |
|
||||
| `ready` | Authenticated; RPCs allowed |
|
||||
| `closed` | Torn down |
|
||||
|
||||
`ConnectionManager` tracks multiple peers, active selection, and reconnect (max `PEARDATA_MAX_RECONNECT`).
|
||||
Same as the template: rate limit → ACL (`MethodRoles`) → schema validate → handler.
|
||||
Hot methods (`queryData`, `subscribeMetrics`, `ping`, …) skip heavy audit.
|
||||
|
||||
## Identity planes
|
||||
|
||||
| Plane | Storage | Purpose |
|
||||
|-------|---------|---------|
|
||||
| **Server keypair** | `.env` (`SERVER_SEED`) | DHT listen address + HMAC root |
|
||||
| **Client keypair** | `~/.config/peardata/identity.json` | Stable peerId for AuthZ / revoke |
|
||||
| **Capabilities** | Issued as tokens / `pd1.` invites | Role grants with optional expiry & peer bind |
|
||||
| **Peer policy** | `data/peer-policy.json` | Registered roles, revocations, spent JTIs |
|
||||
| **Audit** | `data/audit.log` | Mutating RPC trail |
|
||||
| Agent keypair | `.env` `SERVER_SEED` | DHT address + HMAC root |
|
||||
| Client keypair | `~/.config/peardata/identity.json` | Stable peerId |
|
||||
| Capabilities | `pd1.` invites / raw tokens | Role grants |
|
||||
| Peer policy | `data/peer-policy.json` | Roles, revokes, spent JTIs |
|
||||
| Audit | `data/audit.log` | Mutating RPC trail |
|
||||
|
||||
## Parent peer (future)
|
||||
|
||||
A heavier agent may subscribe to child agents over P2P, downsample into its own store, and expose fleet REST — still no central SaaS. Design leaves room via `stream_path` REST stub and multi-peer desktop manager.
|
||||
|
||||
## Module map
|
||||
|
||||
### `shared/`
|
||||
### Keep from template
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `protocol.js` | `PROTOCOL`, roles, `MethodRoles`, `Methods`, `Pushes` |
|
||||
| `encodings.js` | compact-encoding JSON for protomux-rpc |
|
||||
| `crypto-auth.js` | MAC key, capabilities, admin proof, invites |
|
||||
| `schema.js` | Lightweight request validation |
|
||||
`server/core/*`, `server/rpc/session.js`, `client/*`, `shared/crypto-auth.js`, `shared/encodings.js`, Pear titlebar patterns.
|
||||
|
||||
### `server/`
|
||||
### PearData domain
|
||||
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `server.js` | Boot + DHT accept loop |
|
||||
| `core/keys.js` | Seed load / generate |
|
||||
| `core/auth-keys.js` | Process-wide MAC key |
|
||||
| `core/acl.js` | Role resolution + assert |
|
||||
| `core/peer-policy.js` | File-backed policy |
|
||||
| `core/peer-registry.js` | Live sessions |
|
||||
| `core/audit.js` | Audit log writer |
|
||||
| `rpc/session.js` | ProtomuxRPC + middleware |
|
||||
| `rpc/register.js` | Wire handlers per session |
|
||||
| `handlers/demo.js` | **Replace** — domain RPCs |
|
||||
| `services/room.js` | **Replace** — domain state |
|
||||
| `utils/logger.js` | Structured / pretty logs |
|
||||
| `utils/rateLimiter.js` | Per-peer RPM |
|
||||
|
||||
### `client/`
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `identity.js` | Persistent client seed |
|
||||
| `connection.js` | Single peer RPC client |
|
||||
| `manager.js` | Multi-peer + reconnect |
|
||||
| `errors.js` | Error normalization |
|
||||
| `index.js` | Public re-exports |
|
||||
|
||||
### Desktop shell
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `index.js` | Pear Runtime + Bridge |
|
||||
| `index.html` | Titlebar (`pear-ctrl`) + layout |
|
||||
| `app.js` | UI → manager |
|
||||
| `ui/styles.css` | Drag regions + theme |
|
||||
|
||||
See [DESKTOP.md](./DESKTOP.md).
|
||||
|
||||
## What to keep vs replace
|
||||
|
||||
| Keep | Replace when productizing |
|
||||
|------|---------------------------|
|
||||
| `shared/*` wire + crypto | Method names / schema for your domain |
|
||||
| `server/rpc/session.js` | Rarely — middleware is generic |
|
||||
| `server/core/*` | Peer policy storage backend if needed |
|
||||
| `client/connection.js` + `manager.js` | UI-specific multi-peer UX |
|
||||
| Titlebar / `pear-ctrl` patterns | Visual design only — keep drag + controls |
|
||||
| `server/handlers/demo.js` + `services/room.js` | **Your product** |
|
||||
| `shared/metrics.js` | Chart/context catalog |
|
||||
| `shared/data-model.js` | Typed shapes |
|
||||
| `server/services/collector.js` | System sampling |
|
||||
| `server/services/store.js` | Tiered buffers + query |
|
||||
| `server/services/anomaly.js` | Thresholds |
|
||||
| `server/services/alerts.js` | Alert CRUD helpers |
|
||||
| `server/services/subscriptions.js` | Push fan-out |
|
||||
| `server/services/jobs.js` | On-demand jobs |
|
||||
| `server/handlers/monitor.js` | RPC surface |
|
||||
| `server/rest/*` | Netdata HTTP API |
|
||||
| `server/pipeline.js` | Wire collector→store→push |
|
||||
|
||||
## Related docs
|
||||
|
||||
- [PROTOCOL.md](./PROTOCOL.md)
|
||||
- [SECURITY.md](./SECURITY.md)
|
||||
- [DESKTOP.md](./DESKTOP.md)
|
||||
- [CONFIGURATION.md](./CONFIGURATION.md)
|
||||
- [EXTENDING.md](./EXTENDING.md)
|
||||
- [PROTOCOL.md](./PROTOCOL.md) — RPC methods & pushes
|
||||
- [DATA-MODEL.md](./DATA-MODEL.md) — metrics / anomalies / health
|
||||
- [REST-API.md](./REST-API.md) — `/api/v1|v2|v3`
|
||||
- [TECH-CHOICES.md](./TECH-CHOICES.md) — collector & charts
|
||||
- [ROADMAP.md](./ROADMAP.md) — phases
|
||||
- [SECURITY.md](./SECURITY.md) — threat model
|
||||
|
||||
+1
-1
@@ -20,7 +20,7 @@
|
||||
|
||||
**lint-docs**
|
||||
|
||||
- Asserts required documentation files exist (README, architecture, protocol, getting started, security, desktop, configuration, LICENSE, etc.)
|
||||
- Asserts required documentation files exist (README, architecture, protocol, data model, REST API, roadmap, tech choices, getting started, security, desktop, configuration, LICENSE, etc.)
|
||||
|
||||
### Release job details (`release.yml`)
|
||||
|
||||
|
||||
+54
-84
@@ -1,6 +1,6 @@
|
||||
# Configuration reference
|
||||
|
||||
All knobs can be set via environment variables. The server loads `.env` through `dotenv` on boot (`server/core/keys.js`). Copy `.env.example` to get started.
|
||||
All knobs can be set via environment variables. The agent loads `.env` through `dotenv` on boot (`server/core/keys.js`). Copy `.env.example` to get started.
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
@@ -8,7 +8,7 @@ cp .env.example .env
|
||||
|
||||
---
|
||||
|
||||
## Server identity
|
||||
## Agent identity
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
@@ -16,8 +16,6 @@ cp .env.example .env
|
||||
| `SERVER_PUBLIC_KEY` | *derived* | 32-byte public key as **64 hex**. Clients dial this. Auto-synced to `.env` when seed loads. |
|
||||
| `SERVER_KEY` | — | Alias accepted for `SERVER_SEED` (legacy). Prefer `SERVER_SEED`. |
|
||||
|
||||
On first boot without a seed, the server appends both values to `.env`.
|
||||
|
||||
Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
||||
|
||||
---
|
||||
@@ -26,10 +24,10 @@ Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PEARDATA_DEFAULT_ROLE` | `viewer` | Baseline role for unknown peers: `viewer` \| `operator` \| `admin` |
|
||||
| `PEARDATA_ADMIN_KEYS` | empty | Comma-separated peer public keys always elevated to **admin** |
|
||||
| `PEARDATA_ALLOWLIST` | empty | If **non-empty**, only listed peer pubs (plus already-registered policy peers) may connect |
|
||||
| `PEARDATA_INSECURE_OPEN_ADMIN` | off | `1` / `true` / `yes` → every peer is admin. **Dev only.** |
|
||||
| `PEARDATA_DEFAULT_ROLE` | `viewer` | Baseline role: `viewer` \| `operator` \| `admin` |
|
||||
| `PEARDATA_ADMIN_KEYS` | empty | Comma-separated peer pubs always elevated to **admin** |
|
||||
| `PEARDATA_ALLOWLIST` | empty | If **non-empty**, only listed peers may connect |
|
||||
| `PEARDATA_INSECURE_OPEN_ADMIN` | off | `1` → every peer is admin. **Dev only.** |
|
||||
|
||||
---
|
||||
|
||||
@@ -38,20 +36,41 @@ Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PEARDATA_DATA_DIR` | `./data` | Peer policy JSON + `audit.log` |
|
||||
| `PEARDATA_HOME` | OS home | Root for client identity path construction (`client/identity.js`) |
|
||||
| `PEARDATA_MAX_MESSAGES` | `500` | In-memory demo room ring buffer size |
|
||||
| `PEARDATA_HOME` | OS home | Root for client identity (`~/.config/peardata/identity.json`) |
|
||||
| `PEARDATA_RATE_LIMIT_RPM` | `120` | Per-peer RPC requests per minute |
|
||||
| `PEARDATA_MAX_RECONNECT` | `20` | Client manager reconnect attempts per peer |
|
||||
|
||||
### Data directory layout
|
||||
|
||||
```
|
||||
data/
|
||||
├── peer-policy.json # registered peers, roles, revocations, spent JTIs
|
||||
└── audit.log # JSON lines for mutating RPCs + failures
|
||||
├── peer-policy.json
|
||||
└── audit.log
|
||||
```
|
||||
|
||||
Recommended permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
||||
Permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
||||
|
||||
---
|
||||
|
||||
## Metrics pipeline
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PEARDATA_SAMPLE_MS` | `1000` | Collector interval |
|
||||
| `PEARDATA_TIER0_POINTS` | `3600` | High-res ring size (~1h @ 1s) |
|
||||
| `PEARDATA_TIER1_POINTS` | `1440` | Downsampled ring size |
|
||||
| `PEARDATA_TIER1_EVERY` | `60` | Samples per tier1 average |
|
||||
|
||||
---
|
||||
|
||||
## REST API (Netdata-style)
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PEARDATA_REST` | on | Set `0` / `off` to disable HTTP API |
|
||||
| `PEARDATA_REST_HOST` | `127.0.0.1` | Bind address (`0.0.0.0` exposes LAN — firewall!) |
|
||||
| `PEARDATA_REST_PORT` | `19999` | HTTP port (Netdata classic) |
|
||||
| `PEARDATA_REST_CORS` | `*` | `Access-Control-Allow-Origin` |
|
||||
|
||||
See [REST-API.md](./REST-API.md).
|
||||
|
||||
---
|
||||
|
||||
@@ -60,87 +79,38 @@ Recommended permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `LOG_LEVEL` | `info` | `debug` \| `info` \| `warn` \| `error` |
|
||||
| `LOG_JSON` | off | `1` → structured JSON logs (good for journald) |
|
||||
| `LOG_JSON` | off | `1` → JSON lines |
|
||||
|
||||
---
|
||||
|
||||
## Healthcheck & soak
|
||||
## Healthcheck / soak / tests
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PEARDATA_HEALTH_KEY` | — | Public key for remote health dial (falls back to `SERVER_PUBLIC_KEY`) |
|
||||
| `HEALTHCHECK_TIMEOUT_MS` | `8000` | Healthcheck hard timeout (ms) |
|
||||
| `SOAK_DURATION_MS` | `60000` | Soak test run length |
|
||||
| `SOAK_INTERVAL_MS` | `500` | Delay between soak posts |
|
||||
|
||||
```bash
|
||||
# Liveness only (no dial) when no key is set
|
||||
npm run healthcheck
|
||||
|
||||
# Full dial + ping (server must be running)
|
||||
SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run healthcheck
|
||||
|
||||
# Load soak (admin seed recommended so postMessage is allowed)
|
||||
SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run soak
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `SKIP_INTEGRATION` | unset | `1` skips live HyperDHT integration test |
|
||||
|
||||
---
|
||||
|
||||
## Pear GUI (package.json)
|
||||
|
||||
Not environment variables — set under `package.json` → `pear.gui`. See [DESKTOP.md](./DESKTOP.md).
|
||||
|
||||
| Field | Template |
|
||||
|-------|----------|
|
||||
| `width` × `height` | 1100 × 780 |
|
||||
| `minWidth` × `minHeight` | 720 × 480 |
|
||||
| `resizable` / `movable` | `true` |
|
||||
| `backgroundColor` | `#0b1020` |
|
||||
|
||||
---
|
||||
|
||||
## systemd
|
||||
|
||||
`deploy/peardata.service` expects:
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/opt/peardata` | WorkingDirectory |
|
||||
| `/opt/peardata/.env` | `EnvironmentFile` |
|
||||
| `/opt/peardata/data` | Writable data dir |
|
||||
|
||||
Edit unit paths before enabling. See [GETTING-STARTED.md](./GETTING-STARTED.md#systemd).
|
||||
| `PEARDATA_HEALTH_KEY` | `SERVER_PUBLIC_KEY` | Key for remote ping |
|
||||
| `HEALTHCHECK_TIMEOUT_MS` | `8000` | Dial timeout |
|
||||
| `SOAK_DURATION_MS` | `60000` | Soak length |
|
||||
| `SOAK_INTERVAL_MS` | `500` | Soak RPC interval |
|
||||
| `SKIP_INTEGRATION` | off | `1` skips DHT integration test |
|
||||
|
||||
---
|
||||
|
||||
## npm scripts
|
||||
|
||||
| Script | Command | Purpose |
|
||||
|--------|---------|---------|
|
||||
| `npm start` / `npm run dev` | `pear run -d .` | Pear desktop UI |
|
||||
| `npm run start:server` / `server` | `node server/server.js` | P2P server |
|
||||
| `npm run start:server:bin` | `node bin/peardata-server.mjs` | Alternate server entry |
|
||||
| `npm test` | brittle suite | Unit + integration |
|
||||
| `npm run test:integration` | integration only | Live DHT test |
|
||||
| `npm run healthcheck` | dial or liveness | Process / network check |
|
||||
| `npm run soak` | long connect loop | Stability exercise |
|
||||
| `npm run mint-invite -- [role] [ttlMs]` | mint `pd1.` invite | Offline invite tooling |
|
||||
| `npm run rename -- <slug> <Product>` | rebrand tree | New product from template |
|
||||
| `bash scripts/release.sh` | source tarball | Local release artifacts |
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `npm start` / `npm run dev` | Pear desktop |
|
||||
| `npm run start:server` | Agent (P2P + REST) |
|
||||
| `npm run start:server:bin` | `bin/peardata-server.mjs` |
|
||||
| `npm test` | brittle suite |
|
||||
| `npm run mint-invite -- [role] [ttlMs]` | Offline `pd1.` invite |
|
||||
| `npm run healthcheck` | Liveness |
|
||||
| `npm run soak` | Load exercise |
|
||||
|
||||
---
|
||||
|
||||
## Security notes
|
||||
## systemd
|
||||
|
||||
- Never ship `SERVER_SEED` in client bundles or public repos.
|
||||
- Prefer invites over seed distribution.
|
||||
- Rotate seed = new public key → all clients re-dial and re-invite.
|
||||
- See [SECURITY.md](./SECURITY.md) for the full checklist.
|
||||
Unit: `deploy/peardata.service`
|
||||
WorkingDirectory: `/opt/peardata`
|
||||
EnvironmentFile: `/opt/peardata/.env`
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
# Data model
|
||||
|
||||
Canonical shapes for metrics, anomalies, alerts, health, and jobs.
|
||||
Implemented in `shared/data-model.js`, `shared/metrics.js`, and agent services.
|
||||
|
||||
## NodeInfo
|
||||
|
||||
```json
|
||||
{
|
||||
"nodeId": "a1b2c3…",
|
||||
"hostname": "homelab-1",
|
||||
"publicKeyHex": "…64 hex…",
|
||||
"platform": "linux",
|
||||
"arch": "x64",
|
||||
"release": "6.x",
|
||||
"cpus": 8,
|
||||
"totalMemMiB": 32000,
|
||||
"agentVersion": "0.1.0",
|
||||
"protocolVersion": 1,
|
||||
"startedAt": 1710000000000,
|
||||
"charts": ["system.cpu", "system.ram", "…"],
|
||||
"sampleIntervalMs": 1000,
|
||||
"sampleCount": 42
|
||||
}
|
||||
```
|
||||
|
||||
## Metric contexts & charts
|
||||
|
||||
Netdata-inspired IDs:
|
||||
|
||||
| Context | Chart id | Units | Dimensions (MVP) |
|
||||
|---------|----------|-------|------------------|
|
||||
| `system.cpu` | `system.cpu` | percentage | user, system, nice, iowait, irq, softirq, idle |
|
||||
| `system.ram` | `system.ram` | MiB | used, cached, buffers, free |
|
||||
| `mem.available` | `mem.available` | MiB | avail |
|
||||
| `system.load` | `system.load` | load | load1, load5, load15 |
|
||||
| `system.io` | `system.io` | KiB/s | reads, writes |
|
||||
| `system.net` | `system.net` | kilobits/s | received, sent |
|
||||
| `system.processes` | `system.processes` | processes | running, blocked, total |
|
||||
| `system.uptime` | `system.uptime` | seconds | uptime |
|
||||
|
||||
Chart summary objects mirror Netdata’s `/api/v1/charts` fields (`id`, `context`, `units`, `dimensions`, `update_every`, `first_entry`, `last_entry`, …).
|
||||
|
||||
## MetricSample (live push)
|
||||
|
||||
```json
|
||||
{
|
||||
"chart": "system.cpu",
|
||||
"context": "system.cpu",
|
||||
"ts": 1710000000123,
|
||||
"values": { "user": 12.4, "system": 3.1, "idle": 84.0 }
|
||||
}
|
||||
```
|
||||
|
||||
## QueryResult (history)
|
||||
|
||||
```json
|
||||
{
|
||||
"chart": "system.cpu",
|
||||
"context": "system.cpu",
|
||||
"labels": ["time", "user", "system", "nice", "iowait", "irq", "softirq", "idle"],
|
||||
"data": [[1710000000, 12.4, 3.1, 0, 0, 0, 0, 84.0]],
|
||||
"view_update_every": 1,
|
||||
"after": 1709999940,
|
||||
"before": 1710000000,
|
||||
"points": 60,
|
||||
"format": "json"
|
||||
}
|
||||
```
|
||||
|
||||
`time` is unix seconds. Nulls allowed for missing dimensions.
|
||||
|
||||
## Storage tiers
|
||||
|
||||
| Tier | Resolution | Default retention | Env |
|
||||
|------|------------|-------------------|-----|
|
||||
| 0 | 1s | 3600 points (~1h) | `PEARDATA_TIER0_POINTS` |
|
||||
| 1 | avg every N samples (default 60 → ~1m) | 1440 points (~24h) | `PEARDATA_TIER1_POINTS`, `PEARDATA_TIER1_EVERY` |
|
||||
|
||||
## AnomalyEvent
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "cpu_user_high:1710000000123",
|
||||
"chart": "system.cpu",
|
||||
"context": "system.cpu",
|
||||
"dimension": "user",
|
||||
"severity": "warning",
|
||||
"score": 0.6,
|
||||
"value": 88.2,
|
||||
"threshold": 80,
|
||||
"comparator": ">",
|
||||
"message": "CPU user time high: user=88.2 > 80",
|
||||
"ts": 1710000000123,
|
||||
"cleared": false
|
||||
}
|
||||
```
|
||||
|
||||
## AlertConfig / AlertState
|
||||
|
||||
Config fields: `id`, `chart`, `dimension`, `warn`, `crit`, `comparator`, `lookbackSec`, `enabled`, `info`.
|
||||
|
||||
State adds: `status` (`CLEAR`|`WARNING`|`CRITICAL`|`UNDEFINED`), `value`, `lastStatusChange`.
|
||||
|
||||
## HealthSnapshot
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"score": 1,
|
||||
"checks": [{ "id": "cpu_user_high", "ok": true, "detail": "…" }],
|
||||
"ts": 1710000000123
|
||||
}
|
||||
```
|
||||
|
||||
Aggregate: any CRITICAL → `critical`; else any WARNING → `degraded`; else `ok`.
|
||||
|
||||
## JobRecord
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "uuid",
|
||||
"name": "snapshot",
|
||||
"status": "done",
|
||||
"startedAt": 1710000000000,
|
||||
"finishedAt": 1710000000500,
|
||||
"result": { "ok": true }
|
||||
}
|
||||
```
|
||||
|
||||
## REST ↔ RPC parity
|
||||
|
||||
| Concept | RPC | REST |
|
||||
|---------|-----|------|
|
||||
| Charts | `listCharts` | `GET /api/v1/charts` |
|
||||
| Data | `queryData` | `GET /api/v3/data` |
|
||||
| Contexts | `listContexts` | `GET /api/v3/contexts` |
|
||||
| Nodes | `getNodeInfo` | `GET /api/v3/nodes` |
|
||||
| Alerts | `listAlerts` | `GET /api/v3/alerts` |
|
||||
| Export | `getAllMetrics` | `GET /api/v3/allmetrics` |
|
||||
| Health | `getHealth` | `GET /api/v3/health` |
|
||||
+3
-3
@@ -8,7 +8,7 @@ The client shell is a **Pear desktop application** built with `pear-electron` +
|
||||
|------|------|
|
||||
| `index.js` | Pear process entry — starts `pear-electron` Runtime + `pear-bridge` |
|
||||
| `index.html` | GUI main (`pear.gui.main`) — titlebar + panels |
|
||||
| `app.js` | UI logic (connect, room, presence, invites) |
|
||||
| `app.js` | UI logic (connect, fleet, live charts, invites) |
|
||||
| `ui/styles.css` | Layout, theme, **titlebar drag regions** |
|
||||
| `client/*` | HyperDHT connection stack used by the UI |
|
||||
|
||||
@@ -25,8 +25,8 @@ Requires the [Pear](https://docs.pears.com) CLI installed and bootstrapped (`pea
|
||||
| Field | Template default | Purpose |
|
||||
|-------|------------------|---------|
|
||||
| `main` | `index.html` | HTML entry |
|
||||
| `width` / `height` | `1100` / `780` | Initial size |
|
||||
| `minWidth` / `minHeight` | `720` / `480` | Resize floor |
|
||||
| `width` / `height` | `1280` / `860` | Initial size |
|
||||
| `minWidth` / `minHeight` | `900` / `560` | Resize floor |
|
||||
| `resizable` | `true` | Edge/corner resize |
|
||||
| `movable` | `true` | Allow OS move (with drag region) |
|
||||
| `minimizable` / `maximizable` / `closable` | `true` | Window buttons |
|
||||
|
||||
+48
-119
@@ -1,159 +1,88 @@
|
||||
# Extending the template
|
||||
# Extending PearData
|
||||
|
||||
## 1. Rebrand
|
||||
## Add a chart / context
|
||||
|
||||
```bash
|
||||
npm run rename -- notes-mesh NotesMesh
|
||||
```
|
||||
1. Define the chart in `shared/metrics.js` (`CHART_DEFS`).
|
||||
2. Emit samples from `server/services/collector.js` (or a new collector module).
|
||||
3. Store + REST/RPC pick it up automatically via `CHART_BY_ID`.
|
||||
4. Document dimensions in [DATA-MODEL.md](./DATA-MODEL.md).
|
||||
5. Optionally add a canvas panel in `index.html` + `app.js`.
|
||||
|
||||
This rewrites:
|
||||
|
||||
- package name / product name
|
||||
- protocol id (`notes-mesh/rpc`)
|
||||
- env prefix (`NOTES_MESH_`)
|
||||
- invite prefix (derived, e.g. `no1.`)
|
||||
- binary + systemd unit filenames
|
||||
|
||||
Then:
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm test
|
||||
git diff # review
|
||||
```
|
||||
|
||||
## 1b. Titlebar / window chrome
|
||||
|
||||
Keep these when restyling:
|
||||
|
||||
| Piece | Role |
|
||||
|-------|------|
|
||||
| `<pear-ctrl>` in `#titlebar` | Close / minimize / maximize (Pear runtime custom element) |
|
||||
| `#titlebar { -webkit-app-region: drag }` | Drag the window |
|
||||
| Interactive children `no-drag` | Buttons, chips, inputs stay clickable |
|
||||
| `pear.gui.resizable: true` | Edge/corner resize |
|
||||
| `pear.gui.minWidth` / `minHeight` | Floor size while resizing |
|
||||
|
||||
Do not remove `<pear-ctrl>` unless you intentionally want a frame without in-content controls (and understand platform differences). Details: [DESKTOP.md](./DESKTOP.md).
|
||||
|
||||
## 2. Add an RPC method
|
||||
## Add an RPC method
|
||||
|
||||
### `shared/protocol.js`
|
||||
|
||||
```js
|
||||
export const MethodRoles = Object.freeze({
|
||||
// ...
|
||||
listNotes: Roles.viewer,
|
||||
createNote: Roles.operator,
|
||||
listContainers: Roles.viewer,
|
||||
restartCollector: Roles.admin,
|
||||
})
|
||||
```
|
||||
|
||||
`Methods` is derived automatically from `MethodRoles` keys.
|
||||
|
||||
### `shared/schema.js`
|
||||
|
||||
Validate `createNote` args (required fields, max lengths).
|
||||
Validate args.
|
||||
|
||||
### `server/services/notes.js`
|
||||
### `server/services/…` + `server/handlers/monitor.js`
|
||||
|
||||
Domain logic / storage (keep IO out of handlers when possible).
|
||||
Register with `session.respond(...)`. Use `{ hot: true }` for high-frequency paths.
|
||||
|
||||
### `server/handlers/notes.js`
|
||||
### Client
|
||||
|
||||
```js
|
||||
export function registerNoteHandlers(session) {
|
||||
session.respond('listNotes', async () => ({ notes: [] }))
|
||||
session.respond('createNote', async (args, s) => { /* ... */ })
|
||||
}
|
||||
await manager.request(Methods.listContainers, {})
|
||||
```
|
||||
|
||||
### `server/rpc/register.js`
|
||||
### Docs + tests
|
||||
|
||||
Call `registerNoteHandlers(session)`.
|
||||
Update [PROTOCOL.md](./PROTOCOL.md) and add a brittle test.
|
||||
|
||||
### Client / UI
|
||||
## Add a REST route
|
||||
|
||||
Edit `server/rest/routes.js` — keep Netdata path naming when emulating Agent APIs (`/api/v3/...`).
|
||||
|
||||
## Add a job
|
||||
|
||||
Register in `server/services/jobs.js` `JOB_HANDLERS`.
|
||||
|
||||
```js
|
||||
await manager.request(Methods.createNote, { title: '…' })
|
||||
retrainAnomaly: async () => { /* … */ return { ok: true } }
|
||||
```
|
||||
|
||||
### Tests
|
||||
Operators run via `runJob` or future REST function execute.
|
||||
|
||||
- Unit for pure helpers + schema
|
||||
- Integration for the happy path if AuthZ/wire matter
|
||||
## Add a collector plugin (pattern)
|
||||
|
||||
## 3. Add a push channel
|
||||
|
||||
1. Add to `Pushes` in `shared/protocol.js`
|
||||
2. Optionally map in `PushToType`
|
||||
3. `session.push(Pushes.foo, payload)` or broadcast via peer registry
|
||||
4. Listen in UI: `manager.on('push', …)` or `conn.on(Pushes.foo, …)`
|
||||
(`connection.js` auto-registers all `Pushes` values)
|
||||
|
||||
## 4. Persistence
|
||||
|
||||
Demo room is in-memory (`server/services/room.js`). Swap for:
|
||||
|
||||
| Store | Good for |
|
||||
|-------|----------|
|
||||
| **Corestore / Hypercore** | Append-only logs, P2P replication |
|
||||
| **SQLite** | Structured queries |
|
||||
| **JSON files** under `PEARDATA_DATA_DIR` | Small config (peer-policy already does this) |
|
||||
|
||||
Keep RPC handlers thin; put IO in `services/`.
|
||||
|
||||
## 5. Binary streams
|
||||
|
||||
Peardock-class apps use chunked binary RPC for uploads. Pattern:
|
||||
|
||||
- methods `binaryStreamOpen` / `Chunk` / `Close` marked `hot: true` in `session.respond`
|
||||
- skip heavy schema/audit on the hot path
|
||||
- still enforce ACL + rate limits
|
||||
|
||||
Stub hooks exist via `rateLimiter.isStreamMethod`.
|
||||
|
||||
## 6. Multi-server fleet
|
||||
|
||||
`client/manager.js` already holds many connections + active selection + reconnect:
|
||||
|
||||
```js
|
||||
await manager.connect(keyOrInviteA, { autoReconnect: true })
|
||||
await manager.connect(keyOrInviteB, { autoReconnect: true })
|
||||
manager.setActive(keyA)
|
||||
await manager.request(Methods.ping, {})
|
||||
```
|
||||
server/services/collectors/
|
||||
system.js # default
|
||||
docker.js # Phase 3
|
||||
peardock.js # bridge
|
||||
```
|
||||
|
||||
Point UI at a peer list stored in `localStorage` or a file cache for a fuller fleet UX.
|
||||
Have `pipeline.js` start each enabled collector; all emit `samples` batches into the same store.
|
||||
|
||||
## 7. Custom desktop UX
|
||||
## Parent peer (fleet aggregator)
|
||||
|
||||
| Goal | Touch |
|
||||
|------|-------|
|
||||
| New screens | `index.html` + `app.js` + `ui/styles.css` |
|
||||
| Window size | `package.json` → `pear.gui` |
|
||||
| Branding in titlebar | `.app-brand` markup/CSS |
|
||||
| Persist UI prefs | replace demo `localStorage` keys |
|
||||
1. Parent dials child agents with `ConnectionManager`.
|
||||
2. Subscribes to `push:metrics` / periodically `queryData`.
|
||||
3. Ingests into local store under namespaced chart ids (`childPk.system.cpu`) or labels.
|
||||
4. Exposes `/api/v3/nodes` with multiple entries + `/api/v3/data` across nodes.
|
||||
|
||||
Preserve drag / `pear-ctrl` behavior — [DESKTOP.md](./DESKTOP.md).
|
||||
## PearDock / PearVirt hooks
|
||||
|
||||
## 8. Desktop packaging
|
||||
- Prefer **RPC adapters** over scraping: call dock/virt methods, map to PearData contexts.
|
||||
- Keep AGPL boundaries clean — depend on public APIs / your own MIT bridges.
|
||||
|
||||
This template ships the **Pear** GUI path (`pear run`). For Electron-forge / bare-standalone multi-arch releases, copy more complex packaging scripts from a production app (forge config, make scripts) once your protocol stabilizes.
|
||||
## Desktop chrome
|
||||
|
||||
## 9. Checklist for a new product
|
||||
Keep `<pear-ctrl>`, titlebar drag regions, and `pear.gui.resizable`. See [DESKTOP.md](./DESKTOP.md).
|
||||
|
||||
- [ ] `npm run rename -- …`
|
||||
- [ ] Replace demo handlers + services
|
||||
- [ ] Update PROTOCOL.md method table
|
||||
- [ ] Update SECURITY / CONFIG if new secrets
|
||||
- [ ] Tests green (`npm test`)
|
||||
- [ ] README product description
|
||||
- [ ] CI still green
|
||||
- [ ] First release tag ([RELEASE.md](./RELEASE.md))
|
||||
## Rebrand (fork)
|
||||
|
||||
## Related
|
||||
```bash
|
||||
npm run rename -- my-monitor MyMonitor
|
||||
```
|
||||
|
||||
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||
- [PROTOCOL.md](./PROTOCOL.md)
|
||||
- [TESTING.md](./TESTING.md)
|
||||
- [DESKTOP.md](./DESKTOP.md)
|
||||
Review invite prefix / env prefix, then `npm test`.
|
||||
|
||||
+63
-146
@@ -1,188 +1,105 @@
|
||||
# Getting started
|
||||
|
||||
## Prerequisites
|
||||
## Requirements
|
||||
|
||||
| Tool | Required | Notes |
|
||||
|------|----------|--------|
|
||||
| **Node.js ≥ 20** | Yes | Server + tests |
|
||||
| **npm** | Yes | Install deps |
|
||||
| **[Pear](https://docs.pears.com) CLI** | For desktop UI | Run `pear` once to bootstrap the runtime |
|
||||
| UDP / network | For real peers | HyperDHT hole-punching |
|
||||
- Node.js **≥ 20**
|
||||
- Pear runtime (for desktop): install via Holepunch/Pear docs
|
||||
- Linux recommended for full net/disk `/proc` collectors (macOS gets CPU/RAM/load)
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
git clone <your-fork-or-template-url> my-app
|
||||
cd my-app
|
||||
cd peardata
|
||||
npm install
|
||||
cp .env.example .env # optional; server auto-writes seed on first boot
|
||||
```
|
||||
|
||||
## Run the server
|
||||
Copy env template if needed:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
## Run the agent
|
||||
|
||||
```bash
|
||||
npm run start:server
|
||||
# alias: npm run server
|
||||
```
|
||||
|
||||
On first boot the server appends to `.env`:
|
||||
Banner shows:
|
||||
|
||||
```
|
||||
SERVER_SEED=<64 hex secret>
|
||||
SERVER_PUBLIC_KEY=<64 hex public>
|
||||
- **publicKey** — dial this from the desktop (viewer)
|
||||
- **rest** — e.g. `http://127.0.0.1:19999/api/v3/info`
|
||||
- **admin** — use `SERVER_SEED` from `.env` or mint `pd1.` invites
|
||||
|
||||
Agent persists identity in `.env` on first boot.
|
||||
|
||||
### REST smoke tests
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:19999/api/v3/info
|
||||
curl -s http://127.0.0.1:19999/api/v1/charts | head
|
||||
curl -s 'http://127.0.0.1:19999/api/v3/data?chart=system.cpu&after=-60&points=60'
|
||||
curl -s 'http://127.0.0.1:19999/api/v3/allmetrics?format=prometheus' | head
|
||||
```
|
||||
|
||||
**Treat `SERVER_SEED` like a root password.** Anyone with it can mint admin proofs and capabilities.
|
||||
Disable REST: `PEARDATA_REST=0`.
|
||||
|
||||
Banner output shows the public key clients dial, for example:
|
||||
## Mint an invite
|
||||
|
||||
```
|
||||
Client → dial <SERVER_PUBLIC_KEY>
|
||||
```bash
|
||||
npm run mint-invite -- operator
|
||||
# → pd1.…
|
||||
```
|
||||
|
||||
Leave this process running while clients connect.
|
||||
Share the invite string. Recipients paste it into the desktop connect box.
|
||||
|
||||
## Run the desktop UI
|
||||
## Run the desktop
|
||||
|
||||
```bash
|
||||
npm start
|
||||
# or: npm run dev
|
||||
# or: pear run -d .
|
||||
# pear run -d .
|
||||
```
|
||||
|
||||
### Desktop window chrome
|
||||
1. Paste agent public key or `pd1.` invite
|
||||
2. Optional: paste `SERVER_SEED` for admin
|
||||
3. Connect → live overview + charts
|
||||
4. Admin: **Mint invite**
|
||||
|
||||
The Pear UI uses a custom titlebar:
|
||||
|
||||
| Piece | Behavior |
|
||||
|-------|----------|
|
||||
| `<pear-ctrl>` | Platform window controls (macOS traffic lights / Windows & Linux min·max·close) |
|
||||
| `#titlebar` | `-webkit-app-region: drag` — drag to move the window |
|
||||
| Interactive children | `no-drag` so buttons and chips stay clickable |
|
||||
| `pear.gui.resizable` | Edge/corner resize (`true` by default) |
|
||||
| `minWidth` / `minHeight` | 720 × 480 floor |
|
||||
|
||||
Full details: [DESKTOP.md](./DESKTOP.md).
|
||||
|
||||
## Connect as admin (dev)
|
||||
|
||||
1. Start the Pear UI: `npm start`
|
||||
2. Paste `SERVER_PUBLIC_KEY` into **Server / invite**
|
||||
3. Paste `SERVER_SEED` into **Admin seed**
|
||||
4. Optionally set a **Display name**
|
||||
5. Click **Connect** → role badge should show `admin`
|
||||
|
||||
The seed never goes over the wire as plaintext — the client sends an HMAC **admin proof**.
|
||||
|
||||
## Connect via invite
|
||||
## systemd (Linux)
|
||||
|
||||
```bash
|
||||
# Persistent operator invite (default)
|
||||
npm run mint-invite -- operator
|
||||
|
||||
# 7-day operator invite (ttl in ms)
|
||||
npm run mint-invite -- operator 604800000
|
||||
|
||||
# Admin invite
|
||||
npm run mint-invite -- admin
|
||||
```
|
||||
|
||||
Stdout prints a `pd1.…` string. Paste it into the UI connect field (no seed needed).
|
||||
|
||||
You can also mint from a connected admin session with **Mint invite** in the UI.
|
||||
|
||||
## Viewer-only
|
||||
|
||||
Paste only the public key. You can:
|
||||
|
||||
- `listMessages`, `getPresence`, `getServerInfo`, `setDisplayName`, `ping`
|
||||
|
||||
You cannot:
|
||||
|
||||
- `postMessage` (needs `operator+`)
|
||||
- `clearMessages`, `mintInvite`, `listPeers`, `revokePeer` (needs `admin`)
|
||||
|
||||
Unless you raise `PEARDATA_DEFAULT_ROLE` (not recommended for multi-user hosts).
|
||||
|
||||
## Quick verification
|
||||
|
||||
```bash
|
||||
npm test
|
||||
SKIP_INTEGRATION=1 npm test # unit only
|
||||
|
||||
# With server running:
|
||||
export SERVER_PUBLIC_KEY=… # from .env
|
||||
export SERVER_SEED=… # optional but enables admin dial
|
||||
npm run healthcheck
|
||||
npm run soak # optional load exercise
|
||||
```
|
||||
|
||||
## Environment knobs (summary)
|
||||
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `SERVER_SEED` / `SERVER_PUBLIC_KEY` | Server identity |
|
||||
| `PEARDATA_DEFAULT_ROLE` | Baseline role (`viewer` default) |
|
||||
| `PEARDATA_ADMIN_KEYS` | Peer pubs always admin |
|
||||
| `PEARDATA_INSECURE_OPEN_ADMIN` | Dev only — all peers admin |
|
||||
| `PEARDATA_ALLOWLIST` | If set, only listed / registered peers |
|
||||
| `PEARDATA_DATA_DIR` | Peer policy + audit log directory |
|
||||
| `PEARDATA_RATE_LIMIT_RPM` | Per-peer RPC budget |
|
||||
| `PEARDATA_MAX_MESSAGES` | Demo room history cap |
|
||||
| `PEARDATA_MAX_RECONNECT` | Client reconnect tries |
|
||||
| `LOG_LEVEL` / `LOG_JSON` | Logging |
|
||||
|
||||
Full table: [CONFIGURATION.md](./CONFIGURATION.md).
|
||||
|
||||
## Rebrand for a new product
|
||||
|
||||
```bash
|
||||
npm run rename -- my-app MyApp
|
||||
npm install
|
||||
npm test
|
||||
```
|
||||
|
||||
Rewrites package name, protocol id, env prefixes, invite prefix (`pd1.` → derived), product strings, and renames the server binary / systemd unit. Review `git diff` after.
|
||||
|
||||
## systemd
|
||||
|
||||
```bash
|
||||
# Install tree to /opt/peardata (example)
|
||||
sudo mkdir -p /opt/peardata
|
||||
sudo rsync -a --exclude node_modules --exclude .git ./ /opt/peardata/
|
||||
cd /opt/peardata && sudo npm install --omit=dev
|
||||
|
||||
sudo rsync -a ./ /opt/peardata/ --exclude node_modules
|
||||
cd /opt/peardata && sudo npm ci --omit=dev
|
||||
sudo cp deploy/peardata.service /etc/systemd/system/
|
||||
# Edit WorkingDirectory, EnvironmentFile, ReadWritePaths if paths differ
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now peardata
|
||||
sudo journalctl -u peardata -f
|
||||
journalctl -u peardata -f
|
||||
```
|
||||
|
||||
The unit sets `NoNewPrivileges`, `ProtectSystem=strict`, and writable paths for `data/` + `.env`.
|
||||
Ensure `ReadWritePaths` can write `/opt/peardata/data` and `.env`.
|
||||
|
||||
## Roles cheat sheet
|
||||
|
||||
| Goal | Action |
|
||||
|------|--------|
|
||||
| Read-only monitoring | Dial pubkey only |
|
||||
| Ack alerts / run jobs | `pd1.` operator invite |
|
||||
| Mint invites / revoke | Admin seed or admin invite |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | What to check |
|
||||
|---------|----------------|
|
||||
| `Connection timeout` | Server running? Correct 64-hex key? Firewall / UDP? |
|
||||
| `PERMISSION_DENIED` on send | Role is viewer — use invite or admin seed |
|
||||
| `Rate limit exceeded` | Raise `PEARDATA_RATE_LIMIT_RPM` or slow clients |
|
||||
| Window won’t drag | Titlebar drag CSS; don’t cover bar with full-screen `no-drag` overlay |
|
||||
| Window won’t resize | `pear.gui.resizable` must be true; try edges not just corners |
|
||||
| No `<pear-ctrl>` buttons | Running under Pear (`pear run`)? Element is runtime-provided |
|
||||
| Integration test fails in CI | Set `SKIP_INTEGRATION=1` or allow UDP |
|
||||
| Seed regenerated every boot | `.env` not writable / wrong cwd |
|
||||
| Symptom | Check |
|
||||
|---------|-------|
|
||||
| REST connection refused | Agent running? `PEARDATA_REST` not `0`? Port free? |
|
||||
| Desktop can’t dial | Firewall / DHT; same machine should work; wait for punch |
|
||||
| Empty net/disk charts | Non-Linux host — expected until platform collectors land |
|
||||
| `PERMISSION_DENIED` | Need higher role invite or admin seed |
|
||||
| Integration tests hang | `SKIP_INTEGRATION=1 npm test` on restricted networks |
|
||||
|
||||
## Next steps
|
||||
## Next reading
|
||||
|
||||
| Doc | When |
|
||||
|-----|------|
|
||||
| [DESKTOP.md](./DESKTOP.md) | Titlebar, pear-ctrl, packaging |
|
||||
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Stack & session pipeline |
|
||||
| [PROTOCOL.md](./PROTOCOL.md) | Methods, pushes, versioning |
|
||||
| [SECURITY.md](./SECURITY.md) | Production hardening |
|
||||
| [CONFIGURATION.md](./CONFIGURATION.md) | Full env reference |
|
||||
| [EXTENDING.md](./EXTENDING.md) | Replace the demo room |
|
||||
| [TESTING.md](./TESTING.md) | Tests & soak |
|
||||
| [CI.md](./CI.md) / [RELEASE.md](./RELEASE.md) | Pipelines & shipping |
|
||||
- [REST-API.md](./REST-API.md)
|
||||
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||
- [ROADMAP.md](./ROADMAP.md)
|
||||
- [CONFIGURATION.md](./CONFIGURATION.md)
|
||||
|
||||
+104
-131
@@ -1,158 +1,131 @@
|
||||
# Protocol
|
||||
|
||||
## Constants
|
||||
Wire contract for PearData P2P RPC. Source of truth: `shared/protocol.js`, `shared/schema.js`, `shared/metrics.js`.
|
||||
|
||||
## Identity
|
||||
|
||||
| Constant | Value |
|
||||
|----------|--------|
|
||||
| `PROTOCOL` | `peardata/rpc` |
|
||||
| `PROTOCOL_VERSION` | `1` |
|
||||
| `APP_NAME` | `peardata` |
|
||||
| `APP_VERSION` | `0.1.0` (keep in sync with package where useful) |
|
||||
| Encoding | compact-encoding JSON (`shared/encodings.js`) |
|
||||
| Schema | lightweight validators (`shared/schema.js`) `SCHEMA_VERSION=1` |
|
||||
|----------|-------|
|
||||
| Protocol id | `peardata/rpc` |
|
||||
| Protocol version | `1` |
|
||||
| Invite prefix | `pd1.` |
|
||||
| Default role | `viewer` |
|
||||
|
||||
Bump `PROTOCOL_VERSION` on breaking request/response shapes. Additive methods may land without a bump if clients ignore unknown methods.
|
||||
|
||||
After `npm run rename`, `PROTOCOL` becomes `<slug>/rpc` and invite prefix is regenerated.
|
||||
|
||||
## Transport
|
||||
|
||||
1. Client opens HyperDHT secret stream to server public key (Noise, mutual key auth).
|
||||
2. `ProtomuxRPC` is attached with `protocol: PROTOCOL` and shared encodings.
|
||||
3. Client calls `handshake` before other RPCs (connection helper does this automatically).
|
||||
4. Server may `push` events on named channels.
|
||||
Bump `PROTOCOL_VERSION` on breaking argument/result shapes.
|
||||
|
||||
## Roles
|
||||
|
||||
| Role | Intent |
|
||||
|------|--------|
|
||||
| `viewer` | Read-only |
|
||||
| `operator` | Mutate domain data |
|
||||
| `admin` | Invite mint, clear, revoke, config |
|
||||
| Role | Rank | Typical access |
|
||||
|------|------|----------------|
|
||||
| `viewer` | 1 | Read metrics, subscribe, list alerts |
|
||||
| `operator` | 2 | Ack/silence alerts, run jobs, set alert config |
|
||||
| `admin` | 3 | Mint invites, revoke peers, export snapshot |
|
||||
|
||||
Hierarchy: `admin > operator > viewer` (`roleAllows`).
|
||||
|
||||
Unknown methods default to **admin** required in `assertAllowed` if missing from `MethodRoles` — always register new methods.
|
||||
Auth modes at handshake: public key (viewer), capability token / `pd1.` invite, admin seed proof, allowlist.
|
||||
|
||||
## Methods
|
||||
|
||||
| Method | Min role | Request args | Response (summary) |
|
||||
|--------|----------|--------------|--------------------|
|
||||
| `handshake` | viewer | `clientName`, `clientVersion`, optional `capability`, `adminProof` | role, auth, versions, features |
|
||||
| `ping` | viewer | `{}` | `{ ok, pong, peerId }` |
|
||||
| `getServerInfo` | viewer | `{}` | app, versions, host, peer count |
|
||||
| `getAuthStatus` | viewer | `{}` | peerId, role, authMode, displayName |
|
||||
| `listMessages` | viewer | `{ limit? }` | `{ messages: [...] }` |
|
||||
| `getPresence` | viewer | `{}` | `{ peers: [...] }` |
|
||||
| `postMessage` | operator | `{ text }` (1–2000 chars) | created message |
|
||||
| `setDisplayName` | viewer | `{ name }` (1–40 chars) | updated label |
|
||||
| `clearMessages` | admin | `{}` | success + system push |
|
||||
| `mintInvite` | admin | `{ role?, ttlMs?, peerId?, alias? }` | `invite` (`pd1.…`), jti, exp |
|
||||
| `listPeers` | admin | `{}` | live + policy peers |
|
||||
| `revokePeer` | admin | `{ peerId }` (64 hex) | success; target dropped |
|
||||
### Session
|
||||
|
||||
### Handshake request
|
||||
| Method | Role | Notes |
|
||||
|--------|------|-------|
|
||||
| `handshake` | viewer | Negotiate version + elevate role |
|
||||
| `ping` | viewer | Hot |
|
||||
| `getServerInfo` | viewer | Agent metadata |
|
||||
| `getAuthStatus` | viewer | Peer role / auth mode |
|
||||
| `setDisplayName` | viewer | Label for audit / UI |
|
||||
|
||||
```json
|
||||
{
|
||||
"clientName": "peardata",
|
||||
"clientVersion": "0.1.0",
|
||||
"capability": "<optional HMAC token>",
|
||||
"adminProof": { "nonce": "<hex>", "mac": "<hex>" }
|
||||
}
|
||||
```
|
||||
### Node
|
||||
|
||||
### Handshake response
|
||||
| Method | Role | Result |
|
||||
|--------|------|--------|
|
||||
| `getNodeInfo` | viewer | Hostname, CPUs, charts, sample interval |
|
||||
| `getHealth` | viewer | `{ status, score, checks }` |
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"protocol": "peardata/rpc",
|
||||
"protocolVersion": 1,
|
||||
"schemaVersion": 1,
|
||||
"role": "operator",
|
||||
"peerId": "<64 hex>",
|
||||
"serverTime": 0,
|
||||
"auth": { "mode": "capability", "role": "operator" },
|
||||
"features": { "hmacAuth": true, "invites": true, "room": true }
|
||||
}
|
||||
```
|
||||
### Metrics discovery & query
|
||||
|
||||
### Auth modes (`auth.mode` / `session.authMode`)
|
||||
| Method | Role | Args | Result |
|
||||
|--------|------|------|--------|
|
||||
| `listContexts` | viewer | — | Context catalog |
|
||||
| `getContext` | viewer | `{ id }` | Charts in context |
|
||||
| `listCharts` | viewer | — | Netdata-ish chart map |
|
||||
| `getChart` | viewer | `{ id }` | Chart summary |
|
||||
| `queryData` | viewer | `{ chart, after, before, points, group, tier }` | Time series |
|
||||
| `getAllMetrics` | viewer | `{ format: json\|prometheus\|shell }` | Latest export |
|
||||
|
||||
| Mode | How obtained |
|
||||
|------|----------------|
|
||||
| `viewer` | Default after connect with no grant |
|
||||
| `capability` | Valid capability / invite |
|
||||
| `seed` | Valid admin proof from `SERVER_SEED` |
|
||||
| `allowlist` / registered | Elevated via policy / admin keys (implementation in ACL + policy) |
|
||||
`after` / `before`: absolute unix seconds, or relative (negative = relative to `before`/`now`), Netdata-style.
|
||||
|
||||
Exact labels depend on handshake path; UI shows `authMode` from `getAuthStatus`.
|
||||
### Live subscriptions
|
||||
|
||||
## Pushes (server → client events)
|
||||
| Method | Role | Args |
|
||||
|--------|------|------|
|
||||
| `subscribeMetrics` | viewer | `{ charts: string[]\|['*'], intervalMs }` |
|
||||
| `unsubscribeMetrics` | viewer | — |
|
||||
| `subscribeAnomalies` | viewer | — |
|
||||
| `unsubscribeAnomalies` | viewer | — |
|
||||
|
||||
| Push | Payload |
|
||||
### Anomalies & alerts
|
||||
|
||||
| Method | Role | Notes |
|
||||
|--------|------|-------|
|
||||
| `listAnomalies` | viewer | Recent events |
|
||||
| `listAlerts` / `getAlert` | viewer | State + config |
|
||||
| `setAlertConfig` | operator | Upsert threshold |
|
||||
| `ackAlert` | operator | Clear until next breach |
|
||||
| `silenceAlert` | operator | Disable temporarily |
|
||||
|
||||
### Jobs
|
||||
|
||||
| Method | Role | Known jobs |
|
||||
|--------|------|------------|
|
||||
| `listJobs` | viewer | — |
|
||||
| `runJob` | operator | `collectOnce`, `snapshot`, `gcBuffers` |
|
||||
| `cancelJob` | operator | By job id |
|
||||
|
||||
### Admin
|
||||
|
||||
| Method | Role |
|
||||
|--------|------|
|
||||
| `mintInvite` | admin |
|
||||
| `listPeers` | admin |
|
||||
| `revokePeer` | admin |
|
||||
| `exportSnapshot` | admin |
|
||||
|
||||
## Pushes (server → client)
|
||||
|
||||
| Event | Payload |
|
||||
|-------|---------|
|
||||
| `push:metrics` | `{ samples: MetricSample[] }` |
|
||||
| `push:anomaly` | `AnomalyEvent` |
|
||||
| `push:alert` | Alert transition |
|
||||
| `push:health` | `HealthSnapshot` |
|
||||
| `push:job` | `JobRecord` |
|
||||
| `push:system` | Generic notices |
|
||||
|
||||
## Errors
|
||||
|
||||
Handlers throw `Error` with `.code`:
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| `push:message` | `{ id, peerId, displayName, text, ts }` |
|
||||
| `push:presence` | `{ peers: [...] }` |
|
||||
| `push:system` | `{ type, ... }` e.g. `{ type: "cleared" }` |
|
||||
|
||||
Registered via `rpc.event` / `session.push`. Client `connection.js` binds all `Pushes` values.
|
||||
|
||||
## Invites
|
||||
|
||||
Envelope: `pd1.` + base64url(JSON):
|
||||
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"publicKeyHex": "<server>",
|
||||
"capability": "<token>",
|
||||
"role": "operator",
|
||||
"jti": "...",
|
||||
"expiresAt": null
|
||||
}
|
||||
```
|
||||
|
||||
Capability token: `base64url(payload).base64url(HMAC-SHA256)`.
|
||||
|
||||
Payload fields (canonical order for MAC): `v`, `role`, `peerId`, `exp`, `jti`, `iat`.
|
||||
|
||||
### Connection input classification
|
||||
|
||||
`classifyConnectionInput(string)` accepts:
|
||||
|
||||
| Input | Kind |
|
||||
|-------|------|
|
||||
| 64 hex chars | `publicKey` |
|
||||
| `pd1.…` | `invite` (extracts key + capability) |
|
||||
| other | error |
|
||||
|
||||
## Error codes
|
||||
|
||||
| Code | When |
|
||||
|------|------|
|
||||
| `RATE_LIMIT_EXCEEDED` | Peer over RPM budget |
|
||||
| `PERMISSION_DENIED` | Role too low for method |
|
||||
| `INVALID_ARGS` | Schema validation failed |
|
||||
| `CONNECTION_TIMEOUT` | Client dial timeout |
|
||||
| `RPC_ERROR` | Generic client-normalized failure |
|
||||
| `UNKNOWN_ERROR` | Unclassified server handler error |
|
||||
|
||||
Clients should read `error.code` when present (`client/errors.js` preserves codes).
|
||||
| `PERMISSION_DENIED` | Role too low |
|
||||
| `RATE_LIMIT_EXCEEDED` | RPM exceeded |
|
||||
| `INVALID_ARGS` | Schema failure |
|
||||
| `NOT_CONNECTED` | Client-side |
|
||||
| `CAPABILITY_*` | Invite/token issues |
|
||||
|
||||
## Versioning policy
|
||||
|
||||
1. Document every method in this file.
|
||||
2. Add `MethodRoles` entry before implementing handlers.
|
||||
3. Add `validateMethodArgs` case for mutating methods.
|
||||
4. Add brittle tests for pure helpers; integration test for critical paths.
|
||||
5. Bump `PROTOCOL_VERSION` when existing response shapes break.
|
||||
6. Bump `SCHEMA_VERSION` when validation semantics change meaningfully.
|
||||
1. Additive methods/fields: no version bump if old clients ignore unknowns.
|
||||
2. Rename/remove/change meaning: bump `PROTOCOL_VERSION`; reject or compat-negotiate in handshake.
|
||||
3. REST API versions (`v1`/`v2`/`v3`) are independent of RPC version but share the store.
|
||||
|
||||
## Related
|
||||
## Example session
|
||||
|
||||
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||
- [SECURITY.md](./SECURITY.md)
|
||||
- [EXTENDING.md](./EXTENDING.md)
|
||||
- [TESTING.md](./TESTING.md)
|
||||
```text
|
||||
client → handshake { clientName, clientVersion, capability? }
|
||||
server → { role, protocolVersion, auth }
|
||||
client → subscribeMetrics { charts: ['*'], intervalMs: 1000 }
|
||||
server → push:metrics { samples: [...] } # ~1 Hz
|
||||
client → queryData { chart: 'system.cpu', after: -300, points: 300 }
|
||||
```
|
||||
|
||||
+14
-11
@@ -2,17 +2,20 @@
|
||||
|
||||
| Doc | Audience | Contents |
|
||||
|-----|----------|----------|
|
||||
| [GETTING-STARTED.md](./GETTING-STARTED.md) | New operators | Install, run, connect, systemd, troubleshooting |
|
||||
| [DESKTOP.md](./DESKTOP.md) | UI developers | Pear shell, `pear-ctrl`, drag/resize, identity |
|
||||
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Engineers | Planes, boot, middleware, module map |
|
||||
| [PROTOCOL.md](./PROTOCOL.md) | Protocol owners | Methods, pushes, errors, versioning |
|
||||
| [SECURITY.md](./SECURITY.md) | Operators / security | Trust model, checklist, crypto, incidents |
|
||||
| [GETTING-STARTED.md](./GETTING-STARTED.md) | Operators | Install, run agent, REST, desktop, systemd |
|
||||
| [ROADMAP.md](./ROADMAP.md) | Everyone | Phased MVP → advanced Netdata-class features |
|
||||
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Engineers | Planes, PearDock mapping, module map |
|
||||
| [PROTOCOL.md](./PROTOCOL.md) | Protocol owners | RPC methods, pushes, versioning |
|
||||
| [DATA-MODEL.md](./DATA-MODEL.md) | Engineers | Metrics, anomalies, alerts, jobs |
|
||||
| [REST-API.md](./REST-API.md) | Integrators | Netdata-style `/api/v1\|v2\|v3` |
|
||||
| [TECH-CHOICES.md](./TECH-CHOICES.md) | Engineers | Collector, charts, libraries |
|
||||
| [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) | Engineers | HyperDB / Corestore / Autobase sync design |
|
||||
| [DESKTOP.md](./DESKTOP.md) | UI developers | Pear shell, titlebar, identity |
|
||||
| [SECURITY.md](./SECURITY.md) | Operators / security | Trust model, secrets, hardening |
|
||||
| [CONFIGURATION.md](./CONFIGURATION.md) | Operators | Full env vars + npm scripts |
|
||||
| [TESTING.md](./TESTING.md) | Contributors | brittle suite, soak, manual QA |
|
||||
| [TESTING.md](./TESTING.md) | Contributors | brittle, soak, manual QA |
|
||||
| [CI.md](./CI.md) | Maintainers | GitHub/Gitea pipelines |
|
||||
| [RELEASE.md](./RELEASE.md) | Maintainers | Version, tag, tarball, rollback |
|
||||
| [EXTENDING.md](./EXTENDING.md) | Product builders | Rebrand, new RPCs, persistence, fleet |
|
||||
| [RELEASE.md](./RELEASE.md) | Maintainers | Version, tag, tarball |
|
||||
| [EXTENDING.md](./EXTENDING.md) | Product builders | New charts, collectors, parents |
|
||||
|
||||
Start here if you are new: **[GETTING-STARTED.md](./GETTING-STARTED.md)**.
|
||||
|
||||
Root overview: **[../README.md](../README.md)**.
|
||||
Start here: **[GETTING-STARTED.md](./GETTING-STARTED.md)** · Product overview: **[../README.md](../README.md)**.
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
# REST API (Netdata-compatible)
|
||||
|
||||
PearMonitor agents expose an optional HTTP API modeled on **Netdata Agent** endpoints (`/api/v1`, `/api/v2`, `/api/v3`).
|
||||
|
||||
Default bind: `http://127.0.0.1:19999` (Netdata’s classic port).
|
||||
Disable: `PEARDATA_REST=0`.
|
||||
Bind all interfaces (careful): `PEARDATA_REST_HOST=0.0.0.0`.
|
||||
|
||||
This is a **compatibility layer**, not a byte-for-byte Netdata clone. Core query/metadata paths are implemented for scripts, Grafana, and Prometheus scrapers.
|
||||
|
||||
## Quick examples
|
||||
|
||||
```bash
|
||||
# Agent info
|
||||
curl -s http://127.0.0.1:19999/api/v3/info | jq
|
||||
|
||||
# Chart catalog (v1 style)
|
||||
curl -s http://127.0.0.1:19999/api/v1/charts | jq '.charts | keys'
|
||||
|
||||
# Last 60s of CPU (v3)
|
||||
curl -s 'http://127.0.0.1:19999/api/v3/data?chart=system.cpu&after=-60&points=60' | jq
|
||||
|
||||
# Contexts
|
||||
curl -s http://127.0.0.1:19999/api/v3/contexts | jq
|
||||
|
||||
# Prometheus export
|
||||
curl -s 'http://127.0.0.1:19999/api/v3/allmetrics?format=prometheus'
|
||||
|
||||
# Health
|
||||
curl -s http://127.0.0.1:19999/api/v3/health | jq
|
||||
```
|
||||
|
||||
## Endpoint matrix
|
||||
|
||||
### Info & identity
|
||||
|
||||
| Method | Path | Notes |
|
||||
|--------|------|-------|
|
||||
| GET | `/` or `/api` | Service index + P2P pubkey |
|
||||
| GET | `/api/v1/info` | Agent info |
|
||||
| GET | `/api/v2/info` | same |
|
||||
| GET | `/api/v3/info` | **preferred** |
|
||||
| GET | `/api/v3/versions` | Agent / protocol / API versions |
|
||||
| GET | `/api/v3/me` | Anonymous REST identity note |
|
||||
| GET | `/api/v3/settings` | Runtime knobs |
|
||||
| GET | `/api/v3/config` | alias of settings |
|
||||
| GET | `/health`, `/api/v1/health`, `/api/v3/health` | Aggregate health |
|
||||
|
||||
### Nodes
|
||||
|
||||
| Method | Path |
|
||||
|--------|------|
|
||||
| GET | `/api/v2/nodes` |
|
||||
| GET | `/api/v3/nodes` |
|
||||
| GET | `/api/v3/node_instances` |
|
||||
| GET | `/api/v3/stream_path` |
|
||||
|
||||
Single-agent MVP returns one node (this host). Parent/fleet aggregation is roadmap.
|
||||
|
||||
### Contexts & charts
|
||||
|
||||
| Method | Path | Notes |
|
||||
|--------|------|-------|
|
||||
| GET | `/api/v3/contexts` | Context map |
|
||||
| GET | `/api/v2/contexts` | same |
|
||||
| GET | `/api/v3/context?context=` | One context |
|
||||
| GET | `/api/v1/charts` | Full chart summary (legacy but useful) |
|
||||
| GET | `/api/v1/chart?chart=` | One chart |
|
||||
|
||||
### Data queries
|
||||
|
||||
| Method | Path | Query params |
|
||||
|--------|------|--------------|
|
||||
| GET | `/api/v3/data` | `chart` or `context`, `after`, `before`, `points`, `group`, `tier`, `format` |
|
||||
| GET | `/api/v2/data` | same |
|
||||
| GET | `/api/v1/data` | same (legacy) |
|
||||
|
||||
**Params (Netdata-style)**
|
||||
|
||||
| Param | Default | Description |
|
||||
|-------|---------|-------------|
|
||||
| `chart` / `context` | required | Chart id or context id |
|
||||
| `after` | `-60` | Absolute unix sec, or relative (negative) |
|
||||
| `before` | `0` (now) | Absolute or relative |
|
||||
| `points` | `60` | Max points returned (downsampled) |
|
||||
| `group` | `average` | `average` \| `min` \| `max` \| `sum` |
|
||||
| `tier` | `0` | `0` = 1s buffer, `1` = downsampled |
|
||||
| `format` | `json` | `json` \| `csv` \| `array` |
|
||||
|
||||
### Search & weights
|
||||
|
||||
| Method | Path | Notes |
|
||||
|--------|------|-------|
|
||||
| GET | `/api/v3/q?q=` | Full-text over chart ids/titles |
|
||||
| GET | `/api/v3/weights` | MVP: health-derived scores |
|
||||
|
||||
### Alerts
|
||||
|
||||
| Method | Path |
|
||||
|--------|------|
|
||||
| GET | `/api/v3/alerts` |
|
||||
| GET | `/api/v2/alerts` |
|
||||
| GET | `/api/v1/alarms` |
|
||||
| GET | `/api/v3/alert_transitions` |
|
||||
| GET | `/api/v3/alert_config` |
|
||||
| GET | `/api/v3/variable` |
|
||||
|
||||
### Export & badges
|
||||
|
||||
| Method | Path | Params |
|
||||
|--------|------|--------|
|
||||
| GET | `/api/v3/allmetrics` | `format=json\|prometheus\|shell` |
|
||||
| GET | `/api/v1/allmetrics` | same |
|
||||
| GET | `/api/v3/badge.svg` | `chart`, `dimensions`, `label` |
|
||||
|
||||
### Functions (stub)
|
||||
|
||||
| Method | Path | Notes |
|
||||
|--------|------|-------|
|
||||
| GET | `/api/v3/functions` | Lists job names; execution remains P2P `runJob` for auth |
|
||||
|
||||
## Auth model (REST)
|
||||
|
||||
- **Default:** localhost-only, no bearer required (like a typical Netdata agent bind).
|
||||
- **P2P remains the secure remote path** (Noise + roles).
|
||||
- If you bind `0.0.0.0`, put REST behind a firewall, reverse proxy, or Holesail tunnel — do not expose raw metrics to the internet.
|
||||
- Future: optional bearer gate (`/api/v3/bearer_protection` parity).
|
||||
|
||||
## CORS
|
||||
|
||||
`Access-Control-Allow-Origin` defaults to `*` (override with `PEARDATA_REST_CORS`).
|
||||
|
||||
## Compatibility notes
|
||||
|
||||
| Netdata | PearData MVP |
|
||||
|---------|--------------|
|
||||
| Full ML weights / metric correlations | Simplified health weights |
|
||||
| Multi-node parent streaming | Single node; parent planned |
|
||||
| Cloud POST `/api/v3/spaces/.../data` | Not implemented (agent GET style only) |
|
||||
| Every chart Netdata ships | Core system charts (expanding) |
|
||||
| Functions execute via HTTP | Listed; run via P2P jobs |
|
||||
|
||||
## Implementation
|
||||
|
||||
- Router: `server/rest/routes.js`
|
||||
- Server: `server/rest/http-server.js`
|
||||
- Exporters: `server/rest/formatters.js`
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
# PearData Roadmap
|
||||
|
||||
Phased plan from MVP agent → Netdata-class fleet observability on pure P2P.
|
||||
|
||||
## Guiding principles
|
||||
|
||||
1. **Instant value** — connect a pubkey, see live charts in seconds.
|
||||
2. **Agent efficiency** — stay in Netdata’s ballpark for CPU/RAM overhead.
|
||||
3. **PearDock patterns** — HyperDHT identity, protomux-rpc, roles, `pd1.` invites.
|
||||
4. **Dual access** — P2P desktop + Netdata-style REST (`/api/v1|v2|v3`).
|
||||
5. **Ecosystem glue** — ready for PearDock / PearVirt / HoneyPeer / BareOS later.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Foundation ✅ (this repo)
|
||||
|
||||
| Item | Status |
|
||||
|------|--------|
|
||||
| Copy pear-app-template → PearData rebrand (`pd1.`, `PEARDATA_*`, `peardata/rpc`) | Done |
|
||||
| Shared protocol + schema + metrics catalog | Done |
|
||||
| Agent collector (CPU/RAM/load/net/disk/processes) @ ~1s | Done |
|
||||
| In-memory tiered store (1s + downsample) | Done |
|
||||
| Threshold anomaly engine + health | Done |
|
||||
| P2P RPC surface (query, subscribe, alerts, jobs, ACL) | Done |
|
||||
| Netdata-compatible REST v1/v2/v3 (local :19999) | Done |
|
||||
| Pear desktop fleet overview + live canvas charts | Done |
|
||||
| Docs: architecture, protocol, data model, REST, roadmap, tech choices | Done |
|
||||
| systemd unit + CI skeletons | Done |
|
||||
|
||||
**MVP acceptance criteria**
|
||||
|
||||
- [x] Agent exposes metrics over P2P by public key
|
||||
- [x] Desktop connects to one+ agents and shows live CPU/RAM/net/disk
|
||||
- [x] REST `/api/v3/data?chart=system.cpu&after=-60` returns series
|
||||
- [x] Viewer vs admin (pubkey / seed / `pd1.` invite)
|
||||
- [x] Simple threshold anomalies pushed to clients
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Hardening & UX (next)
|
||||
|
||||
- Persistent client peer bookmarks + aliases in desktop
|
||||
- Multi-peer compare mode (overlay 2–4 nodes on one chart)
|
||||
- Reconnection / offline banners with last-known samples
|
||||
- Agent process title / `peardata-agent` binary naming polish
|
||||
- Expand collectors: per-core CPU, per-iface net, mount disk space
|
||||
- Alert silence TTL auto-reenable
|
||||
- brittle tests for collector, store query, REST routes
|
||||
- One-line install script (`curl | bash`) for Linux agents
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Storage & history (HyperDB)
|
||||
|
||||
See **[STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md)** for the full design (from Holepunch `hyperdb` / workshop / Autobase patterns).
|
||||
|
||||
- HyperDB (bee + Corestore) for metadata, alerts, peer-links, **warm** downsampled points
|
||||
- Keep memory ring for hot 1s path; do **not** tx every sample into HyperDB
|
||||
- Hyperswarm `store.replicate` for linked-node / desktop seed sync
|
||||
- Configurable retention (hours@1s memory, days@1m HyperDB, weeks@1h)
|
||||
- Historical query: memory miss → HyperDB range
|
||||
- Optional later: Autobase multi-writer parents; Rocks engine for local-only speed
|
||||
- Export snapshot job → JSON / Prometheus remote write (optional)
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Discovery & apps
|
||||
|
||||
- Docker / container auto-discovery (cgroup + Docker API)
|
||||
- Common service collectors (nginx, postgres, redis) as plugins
|
||||
- PearDock integration: container metrics from dock peers
|
||||
- PearVirt / BareOS node metric adapters
|
||||
- Holesail optional expose of REST UI per agent
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Smarter anomalies & fleet
|
||||
|
||||
- Anomaly scoring + chart highlighting in UI
|
||||
- Lightweight ML job (`runJob` retrain) — start with streaming z-score / k-means
|
||||
- Fleet-wide composite views and correlation (`/api/v3/weights` depth)
|
||||
- Parent peer aggregation (P2P “parent” without central SaaS)
|
||||
- Push notifications (desktop + optional webhook)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Product polish
|
||||
|
||||
- Signed release artifacts (agent + Pear desktop)
|
||||
- Role templates (viewer / SRE operator / admin)
|
||||
- Plugin SDK documentation
|
||||
- Grafana datasource (REST) cookbook
|
||||
- HoneyPeer presence for agent directory (opt-in)
|
||||
|
||||
---
|
||||
|
||||
## Non-goals (for now)
|
||||
|
||||
- Replacing Netdata Cloud SaaS multi-tenant product
|
||||
- Full byte-identical Netdata internal DB format
|
||||
- Shipping a browser-only public dashboard without auth by default (REST stays localhost unless explicitly bound)
|
||||
|
||||
---
|
||||
|
||||
## Milestone checklist (operators)
|
||||
|
||||
| Milestone | You can… |
|
||||
|-----------|----------|
|
||||
| M0 | `npm run start:server` + `curl localhost:19999/api/v3/info` |
|
||||
| M1 | Pear UI live charts from agent pubkey |
|
||||
| M2 | Mint `pd1.` operator invite; revoke peer |
|
||||
| M3 | Historical scrub 1h@1s via REST + RPC |
|
||||
| M4 | Container charts from Docker hosts |
|
||||
| M5 | Parent peer rolling up a homelab fleet |
|
||||
+8
-1
@@ -28,13 +28,20 @@
|
||||
| Identity file mode | `0600` |
|
||||
| Data directory | local `./data` (not committed) |
|
||||
|
||||
## REST API exposure
|
||||
|
||||
- Default bind is **localhost only** (`127.0.0.1:19999`).
|
||||
- REST is intentionally open on that bind (Netdata-agent style) — **do not** set `PEARDATA_REST_HOST=0.0.0.0` without a firewall, reverse proxy, or Holesail tunnel.
|
||||
- Prefer **P2P + roles** for remote multi-operator access; use REST for local scrapers/Grafana.
|
||||
|
||||
## Production checklist
|
||||
|
||||
- [ ] Never set `PEARDATA_INSECURE_OPEN_ADMIN` outside local demos
|
||||
- [ ] Keep `PEARDATA_DEFAULT_ROLE=viewer`
|
||||
- [ ] Prefer invites over sharing `SERVER_SEED`
|
||||
- [ ] Prefer `pd1.` invites over sharing `SERVER_SEED`
|
||||
- [ ] Use short `ttlMs` for high-privilege invites when practical
|
||||
- [ ] Set `PEARDATA_ALLOWLIST` if only known operators should dial
|
||||
- [ ] Keep REST on localhost unless explicitly secured
|
||||
- [ ] Back up `SERVER_SEED` offline; rotate by redeploying a new keypair (clients must re-dial)
|
||||
- [ ] Persist `data/` with mode `0700`; `audit.log` may contain peer ids
|
||||
- [ ] Run under systemd with `ProtectSystem` / `NoNewPrivileges` (see `deploy/`)
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
# HyperDB storage & linked-node sync
|
||||
|
||||
How PearData should adopt Holepunch’s **HyperDB + Corestore + Hyperswarm (+ Autobase)** stack for durable storage and P2P sync between linked agents — based on patterns in local clones under `holepunchto_repos` (`hyperdb`, `hyperdb-workshop`, `hyperdb-autobase-workshop`, `corestore`, `hyperswarm`, `autobee`, `pear-hyperdb`).
|
||||
|
||||
## Why HyperDB (not “just SQLite”)
|
||||
|
||||
| Need | HyperDB fit |
|
||||
|------|-------------|
|
||||
| Typed collections + indexes | Hyperschema + `@ns/collection` + secondary indexes |
|
||||
| Local high-perf | `HyperDB.rocks(path, def)` |
|
||||
| P2P replicate | `HyperDB.bee(hypercore, def, { autoUpdate })` over Corestore |
|
||||
| Multi-writer HA parents | Autobase whose **view** is HyperDB (`extension: false`) |
|
||||
| Same query API local + remote | Workshops prove one `Registry` class works for both |
|
||||
|
||||
PearData’s current `server/services/store.js` is an **in-memory ring**. HyperDB replaces durability + sync; the in-memory tier stays as the **hot 1s path**.
|
||||
|
||||
## Stack mapping (from Holepunch repos)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Agent
|
||||
COL[collector 1s] --> HOT[Memory tier0 ring]
|
||||
COL --> DS[Downsample]
|
||||
DS --> HDB[(HyperDB bee/rocks)]
|
||||
POL[peer-policy / alerts / links] --> HDB
|
||||
HOT --> RPC[protomux-rpc + REST]
|
||||
HDB --> RPC
|
||||
end
|
||||
subgraph Sync
|
||||
CS[Corestore] --> HDB
|
||||
SW[Hyperswarm] -->|store.replicate| CS
|
||||
AB[Autobase optional] -->|view| HDB
|
||||
end
|
||||
CHILD[Linked child agent] -.->|discoveryKey| SW
|
||||
PARENT[Parent / peer agent] -.-> SW
|
||||
DESK[Pear desktop cache] -.-> SW
|
||||
```
|
||||
|
||||
| Component | Repo pattern | PearData use |
|
||||
|-----------|--------------|--------------|
|
||||
| **Hyperschema + hyperdb/builder** | `hyperdb-workshop/build.js` | `spec/` codegen for collections |
|
||||
| **HyperDB.bee** | workshop `Registry` | Replicable agent DB |
|
||||
| **HyperDB.rocks** | `hyperdb` README | Optional local-only fast index |
|
||||
| **Corestore** | workshop `bin.js` | Named cores: `metrics-meta`, `alerts`, … |
|
||||
| **Hyperswarm** | `swarm.join(discoveryKey)` + `store.replicate(conn)` | Link nodes / seed DB |
|
||||
| **protomux-rpc** | already in PearData | Control plane (unchanged) |
|
||||
| **Autobase + hyperdispatch** | `hyperdb-autobase-workshop` | Multi-writer **parent** / HA registry |
|
||||
| **autobee** | experimental multiwriter bee | Alternative later; prefer Autobase+HyperDB view for now |
|
||||
| **pear-hyperdb** | Pear-shaped Model wrapper | Optional UX for desktop-local rocks |
|
||||
|
||||
## Critical design rule: don’t put 1s samples in HyperDB txs
|
||||
|
||||
HyperDB is an **indexable document DB** (put/get/find + flush). Writing every chart every second as HyperDB transactions will:
|
||||
|
||||
- Amplify Rocks/Bee write cost
|
||||
- Create huge replication chatter
|
||||
- Fight Netdata-class overhead goals
|
||||
|
||||
**Split planes:**
|
||||
|
||||
| Plane | Storage | Sync |
|
||||
|-------|---------|------|
|
||||
| **Hot live (≤1h @ 1s)** | Memory ring (current) | P2P `push:metrics` (current) |
|
||||
| **Warm history (downsampled)** | Hypercore append **or** HyperDB rows keyed `(chart, tsBucket)` | Corestore replicate |
|
||||
| **Metadata** (peers, alerts, labels, jobs, ACL cache) | **HyperDB** | Corestore replicate |
|
||||
| **Fleet / parent consensus** | Autobase → HyperDB view | Swarm on autobase discoveryKey |
|
||||
|
||||
## Proposed HyperDB schema (`@peardata/*`)
|
||||
|
||||
Modeled after workshop `build.js` namespaces.
|
||||
|
||||
### Collections
|
||||
|
||||
```text
|
||||
@peardata/node
|
||||
key: nodeId (string / pubkey hex)
|
||||
fields: hostname, platform, arch, cpus, agentVersion, labels{}, updatedAt
|
||||
|
||||
@peardata/peer-link
|
||||
key: [localNodeId, remotePublicKey]
|
||||
fields: role, alias, discoveryKey?, linkedAt, lastSeen, syncMode (push|pull|both)
|
||||
|
||||
@peardata/alert-config
|
||||
key: id
|
||||
fields: chart, dimension, warn, crit, comparator, enabled, info
|
||||
|
||||
@peardata/alert-event
|
||||
key: [id, ts] # or ulid
|
||||
fields: severity, value, threshold, message, cleared
|
||||
|
||||
@peardata/metric-point # WARM tier only (e.g. 1m buckets)
|
||||
key: [chart, ts]
|
||||
fields: context, values{} (map), tier
|
||||
|
||||
@peardata/job
|
||||
key: id
|
||||
fields: name, status, startedAt, finishedAt, result?
|
||||
```
|
||||
|
||||
### Indexes
|
||||
|
||||
```text
|
||||
@peardata/node-by-hostname → node.hostname
|
||||
@peardata/peer-link-by-remote → peer-link.remotePublicKey
|
||||
@peardata/alert-event-by-chart → alert-event.chart + ts
|
||||
@peardata/metric-point-by-context → metric-point.context + ts
|
||||
```
|
||||
|
||||
Rebuild with:
|
||||
|
||||
```bash
|
||||
node scripts/build-db.js # Hyperschema + HyperDB.toDisk → spec/
|
||||
```
|
||||
|
||||
## Agent integration shape
|
||||
|
||||
Follow `hyperdb-workshop` / `pear-hyperdb` Model pattern:
|
||||
|
||||
```text
|
||||
server/
|
||||
db/
|
||||
build.js # schema codegen
|
||||
spec/hyperschema/
|
||||
spec/hyperdb/
|
||||
model.js # PearDataModel: putNode, linkPeer, queryWarm, …
|
||||
replicate.js # Hyperswarm join + store.replicate
|
||||
services/
|
||||
store.js # HOT memory (keep)
|
||||
store-hyperdb.js # WARM + metadata facade used by queryData/REST
|
||||
```
|
||||
|
||||
### Boot (single-writer agent — Phase 2)
|
||||
|
||||
```js
|
||||
const store = new Corestore(dataDir + '/corestore')
|
||||
const swarm = new Hyperswarm({ keyPair: await store.createKeyPair('swarm') })
|
||||
swarm.on('connection', (conn) => store.replicate(conn))
|
||||
|
||||
const metaCore = store.get({ name: 'peardata-meta' })
|
||||
const db = HyperDB.bee(metaCore, spec, { autoUpdate: true })
|
||||
|
||||
// announce for linked peers / desktop seeders
|
||||
swarm.join(metaCore.discoveryKey, { server: true, client: true })
|
||||
```
|
||||
|
||||
Collector path:
|
||||
|
||||
1. Ingest → memory tier0 (unchanged)
|
||||
2. Every N samples → downsample → `tx.insert('@peardata/metric-point', …); tx.flush()`
|
||||
3. Alert transitions → `alert-event` collection
|
||||
4. `queryData` / REST: memory first, then HyperDB range scan for older windows
|
||||
|
||||
### Auth note
|
||||
|
||||
HyperDB replication shares **capability to read the core**, not PearData RPC roles. Keep:
|
||||
|
||||
- **Noise + MethodRoles** for mutating RPC (`setAlertConfig`, `runJob`)
|
||||
- Replication topic optionally gated (only invite-linked peers get discoveryKey / capability)
|
||||
- Do **not** announce writable cores to the public swarm without encryption / allowlist
|
||||
|
||||
## Linked nodes: sync modes
|
||||
|
||||
PearData “links” are first-class `@peardata/peer-link` rows + swarm topics.
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| **Pull** | Local agent opens remote DB by key (`HyperDB.bee(store.get({ key }), spec, { writable: false, autoUpdate: true })`) and replicates |
|
||||
| **Push** | Remote peers allowed to replicate our meta/warm cores (seed) |
|
||||
| **Both** | Mutual swarm join (homelab mesh) |
|
||||
| **Parent aggregate** | Parent pulls many children; stores namespaced copies or Autobase fleet view |
|
||||
|
||||
Discovery:
|
||||
|
||||
1. Desktop/admin mints link → stores remote pubkey + optional `dbKey` (z32/hex)
|
||||
2. Agent joins `discoveryKey` of that core
|
||||
3. On connection: `store.replicate(conn)` (workshop pattern)
|
||||
4. `clone.watch` / `autoUpdate` refreshes HyperDB indexes for REST/RPC queries
|
||||
|
||||
This is the same pattern as workshop §3.1 Lookups — **read path is swarm + HyperDB**, not constant RPC polling.
|
||||
|
||||
## Parent / HA (Phase 4) — Autobase workshop
|
||||
|
||||
When you need multi-writer fleet registry or HA parents:
|
||||
|
||||
- Autobase bootstrap key shared across parent instances
|
||||
- `open: (store) => HyperDB.bee(store.get('db-view'), spec, { extension: false, autoUpdate: true })`
|
||||
- hyperdispatch ops: `add-writer`, `put-alert`, `put-link`, `ingest-rollup`
|
||||
- RPC (existing PearData / workshop) appends ops; apply mutates HyperDB view
|
||||
- Clients dial **any** writer; view key stays stable
|
||||
|
||||
Do **not** “backup” by copying Corestore folders (workshop warning). Rotate writers via Autobase instead.
|
||||
|
||||
## What stays on protomux-rpc
|
||||
|
||||
| Keep on RPC | Why |
|
||||
|-------------|-----|
|
||||
| Live `push:metrics` | Sub-second UX; not DB-shaped |
|
||||
| `subscribeMetrics` / handshake / ACL | Session security |
|
||||
| `runJob`, mintInvite | Mutating control |
|
||||
| On-demand `queryData` for hot window | Memory path |
|
||||
|
||||
| Move to HyperDB (+ replicate) | Why |
|
||||
|-------------------------------|-----|
|
||||
| Peer links, aliases, labels | Shared fleet truth |
|
||||
| Alert config + history | Durable, queryable |
|
||||
| Warm/cold metric buckets | History without central server |
|
||||
| Node inventory | Parent `/api/v3/nodes` |
|
||||
|
||||
## Phased delivery (recommended)
|
||||
|
||||
### Phase A — Local durability (rocks or bee, no swarm yet)
|
||||
|
||||
1. Add `hyperdb`, `hyperschema`, `corestore` deps
|
||||
2. `scripts/build-db.js` + `spec/`
|
||||
3. Persist alert configs, peer policy, warm downsample into HyperDB
|
||||
4. REST/RPC history falls back to HyperDB after memory miss
|
||||
5. Keep REST localhost semantics
|
||||
|
||||
### Phase B — Link & replicate
|
||||
|
||||
1. Hyperswarm alongside HyperDHT RPC (or reuse connections carefully — often **separate swarm for store.replicate**)
|
||||
2. `linkPeer` / `unlinkPeer` RPCs write `@peardata/peer-link`
|
||||
3. Desktop can seed/cache agent DB by key for offline scrubbing
|
||||
4. Document `pd1.` invite vs **db discovery key** (two layers)
|
||||
|
||||
### Phase C — Parent fleet
|
||||
|
||||
1. Parent process pulls N child DB keys
|
||||
2. Composite REST `/api/v3/nodes` + scoped `/api/v3/data`
|
||||
3. Optional Autobase for parent HA
|
||||
|
||||
### Phase D — Polish
|
||||
|
||||
1. Encryption at rest (`encryptionKey` on cores / autobee)
|
||||
2. Blind-peer seeding for always-on warm history
|
||||
3. Grafana: already have REST Prometheus export; optionally expose hypercore-stats
|
||||
|
||||
## Concrete file plan (when implementing)
|
||||
|
||||
```text
|
||||
peardata/
|
||||
scripts/build-db.js
|
||||
spec/hyperschema/
|
||||
spec/hyperdb/
|
||||
server/db/model.js
|
||||
server/db/replicate.js
|
||||
server/services/store-hyperdb.js
|
||||
docs/STORAGE-HYPERDB.md ← this file
|
||||
```
|
||||
|
||||
Deps (approximate):
|
||||
|
||||
```json
|
||||
"hyperdb": "^6",
|
||||
"hyperschema": "^1",
|
||||
"corestore": "^7",
|
||||
"hyperswarm": "^4",
|
||||
"autobase": "^7",
|
||||
"hyperdispatch": "^1"
|
||||
```
|
||||
|
||||
(`autobee` only if you prefer that multiwriter path over Autobase+HyperDB view.)
|
||||
|
||||
## Relationship to current PearData roadmap
|
||||
|
||||
| Roadmap item | HyperDB role |
|
||||
|--------------|--------------|
|
||||
| Phase 2 tiered storage | Warm/cold collections |
|
||||
| Phase 3 PearDock/containers | Extra collections / indexes |
|
||||
| Phase 4 parent peer | Autobase + replicated views |
|
||||
| REST v3 historical queries | `find` ranges on `@peardata/metric-point` |
|
||||
|
||||
## References (local clones)
|
||||
|
||||
| Path under `holepunchto_repos` | Takeaway |
|
||||
|--------------------------------|----------|
|
||||
| `hyperdb/README.md` | rocks vs bee, find/get/tx, autoUpdate |
|
||||
| `hyperdb-workshop` | Schema builder, Corestore+Swarm replicate, RPC inserts |
|
||||
| `hyperdb-autobase-workshop` | Multi-writer view, hyperdispatch, HA |
|
||||
| `corestore/README.md` | `store.replicate(conn)`, namespacing |
|
||||
| `pear-hyperdb` | Thin Model wrapper pattern for Pear apps |
|
||||
| `autobee` | Experimental multiwriter bee alternative |
|
||||
|
||||
## Decision summary
|
||||
|
||||
1. **Yes — incorporate HyperDB** for metadata + warm history + linked-node sync.
|
||||
2. **Keep memory + RPC pushes** for live 1s Netdata feel.
|
||||
3. **Sync linked nodes via Corestore replication on Hyperswarm**, not by streaming every sample over RPC.
|
||||
4. **Use Autobase+HyperDB** when you need multi-writer parents / HA — same pattern as Holepunch’s own workshops.
|
||||
5. **Treat discovery keys as sensitive** — link only invited peers; RPC AuthZ remains authoritative for mutations.
|
||||
@@ -0,0 +1,98 @@
|
||||
# Tech choices
|
||||
|
||||
Recommendations for collection, streaming, visualization, and reusable libraries.
|
||||
|
||||
## Transport & identity
|
||||
|
||||
| Choice | Why |
|
||||
|--------|-----|
|
||||
| **HyperDHT** | Same peer identity + hole-punching model as PearDock |
|
||||
| **protomux-rpc** | Multiplexed request/response + server push events |
|
||||
| **compact-encoding JSON** | Fast enough for control + 1s metric batches; shared with template |
|
||||
| **Ed25519 seeds** | Agent address = pubkey; admin proof via HMAC from seed |
|
||||
|
||||
### Streaming metrics
|
||||
|
||||
MVP uses **protomux-rpc `event` pushes** (`push:metrics`) with per-session subscribe + throttle.
|
||||
|
||||
If fleet scale demands it later:
|
||||
|
||||
1. Dedicated protomux channel with binary packs (Float64 arrays)
|
||||
2. Hypercore / Hyperbee for durable streams a parent can replicate
|
||||
3. Extract a shared **`pearrpc`** package (session middleware + hot-path helpers) from this template
|
||||
|
||||
For MVP, JSON pushes keep the stack simple and debuggable.
|
||||
|
||||
## Collection
|
||||
|
||||
| Option | Verdict |
|
||||
|--------|---------|
|
||||
| **Node `os` + `/proc` (chosen)** | Zero native deps, good enough for MVP, low overhead |
|
||||
| `node-os-utils` | Convenient but extra dep / less control |
|
||||
| Native bindings (netdata collectors, `systeminformation`) | Higher fidelity; consider Phase 2 for Windows depth |
|
||||
| Shell out to `vmstat`/`iostat` | Avoid on hot path |
|
||||
|
||||
**Linux:** `/proc/meminfo`, `/proc/loadavg`, `/proc/net/dev`, `/proc/diskstats`
|
||||
**macOS/Windows:** `os` fallbacks (CPU%, freemem, loadavg where available); I/O/net rates may be zero until platform collectors land.
|
||||
|
||||
Target overhead: single timer, no child processes per tick, ring buffers only.
|
||||
|
||||
## Storage
|
||||
|
||||
| Tier | Implementation |
|
||||
|------|----------------|
|
||||
| Now | In-memory ring (`server/services/store.js`) |
|
||||
| Next | **HyperDB** + Corestore (+ Hyperswarm replicate) — see [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) |
|
||||
| HA / multi-writer parents | Autobase with HyperDB view (`extension: false`) |
|
||||
| Avoid | SQLite as primary P2P store; copying corestore folders as “backup” |
|
||||
|
||||
### HyperDB engines
|
||||
|
||||
| Engine | Sync | Use |
|
||||
|--------|------|-----|
|
||||
| `HyperDB.bee(core, spec)` | P2P via Hypercore | Agent meta + warm history + linked peers |
|
||||
| `HyperDB.rocks(path, spec)` | Local only | Fast local index / desktop cache (`pear-hyperdb` style) |
|
||||
|
||||
## Visualization (desktop)
|
||||
|
||||
| Option | Verdict |
|
||||
|--------|---------|
|
||||
| **Canvas sparklines (chosen MVP)** | No extra deps inside Pear; snappy for 1s updates |
|
||||
| **uPlot** | Best next step for Netdata-like interactive charts (tiny, fast) |
|
||||
| Chart.js / ECharts | Heavier; fine for secondary views |
|
||||
| Grafana via REST | External; use `/api/v3/data` + Prometheus export |
|
||||
|
||||
## REST
|
||||
|
||||
Built-in Node `http` (no Express) — small attack surface, enough for Netdata-style GET APIs.
|
||||
|
||||
## Packaging
|
||||
|
||||
| Piece | Approach |
|
||||
|-------|----------|
|
||||
| Agent | Node 20+ , systemd unit `deploy/peardata.service` |
|
||||
| Desktop | Pear (`pear-electron` + `pear-bridge`) |
|
||||
| Invites | `pd1.` tokens (PearDock-style) |
|
||||
| Installer | Phase 1 one-liner script |
|
||||
|
||||
## Reusable library extraction (recommended later)
|
||||
|
||||
From this codebase / PearDock patterns:
|
||||
|
||||
| Package | Contents |
|
||||
|---------|----------|
|
||||
| `@pear/rpc-session` or `pearrpc` | PeerSession middleware, ACL, rate limit, audit hooks |
|
||||
| `pear-metrics-wire` | Context/chart catalog types + query args |
|
||||
| `pear-invite` | `pd1.` encode/decode + capability HMAC |
|
||||
|
||||
Keep MIT-clean relative to AGPL PearDock sources (template already does).
|
||||
|
||||
## Integration seams
|
||||
|
||||
| Ecosystem | Seam |
|
||||
|-----------|------|
|
||||
| PearDock | Container chart plugin reading dock RPC |
|
||||
| PearVirt | VM CPU/mem contexts |
|
||||
| HoneyPeer | Optional agent directory announcements |
|
||||
| BareOS | Bare-compatible collector build |
|
||||
| Holesail | Tunnel REST or future agent web UI |
|
||||
+8
-6
@@ -14,8 +14,10 @@
|
||||
|------|----------|
|
||||
| `test/acl.test.js` | Role hierarchy, `assertAllowed` |
|
||||
| `test/crypto-auth.test.js` | MAC key, capabilities, admin proof, invites, classify input |
|
||||
| `test/protocol.test.js` | Constants, `MethodRoles`, schema validators |
|
||||
| `test/integration.test.js` | Live HyperDHT server + client handshake, post, push |
|
||||
| `test/protocol.test.js` | Constants, monitoring `MethodRoles`, schema validators |
|
||||
| `test/store.test.js` | Metric ring buffer ingest + query |
|
||||
| `test/rest.test.js` | Netdata-style `/api/v3` routes |
|
||||
| `test/integration.test.js` | Live HyperDHT agent + handshake + metrics query |
|
||||
|
||||
```bash
|
||||
npm test
|
||||
@@ -27,10 +29,10 @@ SKIP_INTEGRATION=1 npm test
|
||||
|
||||
## Integration test behavior
|
||||
|
||||
1. Starts an ephemeral HyperDHT server in-process
|
||||
1. Starts metrics pipeline + ephemeral HyperDHT agent in-process
|
||||
2. Sets `PEARDATA_INSECURE_OPEN_ADMIN=1` for the process (restored in teardown)
|
||||
3. Dials as a client, handshakes, posts a message, asserts push delivery
|
||||
4. Tears down sockets / DHT
|
||||
3. Dials as a client, handshakes, lists charts, queries `system.cpu`
|
||||
4. Tears down collector / sockets / DHT
|
||||
|
||||
Requires outbound/inbound UDP for HyperDHT. If the test hangs or fails on a locked-down network, use `SKIP_INTEGRATION=1`.
|
||||
|
||||
@@ -78,7 +80,7 @@ When adding an RPC method:
|
||||
| Desktop chrome | `npm start` — drag titlebar, resize edges, min/max/close |
|
||||
| Admin connect | Paste key + `SERVER_SEED` in UI |
|
||||
| Invite connect | Paste `pd1.…` without seed |
|
||||
| Viewer denial | Public key only → `postMessage` fails with permission error |
|
||||
| Viewer denial | Public key only → `runJob` / `mintInvite` fail with permission error |
|
||||
| Health | With server up: `SERVER_PUBLIC_KEY=… npm run healthcheck` |
|
||||
| Soak | `SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run soak` |
|
||||
|
||||
|
||||
Generated
+240
-3
@@ -11,16 +11,21 @@
|
||||
"dependencies": {
|
||||
"b4a": "^1.8.1",
|
||||
"compact-encoding": "^3.3.0",
|
||||
"corestore": "^7.11.1",
|
||||
"dotenv": "^17.4.2",
|
||||
"graceful-goodbye": "^1.3.3",
|
||||
"hypercore-crypto": "^3.7.0",
|
||||
"hypercore-id-encoding": "^1.3.0",
|
||||
"hyperdb": "^6.7.0",
|
||||
"hyperdht": "^6.33.0",
|
||||
"hyperschema": "^1.21.0",
|
||||
"pear-bridge": "^1.2.5",
|
||||
"pear-electron": "^1.7.28",
|
||||
"pear-pipe": "^1.0.6",
|
||||
"pear-run": "^1.0.8",
|
||||
"protomux": "^3.11.0",
|
||||
"protomux-rpc": "^1.10.0",
|
||||
"ready-resource": "^1.2.0",
|
||||
"safety-catch": "^1.0.3",
|
||||
"z32": "^1.1.0"
|
||||
},
|
||||
@@ -825,6 +830,12 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/big-sparse-array": {
|
||||
"version": "1.0.3",
|
||||
"resolved": "https://registry.npmjs.org/big-sparse-array/-/big-sparse-array-1.0.3.tgz",
|
||||
"integrity": "sha512-6RjV/3mSZORlMdpUaQ6rUSpG637cZm0//E54YYGtQg1c1O+AbZP8UTdJ/TchsDZcTVLmyWZcseBfp2HBeXUXOQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/binary-stream-equals": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/binary-stream-equals/-/binary-stream-equals-1.0.0.tgz",
|
||||
@@ -929,6 +940,15 @@
|
||||
"integrity": "sha512-9z8TZaGM1pfswYeXrUpzPrkx8UnWYdhJclsiYMm6x/w5+nN+8Tf/LnAgfLGQCm59qAOxU8WwHEq2vNwF6i4j+Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/codecs": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/codecs/-/codecs-3.1.0.tgz",
|
||||
"integrity": "sha512-Dqx8NwvBvnMeuPQdVKy/XEF71igjR5apxBvCGeV0pP1tXadOiaLvDTXt7xh+/5wI1ASB195mXQGJbw3Ml4YDWQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.3"
|
||||
}
|
||||
},
|
||||
"node_modules/compact-encoding": {
|
||||
"version": "3.3.0",
|
||||
"resolved": "https://registry.npmjs.org/compact-encoding/-/compact-encoding-3.3.0.tgz",
|
||||
@@ -963,6 +983,24 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/corestore": {
|
||||
"version": "7.11.1",
|
||||
"resolved": "https://registry.npmjs.org/corestore/-/corestore-7.11.1.tgz",
|
||||
"integrity": "sha512-X6UsGFyHeAcqffTqgoDV7WDZjhfvolWabwzws6GbdqeXQY0CTM6gNpNMGEi6RGvAdZDPnFaboF5jFBy1lw1rDA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.7",
|
||||
"bare-events": "^2.8.3",
|
||||
"hypercore": "^11.32.0",
|
||||
"hypercore-crypto": "^3.4.2",
|
||||
"hypercore-errors": "^1.4.0",
|
||||
"hypercore-id-encoding": "^1.3.0",
|
||||
"ready-resource": "^1.1.1",
|
||||
"sodium-universal": "^5.0.1",
|
||||
"streamx": "^2.26.0",
|
||||
"which-runtime": "^1.2.1"
|
||||
}
|
||||
},
|
||||
"node_modules/crc-native": {
|
||||
"version": "1.1.8",
|
||||
"resolved": "https://registry.npmjs.org/crc-native/-/crc-native-1.1.8.tgz",
|
||||
@@ -982,6 +1020,26 @@
|
||||
"crc-native": "^1.0.3"
|
||||
}
|
||||
},
|
||||
"node_modules/debounceify": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/debounceify/-/debounceify-1.1.0.tgz",
|
||||
"integrity": "sha512-eKuHDVfJVg+u/0nPy8P+fhnLgbyuTgVxuCRrS/R7EpDSMMkBDgSes41MJtSAY1F1hcqfHz3Zy/qpqHHIp/EhdA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/device-file": {
|
||||
"version": "2.3.1",
|
||||
"resolved": "https://registry.npmjs.org/device-file/-/device-file-2.3.1.tgz",
|
||||
"integrity": "sha512-bmON44lwxJPle9N2OcH4tqM44pMGZKT8G6OkzXkz0urvqQ9LKkoQdTS6w0ztYfmLdUE27R4UUmx0+y2gEz4Jug==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.7",
|
||||
"bare-fs": "^4.0.1",
|
||||
"bare-path": "^3.0.0",
|
||||
"fd-lock": "^2.1.0",
|
||||
"fs-native-extensions": "^1.4.0",
|
||||
"ready-resource": "^1.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/dht-rpc": {
|
||||
"version": "6.27.0",
|
||||
"resolved": "https://registry.npmjs.org/dht-rpc/-/dht-rpc-6.27.0.tgz",
|
||||
@@ -1056,6 +1114,24 @@
|
||||
"integrity": "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/fd-lock": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/fd-lock/-/fd-lock-2.2.0.tgz",
|
||||
"integrity": "sha512-Il4jWBhjjgvUg+z8d0bC7ncXqB42caqKTUpnZ9jpcqiPmw8bkIixt7Kic1czbZPMxAQD/9kEqfZ5Dq77nSll0w==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"bare-fs": "^4.5.0",
|
||||
"fs-native-extensions": "^1.4.4",
|
||||
"ready-resource": "^1.2.0",
|
||||
"resource-on-exit": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/flat-tree": {
|
||||
"version": "1.13.0",
|
||||
"resolved": "https://registry.npmjs.org/flat-tree/-/flat-tree-1.13.0.tgz",
|
||||
"integrity": "sha512-fT3HIuCPwHhFgJ20QYzDHgUG0zMmFg5cHvFiFo5h+QMSJ28TihsEVY0f8HGliuO+pOzmvjMx1odToeaEWkTnyQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/framed-stream": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/framed-stream/-/framed-stream-1.0.1.tgz",
|
||||
@@ -1132,6 +1208,55 @@
|
||||
"safety-catch": "^1.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/hyperbee": {
|
||||
"version": "2.27.3",
|
||||
"resolved": "https://registry.npmjs.org/hyperbee/-/hyperbee-2.27.3.tgz",
|
||||
"integrity": "sha512-PXURH2U4juUZyJRKHTrY5z1zX851pmI1Q0jfv5F/hCIErDt/ND8jOZuxc3hfOLM9f0W3qJEDTMlV5AJBkVPy8w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.0",
|
||||
"codecs": "^3.0.0",
|
||||
"debounceify": "^1.0.0",
|
||||
"hypercore-errors": "^1.0.0",
|
||||
"mutexify": "^1.4.0",
|
||||
"protocol-buffers-encodings": "^1.2.0",
|
||||
"rache": "^1.0.0",
|
||||
"ready-resource": "^1.0.0",
|
||||
"resolve-reject-promise": "^1.1.0",
|
||||
"safety-catch": "^1.0.2",
|
||||
"streamx": "^2.12.4",
|
||||
"unslab": "^1.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/hypercore": {
|
||||
"version": "11.34.1",
|
||||
"resolved": "https://registry.npmjs.org/hypercore/-/hypercore-11.34.1.tgz",
|
||||
"integrity": "sha512-nlrbXjI4x59akT8AwVEFLN54OicB53JDnH1xYP4j9zKDksHUVgqJFXnFwOkRr4Nr3kwCU0b9Rh3eIsU0JX5HEA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@hyperswarm/secret-stream": "^6.0.0",
|
||||
"b4a": "^1.1.0",
|
||||
"bare-events": "^2.2.0",
|
||||
"big-sparse-array": "^1.0.3",
|
||||
"compact-encoding": "^3.0.0",
|
||||
"fast-fifo": "^1.3.0",
|
||||
"flat-tree": "^1.9.0",
|
||||
"hypercore-crypto": "^3.2.1",
|
||||
"hypercore-errors": "^1.5.0",
|
||||
"hypercore-id-encoding": "^1.2.0",
|
||||
"hypercore-storage": "^3.2.0",
|
||||
"is-options": "^1.0.1",
|
||||
"nanoassert": "^2.0.0",
|
||||
"protomux": "^3.5.0",
|
||||
"quickbit-universal": "^2.2.0",
|
||||
"random-array-iterator": "^1.0.0",
|
||||
"safety-catch": "^1.0.1",
|
||||
"sodium-universal": "^5.0.1",
|
||||
"streamx": "^2.12.4",
|
||||
"unslab": "^1.3.0",
|
||||
"z32": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/hypercore-crypto": {
|
||||
"version": "3.7.0",
|
||||
"resolved": "https://registry.npmjs.org/hypercore-crypto/-/hypercore-crypto-3.7.0.tgz",
|
||||
@@ -1143,6 +1268,15 @@
|
||||
"sodium-universal": "^5.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/hypercore-errors": {
|
||||
"version": "1.5.0",
|
||||
"resolved": "https://registry.npmjs.org/hypercore-errors/-/hypercore-errors-1.5.0.tgz",
|
||||
"integrity": "sha512-5KQ/SuDxsvet+7qWA35Ay6zdD9WyAHQoyWHGcPUTbmJBd300gvNIJoi3oma7kp4TTCSzii6qYumNZe/s0j/saQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"hypercore-id-encoding": "^1.3.0"
|
||||
}
|
||||
},
|
||||
"node_modules/hypercore-id-encoding": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/hypercore-id-encoding/-/hypercore-id-encoding-1.3.0.tgz",
|
||||
@@ -1153,6 +1287,47 @@
|
||||
"z32": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/hypercore-storage": {
|
||||
"version": "3.2.0",
|
||||
"resolved": "https://registry.npmjs.org/hypercore-storage/-/hypercore-storage-3.2.0.tgz",
|
||||
"integrity": "sha512-i5O5ZMkdZaNAWGuNRXUZjuzv7B41OIhDHUwLCZPjucgVmywY0ZE5PDafu6e5oPuhtghjl6S8q6Jn+POVKN2BvQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.7",
|
||||
"bare-path": "^3.0.0",
|
||||
"compact-encoding": "^3.1.0",
|
||||
"device-file": "^2.1.2",
|
||||
"flat-tree": "^1.12.1",
|
||||
"hypercore-crypto": "^3.4.2",
|
||||
"hyperschema": "^1.21.0",
|
||||
"index-encoder": "^3.3.2",
|
||||
"resolve-reject-promise": "^1.0.0",
|
||||
"rocksdb-native": "^3.11.0",
|
||||
"scope-lock": "^1.2.4",
|
||||
"streamx": "^2.21.1",
|
||||
"xache": "^1.2.1"
|
||||
}
|
||||
},
|
||||
"node_modules/hyperdb": {
|
||||
"version": "6.7.0",
|
||||
"resolved": "https://registry.npmjs.org/hyperdb/-/hyperdb-6.7.0.tgz",
|
||||
"integrity": "sha512-SVzV1mWNxD8c28sDlAXalOsac5ZPhSSknYwwVdMZyFz4vteXgANkq6pGDkvI2UUH2ZkF0KvcsGrBgLB+pbX41w==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.6",
|
||||
"compact-encoding": "^3.0.0",
|
||||
"generate-object-property": "^2.0.0",
|
||||
"generate-string": "^1.0.1",
|
||||
"hyperbee": "^2.24.2",
|
||||
"hypercore": "^11.29.0",
|
||||
"hyperschema": "^1.9.2",
|
||||
"index-encoder": "^3.4.0",
|
||||
"refcounter": "^1.0.0",
|
||||
"rocksdb-native": "^3.0.0",
|
||||
"scope-lock": "^1.2.4",
|
||||
"streamx": "^2.20.0"
|
||||
}
|
||||
},
|
||||
"node_modules/hyperdht": {
|
||||
"version": "6.33.0",
|
||||
"resolved": "https://registry.npmjs.org/hyperdht/-/hyperdht-6.33.0.tgz",
|
||||
@@ -1205,6 +1380,24 @@
|
||||
"generate-string": "^1.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/index-encoder": {
|
||||
"version": "3.5.0",
|
||||
"resolved": "https://registry.npmjs.org/index-encoder/-/index-encoder-3.5.0.tgz",
|
||||
"integrity": "sha512-idZ1cxtZz2dRV6rUiaP9Xo99UjXbSzjcMacoQmxUMu/A7fEQcNPngvwDJYeWelQUS5XFlY71/or70lKn6XnwbQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.4"
|
||||
}
|
||||
},
|
||||
"node_modules/is-options": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/is-options/-/is-options-1.0.2.tgz",
|
||||
"integrity": "sha512-u+Ai74c8Q74aS8BuHwPdI1jptGOT1FQXgCq8/zv0xRuE+wRgSMEJLj8lVO8Zp9BeGb29BXY6AsNPinfqjkr7Fg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"b4a": "^1.1.1"
|
||||
}
|
||||
},
|
||||
"node_modules/is-property": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/is-property/-/is-property-1.0.2.tgz",
|
||||
@@ -1345,9 +1538,9 @@
|
||||
"pear-wakeups": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/peardatadrive": {
|
||||
"node_modules/pear-appdrive": {
|
||||
"version": "1.1.2",
|
||||
"resolved": "https://registry.npmjs.org/peardatadrive/-/peardatadrive-1.1.2.tgz",
|
||||
"resolved": "https://registry.npmjs.org/pear-appdrive/-/pear-appdrive-1.1.2.tgz",
|
||||
"integrity": "sha512-AmCrZaNGFU/BvHR+oFP/cNEUCH1vbzezsJV9wqASoggyg8d4sSb0L6kr99ugGkAuUFlOBI2uHni71hBTaO2rDA==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
@@ -1365,7 +1558,7 @@
|
||||
"bare-http1": "^4.0.2",
|
||||
"get-mime-type": "^2.0.0",
|
||||
"listen-async": "^1.0.0",
|
||||
"peardatadrive": "^1.0.0",
|
||||
"pear-appdrive": "^1.0.0",
|
||||
"pear-errors": "^1.0.0",
|
||||
"pear-gunk": "^1.0.0",
|
||||
"pear-stamp": "^1.0.1",
|
||||
@@ -1754,6 +1947,17 @@
|
||||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/protocol-buffers-encodings": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/protocol-buffers-encodings/-/protocol-buffers-encodings-1.2.0.tgz",
|
||||
"integrity": "sha512-daeNPuKh1NlLD1uDfbLpD+xyUTc07nEtfHwmBZmt/vH0B7VOM+JOCOpDcx9ZRpqHjAiIkGqyTDi+wfGSl17R9w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"b4a": "^1.6.0",
|
||||
"signed-varint": "^2.0.1",
|
||||
"varint": "5.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/protomux": {
|
||||
"version": "3.11.0",
|
||||
"resolved": "https://registry.npmjs.org/protomux/-/protomux-3.11.0.tgz",
|
||||
@@ -1840,6 +2044,18 @@
|
||||
"streamx": "^2.23.0"
|
||||
}
|
||||
},
|
||||
"node_modules/rache": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/rache/-/rache-1.0.0.tgz",
|
||||
"integrity": "sha512-e0k0g0w/8jOCB+7YqCIlOa+OJ38k0wrYS4x18pMSmqOvLKoyhmMhmQyCcvfY6VaP8D75cqkEnlakXs+RYYLqNg==",
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/random-array-iterator": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/random-array-iterator/-/random-array-iterator-1.0.0.tgz",
|
||||
"integrity": "sha512-u7xCM93XqKEvPTP6xZp2ehttcAemKnh73oKNf1FvzuVCfpt6dILDt1Kxl1LeBjm2iNIeR49VGFhy4Iz3yOun+Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/read-write-mutexify": {
|
||||
"version": "2.1.0",
|
||||
"resolved": "https://registry.npmjs.org/read-write-mutexify/-/read-write-mutexify-2.1.0.tgz",
|
||||
@@ -1931,6 +2147,12 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/scope-lock": {
|
||||
"version": "1.2.4",
|
||||
"resolved": "https://registry.npmjs.org/scope-lock/-/scope-lock-1.2.4.tgz",
|
||||
"integrity": "sha512-BpSd8VCuCxW9ZitcdIC/vjs3gMaP9bRBL5nkHcyfX2VrS52n13/rHuBA2xJ/S/4DPuRdAO/Bk8pWd8eD/gHCIA==",
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/script-linker": {
|
||||
"version": "2.5.4",
|
||||
"resolved": "https://registry.npmjs.org/script-linker/-/script-linker-2.5.4.tgz",
|
||||
@@ -1954,6 +2176,15 @@
|
||||
"integrity": "sha512-WBgv0UnIq2C+Aeh0/n+IRpP6967eIx9WpynTUoiW3isPpfe1zu2LJzyfXdo9Tgef8yR/sGjcMvoUXD7EYdiz+g==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/signed-varint": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/signed-varint/-/signed-varint-2.0.1.tgz",
|
||||
"integrity": "sha512-abgDPg1106vuZZOvw7cFwdCABddfJRz5akcCcchzTbhyhYnsG31y4AlZEgp315T7W3nQq5P4xeOm186ZiPVFzw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"varint": "~5.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/simdle-native": {
|
||||
"version": "1.3.9",
|
||||
"resolved": "https://registry.npmjs.org/simdle-native/-/simdle-native-1.3.9.tgz",
|
||||
@@ -2188,6 +2419,12 @@
|
||||
"node": ">=10.12.0"
|
||||
}
|
||||
},
|
||||
"node_modules/varint": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/varint/-/varint-5.0.0.tgz",
|
||||
"integrity": "sha512-gC13b/bWrqQoKY2EmROCZ+AR0jitc6DnDGaQ6Ls9QpKmuSgJB1eQ7H3KETtQm7qSdMWMKCmsshyCmUwMLh3OAA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/webidl-conversions": {
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-4.0.2.tgz",
|
||||
|
||||
+10
-5
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "peardata",
|
||||
"version": "0.1.0",
|
||||
"description": "Production-ready HyperDHT + protomux-rpc P2P app template (demo room + presence)",
|
||||
"description": "Decentralized P2P Netdata-class monitoring — PearMonitor agent + Pear desktop + REST v3",
|
||||
"type": "module",
|
||||
"license": "MIT",
|
||||
"main": "index.js",
|
||||
@@ -15,10 +15,10 @@
|
||||
"gui": {
|
||||
"main": "index.html",
|
||||
"backgroundColor": "#0b1020",
|
||||
"height": 780,
|
||||
"width": 1100,
|
||||
"minWidth": 720,
|
||||
"minHeight": 480,
|
||||
"height": 860,
|
||||
"width": 1280,
|
||||
"minWidth": 900,
|
||||
"minHeight": 560,
|
||||
"resizable": true,
|
||||
"movable": true,
|
||||
"minimizable": true,
|
||||
@@ -50,16 +50,21 @@
|
||||
"dependencies": {
|
||||
"b4a": "^1.8.1",
|
||||
"compact-encoding": "^3.3.0",
|
||||
"corestore": "^7.11.1",
|
||||
"dotenv": "^17.4.2",
|
||||
"graceful-goodbye": "^1.3.3",
|
||||
"hypercore-crypto": "^3.7.0",
|
||||
"hypercore-id-encoding": "^1.3.0",
|
||||
"hyperdb": "^6.7.0",
|
||||
"hyperdht": "^6.33.0",
|
||||
"hyperschema": "^1.21.0",
|
||||
"pear-bridge": "^1.2.5",
|
||||
"pear-electron": "^1.7.28",
|
||||
"pear-pipe": "^1.0.6",
|
||||
"pear-run": "^1.0.8",
|
||||
"protomux": "^3.11.0",
|
||||
"protomux-rpc": "^1.10.0",
|
||||
"ready-resource": "^1.2.0",
|
||||
"safety-catch": "^1.0.3",
|
||||
"z32": "^1.1.0"
|
||||
},
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Mint a pa1 invite from SERVER_SEED without starting the full server process.
|
||||
* Mint a pd1 invite from SERVER_SEED without starting the full server process.
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/mint-invite.js [role=operator] [ttlMs]
|
||||
|
||||
+30
-16
@@ -27,7 +27,6 @@ fi
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$ROOT"
|
||||
|
||||
# Derive invite prefix from first letters + 1.
|
||||
PREFIX="$(echo "$NEW_SLUG" | tr -cd 'a-z' | cut -c1-2)1."
|
||||
ENV_PREFIX="$(echo "$NEW_SLUG" | tr 'a-z-' 'A-Z_')_"
|
||||
ENV_PREFIX="${ENV_PREFIX//__/_}"
|
||||
@@ -40,24 +39,39 @@ echo " invite: $OLD_PREFIX → $PREFIX"
|
||||
echo " env: $OLD_ENV → $ENV_PREFIX"
|
||||
|
||||
export LC_ALL=C
|
||||
find . -type f \
|
||||
\( -name '*.js' -o -name '*.mjs' -o -name '*.cjs' -o -name '*.json' -o -name '*.md' -o -name '*.html' -o -name '*.css' -o -name '*.yml' -o -name '*.yaml' -o -name '*.service' -o -name '*.example' -o -name '*.sh' \) \
|
||||
! -path './node_modules/*' ! -path './.git/*' ! -path './data/*' \
|
||||
-print0 | while IFS= read -r -d '' f; do
|
||||
perl -pi -e "
|
||||
s/\Q$OLD_NAME\E/$NEW_SLUG/g;
|
||||
s/\Q$OLD_PRODUCT\E/$NEW_PRODUCT/g;
|
||||
s/\Q$OLD_PROTOCOL\E/${NEW_SLUG}\\/rpc/g;
|
||||
s/\Q$OLD_SLUG\E/$NEW_SLUG/g;
|
||||
s/\Q$OLD_PREFIX\E/$PREFIX/g;
|
||||
s/\Q$OLD_ENV\E/$ENV_PREFIX/g;
|
||||
" "$f"
|
||||
done
|
||||
# Use python for safe replacements (perl struggles with slashes in protocol id)
|
||||
python3 - "$OLD_NAME" "$NEW_SLUG" "$OLD_PRODUCT" "$NEW_PRODUCT" "$OLD_PROTOCOL" "${NEW_SLUG}/rpc" "$OLD_SLUG" "$NEW_SLUG" "$OLD_PREFIX" "$PREFIX" "$OLD_ENV" "$ENV_PREFIX" <<'PY'
|
||||
import os, sys
|
||||
from pathlib import Path
|
||||
|
||||
if [[ -f bin/peardata-server.mjs ]]; then
|
||||
pairs = list(zip(sys.argv[1::2], sys.argv[2::2]))
|
||||
# longer keys first
|
||||
pairs.sort(key=lambda x: -len(x[0]))
|
||||
SKIP = {'node_modules', '.git', 'data'}
|
||||
EXTS = {'.js','.mjs','.cjs','.json','.md','.html','.css','.yml','.yaml','.service','.example','.sh'}
|
||||
|
||||
for dirpath, dirnames, filenames in os.walk('.'):
|
||||
dirnames[:] = [d for d in dirnames if d not in SKIP]
|
||||
for name in filenames:
|
||||
p = Path(dirpath) / name
|
||||
if p.suffix not in EXTS:
|
||||
continue
|
||||
try:
|
||||
text = p.read_text(encoding='utf-8')
|
||||
except Exception:
|
||||
continue
|
||||
orig = text
|
||||
for a, b in pairs:
|
||||
text = text.replace(a, b)
|
||||
if text != orig:
|
||||
p.write_text(text, encoding='utf-8')
|
||||
print('updated', p)
|
||||
PY
|
||||
|
||||
if [[ -f bin/peardata-server.mjs && "$NEW_SLUG" != "peardata" ]]; then
|
||||
mv bin/peardata-server.mjs "bin/${NEW_SLUG}-server.mjs"
|
||||
fi
|
||||
if [[ -f deploy/peardata.service ]]; then
|
||||
if [[ -f deploy/peardata.service && "$NEW_SLUG" != "peardata" ]]; then
|
||||
mv deploy/peardata.service "deploy/${NEW_SLUG}.service"
|
||||
fi
|
||||
|
||||
|
||||
+7
-4
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Soak test: connect, post messages, verify pushes for SOAK_DURATION_MS.
|
||||
* Soak test: connect, query metrics, ping for SOAK_DURATION_MS.
|
||||
*
|
||||
* Usage:
|
||||
* SERVER_PUBLIC_KEY=… SERVER_SEED=… node scripts/soak.js
|
||||
@@ -26,18 +26,21 @@ let received = 0
|
||||
let sent = 0
|
||||
let errors = 0
|
||||
|
||||
conn.on(Pushes.message, () => {
|
||||
conn.on(Pushes.metrics, () => {
|
||||
received++
|
||||
})
|
||||
|
||||
await conn.connect()
|
||||
console.log('soak connected as', conn.role)
|
||||
await conn.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
||||
|
||||
const end = Date.now() + duration
|
||||
while (Date.now() < end) {
|
||||
try {
|
||||
await conn.request(Methods.postMessage, {
|
||||
text: `soak ${sent} @ ${new Date().toISOString()}`,
|
||||
await conn.request(Methods.queryData, {
|
||||
chart: 'system.cpu',
|
||||
after: -30,
|
||||
points: 30,
|
||||
})
|
||||
sent++
|
||||
await conn.ping()
|
||||
|
||||
@@ -5,10 +5,14 @@ import fs from 'fs'
|
||||
import path from 'path'
|
||||
|
||||
const MUTATING = new Set([
|
||||
'postMessage',
|
||||
'clearMessages',
|
||||
'mintInvite',
|
||||
'revokePeer',
|
||||
'setAlertConfig',
|
||||
'ackAlert',
|
||||
'silenceAlert',
|
||||
'runJob',
|
||||
'cancelJob',
|
||||
'exportSnapshot',
|
||||
'handshake',
|
||||
])
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Export formatters (JSON / Prometheus / shell) for /api/v*/allmetrics.
|
||||
* Export formatters (JSON / Prometheus / shell) for /api/v1|v2|v3/allmetrics.
|
||||
*/
|
||||
import os from 'os'
|
||||
import { getStore } from '../services/store.js'
|
||||
|
||||
@@ -122,7 +122,9 @@ export class MetricsCollector extends EventEmitter {
|
||||
*/
|
||||
constructor(opts = {}) {
|
||||
super()
|
||||
this.intervalMs = opts.intervalMs ?? Number(process.env.PEARDATA_SAMPLE_MS) || SAMPLE_INTERVAL_MS
|
||||
this.intervalMs =
|
||||
opts.intervalMs ??
|
||||
(Number(process.env.PEARDATA_SAMPLE_MS) || SAMPLE_INTERVAL_MS)
|
||||
this.timer = null
|
||||
this.running = false
|
||||
this.sampleCount = 0
|
||||
@@ -257,7 +259,7 @@ export class MetricsCollector extends EventEmitter {
|
||||
values: {
|
||||
running: load.running,
|
||||
blocked: 0,
|
||||
total: load.total || Object.keys(process || {}).length,
|
||||
total: load.total || 0,
|
||||
},
|
||||
})
|
||||
|
||||
|
||||
@@ -301,7 +301,7 @@ export function encodeInvite(pkg) {
|
||||
export function decodeInvite(invite) {
|
||||
const s = String(invite || '').trim()
|
||||
if (!s.startsWith(INVITE_PREFIX)) {
|
||||
return { ok: false, error: 'Not a pa1 invite', code: 'INVITE_INVALID' }
|
||||
return { ok: false, error: 'Not a pd1 invite', code: 'INVITE_INVALID' }
|
||||
}
|
||||
try {
|
||||
const json = b64urlDecode(s.slice(INVITE_PREFIX.length)).toString('utf8')
|
||||
|
||||
+10
-6
@@ -2,10 +2,12 @@ import test from 'brittle'
|
||||
import { Roles, roleAllows, MethodRoles } from '../shared/protocol.js'
|
||||
import { assertAllowed, maxRole } from '../server/core/acl.js'
|
||||
|
||||
test('assertAllowed allows operator postMessage', (t) => {
|
||||
t.exception(() => assertAllowed(Roles.viewer, 'postMessage'))
|
||||
assertAllowed(Roles.operator, 'postMessage')
|
||||
assertAllowed(Roles.admin, 'clearMessages')
|
||||
test('assertAllowed denies viewer mutating ops', (t) => {
|
||||
t.exception(() => assertAllowed(Roles.viewer, 'runJob'))
|
||||
t.exception(() => assertAllowed(Roles.viewer, 'mintInvite'))
|
||||
assertAllowed(Roles.operator, 'runJob')
|
||||
assertAllowed(Roles.operator, 'setAlertConfig')
|
||||
assertAllowed(Roles.admin, 'mintInvite')
|
||||
t.pass()
|
||||
})
|
||||
|
||||
@@ -14,7 +16,9 @@ test('maxRole elevates', (t) => {
|
||||
t.is(maxRole(Roles.admin, Roles.operator), Roles.admin)
|
||||
})
|
||||
|
||||
test('viewer can read', (t) => {
|
||||
t.ok(roleAllows(Roles.viewer, MethodRoles.listMessages))
|
||||
test('viewer can read metrics', (t) => {
|
||||
t.ok(roleAllows(Roles.viewer, MethodRoles.queryData))
|
||||
t.ok(roleAllows(Roles.viewer, MethodRoles.listCharts))
|
||||
t.ok(roleAllows(Roles.viewer, MethodRoles.handshake))
|
||||
t.ok(roleAllows(Roles.viewer, MethodRoles.subscribeMetrics))
|
||||
})
|
||||
|
||||
+29
-25
@@ -1,10 +1,5 @@
|
||||
/**
|
||||
* End-to-end: ephemeral HyperDHT server + client RPC + push.
|
||||
* Runs against real DHT (local, no bootstrap dependency for same-process? )
|
||||
*
|
||||
* hyperdht connect needs the DHT network; same-machine servers work via
|
||||
* the default bootstrap / local discovery. May be slow on restricted networks.
|
||||
*
|
||||
* End-to-end: ephemeral HyperDHT agent + client RPC + metric push.
|
||||
* Skip with: SKIP_INTEGRATION=1 npm test
|
||||
*/
|
||||
import test from 'brittle'
|
||||
@@ -17,21 +12,22 @@ import { peers } from '../server/core/peer-registry.js'
|
||||
import { initAuthKeys } from '../server/core/auth-keys.js'
|
||||
import { PearDataConnection } from '../client/connection.js'
|
||||
import { Methods, Pushes } from '../shared/protocol.js'
|
||||
import { signCapability } from '../shared/crypto-auth.js'
|
||||
import { startPipeline } from '../server/pipeline.js'
|
||||
import { getCollector } from '../server/services/collector.js'
|
||||
|
||||
const skip = process.env.SKIP_INTEGRATION === '1'
|
||||
|
||||
test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, async (t) => {
|
||||
test('integration: dial, handshake, metrics, subscribe', { skip, timeout: 60_000 }, async (t) => {
|
||||
const seed = crypto.randomBytes(32)
|
||||
const keyPair = DHT.keyPair(seed)
|
||||
const publicKeyHex = b4a.toString(keyPair.publicKey, 'hex')
|
||||
const seedHex = b4a.toString(seed, 'hex')
|
||||
|
||||
initAuthKeys({ seedHex, publicKeyHex })
|
||||
|
||||
// Open admin for this test process
|
||||
process.env.PEARDATA_INSECURE_OPEN_ADMIN = '1'
|
||||
|
||||
startPipeline()
|
||||
|
||||
const dht = new DHT()
|
||||
const server = dht.createServer()
|
||||
|
||||
@@ -49,6 +45,7 @@ test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, asyn
|
||||
|
||||
await server.listen(keyPair)
|
||||
t.teardown(async () => {
|
||||
getCollector().stop()
|
||||
for (const s of peers.list()) s.destroy()
|
||||
await server.close().catch(() => {})
|
||||
await dht.destroy().catch(() => {})
|
||||
@@ -61,8 +58,8 @@ test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, asyn
|
||||
})
|
||||
|
||||
/** @type {object[]} */
|
||||
const pushes = []
|
||||
conn.on(Pushes.message, (m) => pushes.push(m))
|
||||
const metricPushes = []
|
||||
conn.on(Pushes.metrics, (m) => metricPushes.push(m))
|
||||
|
||||
await conn.connect()
|
||||
t.is(conn.role, 'admin')
|
||||
@@ -71,22 +68,29 @@ test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, asyn
|
||||
const pong = await conn.request(Methods.ping, {})
|
||||
t.ok(pong.ok)
|
||||
|
||||
await conn.request(Methods.setDisplayName, { name: 'tester' })
|
||||
const post = await conn.request(Methods.postMessage, { text: 'hello p2p' })
|
||||
t.ok(post.success)
|
||||
t.is(post.message.text, 'hello p2p')
|
||||
const info = await conn.request(Methods.getServerInfo, {})
|
||||
t.is(info.app, 'peardata')
|
||||
|
||||
// Allow push delivery
|
||||
await new Promise((r) => setTimeout(r, 200))
|
||||
const list = await conn.request(Methods.listMessages, {})
|
||||
t.ok(list.messages.some((m) => m.text === 'hello p2p'))
|
||||
const charts = await conn.request(Methods.listCharts, {})
|
||||
t.ok(charts.charts)
|
||||
|
||||
const invite = await conn.request(Methods.mintInvite, { role: 'operator', ttlMs: 3600_000 })
|
||||
t.ok(invite.invite.startsWith('pd1.'))
|
||||
await conn.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
||||
|
||||
// Capability can be verified offline
|
||||
const cap = signCapability(seedHex, { role: 'viewer', forever: true })
|
||||
t.ok(cap.token.includes('.'))
|
||||
// wait for at least one collector tick + push
|
||||
await new Promise((r) => setTimeout(r, 2200))
|
||||
|
||||
const q = await conn.request(Methods.queryData, {
|
||||
chart: 'system.cpu',
|
||||
after: -30,
|
||||
points: 30,
|
||||
})
|
||||
t.ok(q.labels)
|
||||
t.ok(Array.isArray(q.data))
|
||||
|
||||
const health = await conn.request(Methods.getHealth, {})
|
||||
t.ok(health.status)
|
||||
|
||||
t.ok(metricPushes.length >= 0) // push timing may vary; query proves pipeline
|
||||
|
||||
await conn.destroy()
|
||||
})
|
||||
|
||||
+43
-10
@@ -5,11 +5,14 @@ import {
|
||||
MethodRoles,
|
||||
PROTOCOL,
|
||||
PROTOCOL_VERSION,
|
||||
Methods,
|
||||
Pushes,
|
||||
} from '../shared/protocol.js'
|
||||
import { validateMethodArgs, SCHEMA_VERSION } from '../shared/schema.js'
|
||||
import { CHART_DEFS, CONTEXT_IDS } from '../shared/metrics.js'
|
||||
|
||||
test('protocol constants', (t) => {
|
||||
t.ok(PROTOCOL.includes('/rpc'))
|
||||
t.is(PROTOCOL, 'peardata/rpc')
|
||||
t.ok(PROTOCOL_VERSION >= 1)
|
||||
t.ok(SCHEMA_VERSION >= 1)
|
||||
})
|
||||
@@ -22,19 +25,49 @@ test('roleAllows hierarchy', (t) => {
|
||||
t.absent(roleAllows(Roles.viewer, Roles.admin))
|
||||
})
|
||||
|
||||
test('method roles map covers core methods', (t) => {
|
||||
for (const m of ['handshake', 'ping', 'listMessages', 'postMessage', 'mintInvite']) {
|
||||
test('method roles cover monitoring surface', (t) => {
|
||||
for (const m of [
|
||||
'handshake',
|
||||
'ping',
|
||||
'queryData',
|
||||
'subscribeMetrics',
|
||||
'listCharts',
|
||||
'mintInvite',
|
||||
'runJob',
|
||||
]) {
|
||||
t.ok(MethodRoles[m], m)
|
||||
t.is(Methods[m], m)
|
||||
}
|
||||
})
|
||||
|
||||
test('validateMethodArgs postMessage', (t) => {
|
||||
t.absent(validateMethodArgs('postMessage', {}).ok)
|
||||
t.ok(validateMethodArgs('postMessage', { text: 'hi' }).ok)
|
||||
t.absent(validateMethodArgs('postMessage', { text: 'x'.repeat(2001) }).ok)
|
||||
test('pushes include metrics + anomaly', (t) => {
|
||||
t.ok(Pushes.metrics.startsWith('push:'))
|
||||
t.ok(Pushes.anomaly.startsWith('push:'))
|
||||
})
|
||||
|
||||
test('validateMethodArgs setDisplayName', (t) => {
|
||||
t.ok(validateMethodArgs('setDisplayName', { name: 'Ada' }).ok)
|
||||
t.absent(validateMethodArgs('setDisplayName', { name: '' }).ok)
|
||||
test('metrics catalog non-empty', (t) => {
|
||||
t.ok(CHART_DEFS.length >= 5)
|
||||
t.ok(CONTEXT_IDS.includes('system.cpu'))
|
||||
})
|
||||
|
||||
test('validateMethodArgs queryData', (t) => {
|
||||
t.absent(validateMethodArgs('queryData', {}).ok)
|
||||
t.ok(validateMethodArgs('queryData', { chart: 'system.cpu' }).ok)
|
||||
t.absent(validateMethodArgs('queryData', { chart: 'system.cpu', points: 0 }).ok)
|
||||
})
|
||||
|
||||
test('validateMethodArgs subscribeMetrics', (t) => {
|
||||
const r = validateMethodArgs('subscribeMetrics', { charts: ['system.cpu'] })
|
||||
t.ok(r.ok)
|
||||
t.ok(r.args.intervalMs >= 500)
|
||||
})
|
||||
|
||||
test('validateMethodArgs mintInvite', (t) => {
|
||||
t.ok(validateMethodArgs('mintInvite', { role: 'operator' }).ok)
|
||||
t.absent(validateMethodArgs('mintInvite', { role: 'god' }).ok)
|
||||
})
|
||||
|
||||
test('validateMethodArgs revokePeer', (t) => {
|
||||
t.absent(validateMethodArgs('revokePeer', { peerId: 'abc' }).ok)
|
||||
t.ok(validateMethodArgs('revokePeer', { peerId: 'a'.repeat(64) }).ok)
|
||||
})
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
import test from 'brittle'
|
||||
import { handleRest } from '../server/rest/routes.js'
|
||||
import { getStore } from '../server/services/store.js'
|
||||
import { initAuthKeys } from '../server/core/auth-keys.js'
|
||||
import crypto from 'hypercore-crypto'
|
||||
import b4a from 'b4a'
|
||||
|
||||
// REST routes read public key helper — init dummy keys
|
||||
const seed = crypto.randomBytes(32)
|
||||
initAuthKeys({
|
||||
seedHex: b4a.toString(seed, 'hex'),
|
||||
publicKeyHex: b4a.toString(crypto.keyPair(seed).publicKey, 'hex'),
|
||||
})
|
||||
|
||||
test('REST /api/v3/info', (t) => {
|
||||
const res = handleRest('/api/v3/info', new URLSearchParams())
|
||||
t.is(res.status, 200)
|
||||
t.ok(res.body.hostname)
|
||||
t.ok(res.body.peardata)
|
||||
})
|
||||
|
||||
test('REST /api/v1/charts after ingest', (t) => {
|
||||
getStore().ingest([
|
||||
{
|
||||
chart: 'system.load',
|
||||
context: 'system.load',
|
||||
ts: Date.now(),
|
||||
values: { load1: 0.5, load5: 0.4, load15: 0.3 },
|
||||
},
|
||||
])
|
||||
const res = handleRest('/api/v1/charts', new URLSearchParams())
|
||||
t.is(res.status, 200)
|
||||
t.ok(res.body.charts['system.load'] || res.body.charts['system.cpu'])
|
||||
})
|
||||
|
||||
test('REST /api/v3/data', (t) => {
|
||||
const now = Date.now()
|
||||
for (let i = 0; i < 10; i++) {
|
||||
getStore().ingest([
|
||||
{
|
||||
chart: 'system.cpu',
|
||||
context: 'system.cpu',
|
||||
ts: now - (10 - i) * 1000,
|
||||
values: {
|
||||
user: i,
|
||||
system: 1,
|
||||
nice: 0,
|
||||
iowait: 0,
|
||||
irq: 0,
|
||||
softirq: 0,
|
||||
idle: 99 - i,
|
||||
},
|
||||
},
|
||||
])
|
||||
}
|
||||
const res = handleRest(
|
||||
'/api/v3/data',
|
||||
new URLSearchParams({ chart: 'system.cpu', after: '-30', points: '10' })
|
||||
)
|
||||
t.is(res.status, 200)
|
||||
t.ok(Array.isArray(res.body.data))
|
||||
})
|
||||
|
||||
test('REST /api/v3/contexts', (t) => {
|
||||
const res = handleRest('/api/v3/contexts', new URLSearchParams())
|
||||
t.is(res.status, 200)
|
||||
t.ok(res.body.contexts['system.cpu'])
|
||||
})
|
||||
|
||||
test('REST 404', (t) => {
|
||||
const res = handleRest('/api/v9/nope', new URLSearchParams())
|
||||
t.is(res.status, 404)
|
||||
})
|
||||
@@ -0,0 +1,41 @@
|
||||
import test from 'brittle'
|
||||
import { MetricStore } from '../server/services/store.js'
|
||||
|
||||
test('store ingest + query', (t) => {
|
||||
const store = new MetricStore()
|
||||
const now = Date.now()
|
||||
for (let i = 0; i < 30; i++) {
|
||||
store.ingest([
|
||||
{
|
||||
chart: 'system.cpu',
|
||||
context: 'system.cpu',
|
||||
ts: now - (30 - i) * 1000,
|
||||
values: {
|
||||
user: 10 + i,
|
||||
system: 5,
|
||||
nice: 0,
|
||||
iowait: 0,
|
||||
irq: 0,
|
||||
softirq: 0,
|
||||
idle: 85 - i,
|
||||
},
|
||||
},
|
||||
])
|
||||
}
|
||||
|
||||
const meta = store.getMeta('system.cpu')
|
||||
t.ok(meta)
|
||||
t.is(meta.context, 'system.cpu')
|
||||
|
||||
const q = store.query({ chart: 'system.cpu', after: -60, before: 0, points: 15 })
|
||||
t.absent(q.error)
|
||||
t.ok(q.data.length > 0)
|
||||
t.ok(q.labels.includes('user'))
|
||||
t.ok(q.labels.includes('time'))
|
||||
})
|
||||
|
||||
test('store unknown chart', (t) => {
|
||||
const store = new MetricStore()
|
||||
const q = store.query({ chart: 'nope.chart', points: 10 })
|
||||
t.ok(q.error)
|
||||
})
|
||||
+173
@@ -465,3 +465,176 @@ code {
|
||||
font-size: 11px;
|
||||
color: #b7c9f5;
|
||||
}
|
||||
|
||||
/* ─── PearData fleet dashboard ─── */
|
||||
#app {
|
||||
flex-direction: row;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.fleet-rail {
|
||||
width: 280px;
|
||||
flex: 0 0 280px;
|
||||
margin: 12px 0 12px 12px;
|
||||
overflow: auto;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
.fleet-rail h2 {
|
||||
margin-top: 0;
|
||||
}
|
||||
|
||||
.dash-main {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
overflow: auto;
|
||||
padding: 12px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.overview-strip {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(5, 1fr);
|
||||
gap: 12px;
|
||||
padding: 14px 16px;
|
||||
}
|
||||
|
||||
.stat {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
}
|
||||
|
||||
.stat-label {
|
||||
font-size: 11px;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.04em;
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
.stat strong {
|
||||
font-size: 22px;
|
||||
font-weight: 650;
|
||||
letter-spacing: -0.02em;
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.stat[data-health='ok'] strong {
|
||||
color: var(--ok);
|
||||
}
|
||||
.stat[data-health='degraded'] strong {
|
||||
color: #f0b429;
|
||||
}
|
||||
.stat[data-health='critical'] strong {
|
||||
color: var(--danger);
|
||||
}
|
||||
|
||||
.charts-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.chart-panel header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: baseline;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.chart-panel h3 {
|
||||
margin: 0;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.chart-panel canvas {
|
||||
width: 100%;
|
||||
display: block;
|
||||
border-radius: 8px;
|
||||
background: #0a1224;
|
||||
}
|
||||
|
||||
.bottom-row {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.bottom-row h3 {
|
||||
margin: 0 0 8px;
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.peer-list,
|
||||
.event-list {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
margin: 10px 0 0;
|
||||
}
|
||||
|
||||
.peer-list li,
|
||||
.event-list li {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
gap: 8px;
|
||||
padding: 8px 10px;
|
||||
border-radius: 8px;
|
||||
border: 1px solid transparent;
|
||||
font-size: 12px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.peer-list li:hover,
|
||||
.peer-list li.active {
|
||||
background: #0d1426;
|
||||
border-color: var(--border);
|
||||
}
|
||||
|
||||
.event-list li {
|
||||
cursor: default;
|
||||
flex-direction: column;
|
||||
align-items: flex-start;
|
||||
border-bottom: 1px solid rgba(36, 49, 84, 0.7);
|
||||
border-radius: 0;
|
||||
}
|
||||
|
||||
.event-list li.crit {
|
||||
color: var(--danger);
|
||||
}
|
||||
.event-list li.warn {
|
||||
color: #f0b429;
|
||||
}
|
||||
.event-list li.ok {
|
||||
color: var(--ok);
|
||||
}
|
||||
|
||||
.admin-block {
|
||||
margin-top: auto;
|
||||
padding-top: 12px;
|
||||
}
|
||||
|
||||
@media (max-width: 1100px) {
|
||||
.overview-strip {
|
||||
grid-template-columns: repeat(3, 1fr);
|
||||
}
|
||||
.charts-grid,
|
||||
.bottom-row {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 800px) {
|
||||
#app {
|
||||
flex-direction: column;
|
||||
}
|
||||
.fleet-rail {
|
||||
width: auto;
|
||||
flex: 0 0 auto;
|
||||
margin: 12px 12px 0;
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user