Updates
This commit is contained in:
+13
-20
@@ -1,45 +1,38 @@
|
|||||||
# ── Server identity ──────────────────────────────────────────
|
# ── Agent identity ───────────────────────────────────────────
|
||||||
# 32-byte seed as 64 hex chars. Auto-generated into .env on first boot if missing.
|
# 32-byte seed as 64 hex chars. Auto-generated into .env on first boot if missing.
|
||||||
# Treat SERVER_SEED like a root password (capabilities + admin proofs).
|
# Treat SERVER_SEED like a root password (capabilities + admin proofs).
|
||||||
# SERVER_SEED=
|
# SERVER_SEED=
|
||||||
# SERVER_PUBLIC_KEY=
|
# SERVER_PUBLIC_KEY=
|
||||||
|
|
||||||
# ── Roles ────────────────────────────────────────────────────
|
# ── Roles ────────────────────────────────────────────────────
|
||||||
# Baseline role for unknown peers: viewer | operator | admin
|
|
||||||
PEARDATA_DEFAULT_ROLE=viewer
|
PEARDATA_DEFAULT_ROLE=viewer
|
||||||
|
|
||||||
# Comma-separated peer public keys that always get admin
|
|
||||||
# PEARDATA_ADMIN_KEYS=
|
# PEARDATA_ADMIN_KEYS=
|
||||||
|
|
||||||
# DEV ONLY — every peer is admin (never enable in production)
|
|
||||||
# PEARDATA_INSECURE_OPEN_ADMIN=1
|
# PEARDATA_INSECURE_OPEN_ADMIN=1
|
||||||
|
|
||||||
# Optional allowlist (empty = all non-revoked peers accepted)
|
|
||||||
# PEARDATA_ALLOWLIST=
|
# PEARDATA_ALLOWLIST=
|
||||||
|
|
||||||
# ── Runtime paths & limits ───────────────────────────────────
|
# ── Runtime paths & limits ───────────────────────────────────
|
||||||
# Replaces OS home for client identity path:
|
|
||||||
# $PEARDATA_HOME/.config/peardata/identity.json
|
|
||||||
# PEARDATA_HOME=
|
# PEARDATA_HOME=
|
||||||
|
|
||||||
# Peer policy + audit.log (default ./data)
|
|
||||||
# PEARDATA_DATA_DIR=./data
|
# PEARDATA_DATA_DIR=./data
|
||||||
|
|
||||||
# Demo room ring buffer size
|
|
||||||
# PEARDATA_MAX_MESSAGES=500
|
|
||||||
|
|
||||||
# Per-peer RPC requests per minute
|
|
||||||
# PEARDATA_RATE_LIMIT_RPM=120
|
# PEARDATA_RATE_LIMIT_RPM=120
|
||||||
|
|
||||||
# Client ConnectionManager reconnect attempts
|
|
||||||
# PEARDATA_MAX_RECONNECT=20
|
# PEARDATA_MAX_RECONNECT=20
|
||||||
|
|
||||||
|
# ── Metrics pipeline ─────────────────────────────────────────
|
||||||
|
# PEARDATA_SAMPLE_MS=1000
|
||||||
|
# PEARDATA_TIER0_POINTS=3600
|
||||||
|
# PEARDATA_TIER1_POINTS=1440
|
||||||
|
# PEARDATA_TIER1_EVERY=60
|
||||||
|
|
||||||
|
# ── Netdata-style REST API ───────────────────────────────────
|
||||||
|
# PEARDATA_REST=1
|
||||||
|
# PEARDATA_REST_HOST=127.0.0.1
|
||||||
|
# PEARDATA_REST_PORT=19999
|
||||||
|
# PEARDATA_REST_CORS=*
|
||||||
|
|
||||||
# ── Logging ──────────────────────────────────────────────────
|
# ── Logging ──────────────────────────────────────────────────
|
||||||
# LOG_LEVEL=info
|
# LOG_LEVEL=info
|
||||||
# LOG_JSON=1
|
# LOG_JSON=1
|
||||||
|
|
||||||
# ── Healthcheck / soak ───────────────────────────────────────
|
# ── Healthcheck / soak ───────────────────────────────────────
|
||||||
# Remote dial key (falls back to SERVER_PUBLIC_KEY)
|
|
||||||
# PEARDATA_HEALTH_KEY=
|
# PEARDATA_HEALTH_KEY=
|
||||||
# HEALTHCHECK_TIMEOUT_MS=8000
|
# HEALTHCHECK_TIMEOUT_MS=8000
|
||||||
# SOAK_DURATION_MS=60000
|
# SOAK_DURATION_MS=60000
|
||||||
|
|||||||
@@ -61,6 +61,11 @@ jobs:
|
|||||||
LICENSE \
|
LICENSE \
|
||||||
docs/ARCHITECTURE.md \
|
docs/ARCHITECTURE.md \
|
||||||
docs/PROTOCOL.md \
|
docs/PROTOCOL.md \
|
||||||
|
docs/DATA-MODEL.md \
|
||||||
|
docs/REST-API.md \
|
||||||
|
docs/ROADMAP.md \
|
||||||
|
docs/TECH-CHOICES.md \
|
||||||
|
docs/STORAGE-HYPERDB.md \
|
||||||
docs/GETTING-STARTED.md \
|
docs/GETTING-STARTED.md \
|
||||||
docs/SECURITY.md \
|
docs/SECURITY.md \
|
||||||
docs/DESKTOP.md \
|
docs/DESKTOP.md \
|
||||||
|
|||||||
@@ -1,43 +1,45 @@
|
|||||||
# peardata
|
# PearData
|
||||||
|
|
||||||
**Production-oriented boilerplate for Holepunch / HyperDHT P2P apps.**
|
**Decentralized, P2P, Netdata-class real-time monitoring for the Pear / Holepunch ecosystem.**
|
||||||
|
|
||||||
Distilled from patterns used in [peardock](https://github.com/snxraven/peardock)-class apps (MIT template — not a copy of peardock’s AGPL sources):
|
Each machine runs a lightweight **PearMonitor agent**. A Pear desktop client dials agents by public key and shows fleet + per-node live dashboards. Agents also expose a **Netdata-compatible REST API** (`/api/v1`, `/api/v2`, `/api/v3`) for scripts, Grafana, and Prometheus.
|
||||||
|
|
||||||
|
Built from the [pear-app-template](https://github.com/snxraven) HyperDHT + protomux-rpc patterns used by PearDock-class apps (MIT template — not a copy of PearDock’s AGPL sources).
|
||||||
|
|
||||||
| Plane | Stack |
|
| Plane | Stack |
|
||||||
|-------|--------|
|
|-------|--------|
|
||||||
| Transport | **HyperDHT** secret streams (Noise) |
|
| Transport | **HyperDHT** secret streams (Noise) |
|
||||||
| RPC | **protomux-rpc** + compact-encoding JSON |
|
| RPC | **protomux-rpc** + compact-encoding JSON |
|
||||||
| Identity | Ed25519 keypairs (server seed + client identity file) |
|
| Metrics | 1s collector → tiered buffers → push + query |
|
||||||
| AuthZ | Roles (`viewer` / `operator` / `admin`) + HMAC capabilities + admin seed proof |
|
| AuthZ | Roles (`viewer` / `operator` / `admin`) + `pd1.` invites + admin seed |
|
||||||
| Desktop | **Pear** (`pear-electron` + `pear-bridge` + `<pear-ctrl>` titlebar) |
|
| HTTP | Netdata-style REST on `127.0.0.1:19999` |
|
||||||
| Server | Node 20+ (systemd unit included) |
|
| Desktop | **Pear** (`pear-electron` + `pear-bridge` + `<pear-ctrl>`) |
|
||||||
|
|
||||||
The demo product is **PearData**: a multi-peer room with messages, presence, and invite minting. Swap `server/handlers/demo.js` + `server/services/room.js` for your domain.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd pear_app_template # or your clone path
|
cd peardata
|
||||||
npm install
|
npm install
|
||||||
|
|
||||||
# Terminal A — server (prints public key)
|
# Terminal A — agent (prints public key + REST URL)
|
||||||
npm run start:server
|
npm run start:server
|
||||||
|
|
||||||
# Terminal B — mint an operator invite (optional)
|
# Terminal B — REST smoke test
|
||||||
|
curl -s http://127.0.0.1:19999/api/v3/info | jq
|
||||||
|
curl -s 'http://127.0.0.1:19999/api/v3/data?chart=system.cpu&after=-30&points=30' | jq
|
||||||
|
|
||||||
|
# Terminal C — mint an operator invite (optional)
|
||||||
npm run mint-invite -- operator
|
npm run mint-invite -- operator
|
||||||
|
|
||||||
# Terminal C — Pear desktop UI
|
# Terminal D — Pear desktop
|
||||||
npm start
|
npm start
|
||||||
# or: pear run -d .
|
# or: pear run -d .
|
||||||
```
|
```
|
||||||
|
|
||||||
In the UI, paste the **server public key** (viewer) or a **`pd1.` invite** (elevated role).
|
In the UI, paste the **agent public key** (viewer) or a **`pd1.` invite**.
|
||||||
Paste `SERVER_SEED` from `.env` into “Admin seed” for full admin without an invite.
|
Paste `SERVER_SEED` for admin without an invite.
|
||||||
|
|
||||||
Drag the **titlebar** to move the window; resize from the edges (`pear.gui.resizable`).
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm test
|
npm test
|
||||||
@@ -49,26 +51,20 @@ npm run healthcheck
|
|||||||
## Repository layout
|
## Repository layout
|
||||||
|
|
||||||
```
|
```
|
||||||
pear_app_template/
|
peardata/
|
||||||
├── app.js # Desktop UI logic
|
├── app.js / index.html / ui/ # Pear fleet dashboard
|
||||||
├── index.html / index.js # Pear shell (titlebar + pear-ctrl + drag)
|
├── shared/ # Protocol, metrics catalog, schema, crypto
|
||||||
├── ui/styles.css # Titlebar drag regions + layout
|
├── server/ # PearMonitor agent
|
||||||
├── shared/ # Protocol, encodings, schema, crypto-auth
|
│ ├── server.js # DHT + pipeline + REST boot
|
||||||
├── server/ # HyperDHT listener + RPC middleware + demo domain
|
│ ├── pipeline.js # collector → store → anomaly → push
|
||||||
│ ├── server.js
|
│ ├── handlers/monitor.js # Domain RPCs
|
||||||
│ ├── core/ # keys, acl, audit, peer-policy, registry
|
│ ├── services/ # collector, store, anomaly, jobs, …
|
||||||
│ ├── rpc/ # PeerSession + register
|
│ ├── rest/ # Netdata-compatible HTTP API
|
||||||
│ ├── handlers/ # Domain RPCs (replace demo.js)
|
│ └── core/ rpc/ utils/ # Auth, ACL, sessions (template)
|
||||||
│ ├── services/ # Domain state (replace room.js)
|
├── client/ # Multi-peer connection manager
|
||||||
│ └── utils/
|
├── docs/ # Full documentation set
|
||||||
├── client/ # Connection, manager, identity
|
├── deploy/peardata.service # systemd
|
||||||
├── bin/ # Server binary entry
|
└── test/
|
||||||
├── scripts/ # healthcheck, soak, mint-invite, rename, release
|
|
||||||
├── test/ # brittle unit + integration
|
|
||||||
├── docs/ # Full documentation set
|
|
||||||
├── deploy/ # systemd unit
|
|
||||||
├── .github/workflows/ # GitHub CI + release
|
|
||||||
└── .gitea/workflows/ # Gitea CI + rolling release skeleton
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -77,57 +73,23 @@ pear_app_template/
|
|||||||
|
|
||||||
| Script | Purpose |
|
| Script | Purpose |
|
||||||
|--------|---------|
|
|--------|---------|
|
||||||
| `npm start` / `npm run dev` | Pear desktop UI |
|
| `npm start` | Pear desktop UI |
|
||||||
| `npm run start:server` | HyperDHT server |
|
| `npm run start:server` | PearMonitor agent (P2P + REST) |
|
||||||
| `npm test` | brittle unit + integration |
|
| `npm test` | brittle unit + integration |
|
||||||
| `npm run mint-invite -- [role] [ttlMs]` | Offline `pd1.` invite |
|
| `npm run mint-invite -- [role]` | Offline `pd1.` invite |
|
||||||
| `npm run healthcheck` | Liveness / remote ping |
|
| `npm run healthcheck` | Liveness / remote ping |
|
||||||
| `npm run soak` | Load exercise (needs env keys) |
|
| `npm run soak` | Load exercise |
|
||||||
| `npm run rename -- <slug> <Product>` | Rebrand the tree |
|
|
||||||
| `bash scripts/release.sh` | Source tarball + checksum |
|
|
||||||
|
|
||||||
Full env reference: [docs/CONFIGURATION.md](./docs/CONFIGURATION.md).
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Rebrand for a new app
|
## Auth model
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run rename -- my-app MyApp
|
|
||||||
# → package name, protocol id, env prefixes, invite prefix, product strings
|
|
||||||
```
|
|
||||||
|
|
||||||
Then implement your domain:
|
|
||||||
|
|
||||||
1. Extend `shared/protocol.js` (`MethodRoles`, `Pushes`, `Methods`)
|
|
||||||
2. Validate args in `shared/schema.js`
|
|
||||||
3. Add handlers under `server/handlers/`
|
|
||||||
4. Register them in `server/rpc/register.js`
|
|
||||||
5. Call from `client/` + UI
|
|
||||||
6. Update `docs/PROTOCOL.md` and tests
|
|
||||||
|
|
||||||
See [docs/EXTENDING.md](./docs/EXTENDING.md).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Auth model (secure defaults)
|
|
||||||
|
|
||||||
| Mode | How | Role |
|
| Mode | How | Role |
|
||||||
|------|-----|------|
|
|------|-----|------|
|
||||||
| Viewer | Dial public key only | `viewer` (read) |
|
| Viewer | Dial public key only | `viewer` (read + subscribe) |
|
||||||
| Capability | `pd1.` invite or raw HMAC token | grant role |
|
| Capability | `pd1.` invite or HMAC token | granted role |
|
||||||
| Admin seed | HMAC proof from `SERVER_SEED` | `admin` |
|
| Admin seed | HMAC proof from `SERVER_SEED` | `admin` |
|
||||||
| Allowlist | `PEARDATA_ADMIN_KEYS` | admin for listed peers |
|
| Allowlist | `PEARDATA_ADMIN_KEYS` | admin for listed peers |
|
||||||
| Dev escape | `PEARDATA_INSECURE_OPEN_ADMIN=1` | everyone admin |
|
|
||||||
|
|
||||||
See [docs/SECURITY.md](./docs/SECURITY.md).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## CI
|
|
||||||
|
|
||||||
- **GitHub**: `.github/workflows/ci.yml` (Node 20/22 matrix + docs presence), `release.yml` on `v*` tags
|
|
||||||
- **Gitea**: `.gitea/workflows/ci.yml`, `release-rolling.yml` (every push to `main` → `rolling` release; needs `RELEASE_TOKEN`)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -135,20 +97,19 @@ See [docs/SECURITY.md](./docs/SECURITY.md).
|
|||||||
|
|
||||||
| Doc | Contents |
|
| Doc | Contents |
|
||||||
|-----|----------|
|
|-----|----------|
|
||||||
| [Getting started](./docs/GETTING-STARTED.md) | Install, run, connect, systemd, troubleshooting |
|
| [Getting started](./docs/GETTING-STARTED.md) | Install, agent, REST, desktop, systemd |
|
||||||
| [Desktop](./docs/DESKTOP.md) | Pear shell, `pear-ctrl`, drag/resize, identity |
|
| [Roadmap](./docs/ROADMAP.md) | MVP → advanced phases |
|
||||||
| [Architecture](./docs/ARCHITECTURE.md) | Planes, boot, session pipeline, module map |
|
| [Architecture](./docs/ARCHITECTURE.md) | PearDock mapping, planes, modules |
|
||||||
| [Protocol](./docs/PROTOCOL.md) | Methods, pushes, errors, versioning |
|
| [Protocol](./docs/PROTOCOL.md) | RPC methods & pushes |
|
||||||
| [Security](./docs/SECURITY.md) | Threat model, secrets, hardening checklist |
|
| [Data model](./docs/DATA-MODEL.md) | Metrics, anomalies, health |
|
||||||
| [Configuration](./docs/CONFIGURATION.md) | Full environment + scripts reference |
|
| [REST API](./docs/REST-API.md) | `/api/v1\|v2\|v3` |
|
||||||
| [Testing](./docs/TESTING.md) | brittle suite, soak, manual checks |
|
| [Tech choices](./docs/TECH-CHOICES.md) | Collector, charts, libraries |
|
||||||
| [CI](./docs/CI.md) | Pipelines and required docs |
|
| [Security](./docs/SECURITY.md) | Threat model & hardening |
|
||||||
| [Release](./docs/RELEASE.md) | Version, tag, tarball, rollback |
|
| [Configuration](./docs/CONFIGURATION.md) | Environment reference |
|
||||||
| [Extending](./docs/EXTENDING.md) | Grow past the demo room |
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT — use this as a starting point for proprietary or open apps.
|
MIT — Pear ecosystem tooling starter.
|
||||||
(Peardock itself is AGPL; this template does **not** copy peardock source verbatim and is intentionally MIT.)
|
(Peardock itself is AGPL; this project does **not** copy peardock source verbatim.)
|
||||||
|
|||||||
@@ -112,7 +112,7 @@ function drawChart(canvasId, values, color = '#5b8cff') {
|
|||||||
}
|
}
|
||||||
|
|
||||||
function renderPeers() {
|
function renderPeers() {
|
||||||
const list = manager.list?.() || []
|
const list = manager.list() || []
|
||||||
els.peerList.innerHTML = ''
|
els.peerList.innerHTML = ''
|
||||||
if (!list.length) {
|
if (!list.length) {
|
||||||
const li = document.createElement('li')
|
const li = document.createElement('li')
|
||||||
@@ -123,11 +123,12 @@ function renderPeers() {
|
|||||||
}
|
}
|
||||||
for (const p of list) {
|
for (const p of list) {
|
||||||
const li = document.createElement('li')
|
const li = document.createElement('li')
|
||||||
const active = manager.activeId === p.id
|
const id = p.publicKeyHex || p.id
|
||||||
|
const active = manager.active?.publicKeyHex === id
|
||||||
li.className = active ? 'active' : ''
|
li.className = active ? 'active' : ''
|
||||||
li.innerHTML = `<span>${escapeHtml(p.id)}</span><span class="muted">${p.connected ? 'live' : '…'}</span>`
|
li.innerHTML = `<span>${escapeHtml(String(id).slice(0, 12))}…</span><span class="muted">${p.connected ? 'live' : '…'}</span>`
|
||||||
li.addEventListener('click', () => {
|
li.addEventListener('click', () => {
|
||||||
manager.setActive?.(p.id)
|
manager.setActive(id)
|
||||||
renderPeers()
|
renderPeers()
|
||||||
})
|
})
|
||||||
els.peerList.appendChild(li)
|
els.peerList.appendChild(li)
|
||||||
@@ -189,18 +190,6 @@ async function refreshMeta() {
|
|||||||
renderPeers()
|
renderPeers()
|
||||||
}
|
}
|
||||||
|
|
||||||
function parseConnectInput(raw) {
|
|
||||||
const input = raw.trim()
|
|
||||||
if (input.startsWith('pd1.')) {
|
|
||||||
const inv = decodeInvite(input)
|
|
||||||
return {
|
|
||||||
publicKeyHex: inv.publicKeyHex,
|
|
||||||
capability: inv.capability,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return { publicKeyHex: input.toLowerCase(), capability: null }
|
|
||||||
}
|
|
||||||
|
|
||||||
els.btnConnect.addEventListener('click', async () => {
|
els.btnConnect.addEventListener('click', async () => {
|
||||||
const raw = els.connectInput.value.trim()
|
const raw = els.connectInput.value.trim()
|
||||||
const adminSeed = els.adminSeed.value.trim() || null
|
const adminSeed = els.adminSeed.value.trim() || null
|
||||||
@@ -210,9 +199,9 @@ els.btnConnect.addEventListener('click', async () => {
|
|||||||
}
|
}
|
||||||
els.btnConnect.disabled = true
|
els.btnConnect.disabled = true
|
||||||
try {
|
try {
|
||||||
const { publicKeyHex, capability } = parseConnectInput(raw)
|
log(`Dialing…`)
|
||||||
log(`Dialing ${publicKeyHex.slice(0, 16)}…`)
|
const conn = await manager.connect(raw, { adminSeed })
|
||||||
await manager.connect(publicKeyHex, { capability, adminSeed })
|
log(`Connected ${conn.publicKeyHex.slice(0, 16)}…`)
|
||||||
setOnline(true)
|
setOnline(true)
|
||||||
await manager.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
await manager.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
||||||
await manager.request(Methods.subscribeAnomalies, {})
|
await manager.request(Methods.subscribeAnomalies, {})
|
||||||
@@ -250,7 +239,7 @@ els.btnConnect.addEventListener('click', async () => {
|
|||||||
})
|
})
|
||||||
|
|
||||||
els.btnDisconnect.addEventListener('click', async () => {
|
els.btnDisconnect.addEventListener('click', async () => {
|
||||||
await manager.disconnectAll?.()
|
await manager.disconnect()
|
||||||
setOnline(false)
|
setOnline(false)
|
||||||
renderPeers()
|
renderPeers()
|
||||||
log('Disconnected')
|
log('Disconnected')
|
||||||
@@ -267,7 +256,7 @@ els.btnInvite.addEventListener('click', async () => {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
manager.on?.('push', (ev) => {
|
manager.on('push', (ev) => {
|
||||||
if (ev.type === Pushes.metrics) onSamples(ev.data?.samples)
|
if (ev.type === Pushes.metrics) onSamples(ev.data?.samples)
|
||||||
if (ev.type === Pushes.anomaly) {
|
if (ev.type === Pushes.anomaly) {
|
||||||
prependAnomaly(ev.data)
|
prependAnomaly(ev.data)
|
||||||
@@ -278,17 +267,15 @@ manager.on?.('push', (ev) => {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
// Compatibility if manager emits per-connection
|
manager.on('connected', () => {
|
||||||
manager.on?.('connection', (conn) => {
|
setOnline(true)
|
||||||
conn.on?.(Pushes.metrics, (data) => onSamples(data?.samples))
|
renderPeers()
|
||||||
conn.on?.(Pushes.anomaly, (data) => prependAnomaly(data))
|
})
|
||||||
conn.on?.(Pushes.health, (data) => {
|
|
||||||
els.statHealth.textContent = data?.status || '—'
|
manager.on('disconnected', () => {
|
||||||
})
|
if (!manager.active?.connected) setOnline(false)
|
||||||
conn.on?.('disconnected', () => {
|
renderPeers()
|
||||||
setOnline(false)
|
log('Agent disconnected')
|
||||||
log('Agent disconnected')
|
|
||||||
})
|
|
||||||
})
|
})
|
||||||
|
|
||||||
setOnline(false)
|
setOnline(false)
|
||||||
|
|||||||
+1
-1
@@ -20,7 +20,7 @@ export class ConnectionManager extends EventEmitter {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @param {string} input - public key, pa1 invite, or capability+key object fields
|
* @param {string} input - public key, pd1 invite, or capability+key object fields
|
||||||
* @param {{ adminSeed?: string, alias?: string, autoReconnect?: boolean }} [opts]
|
* @param {{ adminSeed?: string, alias?: string, autoReconnect?: boolean }} [opts]
|
||||||
*/
|
*/
|
||||||
async connect(input, opts = {}) {
|
async connect(input, opts = {}) {
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
[Unit]
|
[Unit]
|
||||||
Description=Pear App P2P server (HyperDHT + protomux-rpc)
|
Description=PearData PearMonitor agent (HyperDHT + REST metrics)
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
|
|||||||
+123
-151
@@ -1,193 +1,165 @@
|
|||||||
# Architecture
|
# Architecture
|
||||||
|
|
||||||
|
PearData is a **decentralized, P2P clone of the Netdata real-time monitoring experience**, built on the same HyperDHT + protomux-rpc patterns as PearDock-class apps (via the pear-app template).
|
||||||
|
|
||||||
## Design goals
|
## Design goals
|
||||||
|
|
||||||
1. **No central control plane** — peers dial a public key, not a SaaS tenant.
|
1. **No central control plane** — dial an agent by Ed25519 public key.
|
||||||
2. **Cryptographic identity** — HyperDHT Noise streams authenticate both ends.
|
2. **Instant live truth** — ~1s metric push to connected desktops.
|
||||||
3. **Clear AuthZ** — roles, rate limits, audit, optional allowlist / revoke.
|
3. **Dual API** — P2P RPC for the Pear client; Netdata-style REST for scripts/Grafana.
|
||||||
4. **Shared wire contract** — `shared/*` is the single source of truth for client + server.
|
4. **Clear AuthZ** — viewer (pubkey) vs operator/admin (`pd1.` invite / seed proof).
|
||||||
5. **Replaceable domain** — demo room is a thin layer over the session stack.
|
5. **Low agent overhead** — Node/`os` + `/proc` collectors, ring buffers, hot-path RPCs.
|
||||||
6. **Pear-native desktop** — `pear-electron` shell with `<pear-ctrl>` window chrome.
|
6. **Ecosystem-ready** — shared wire contract in `shared/` for PearDock / PearVirt adapters later.
|
||||||
|
|
||||||
|
## Mapping to PearDock / template components
|
||||||
|
|
||||||
|
| PearDock-class concept | PearData component |
|
||||||
|
|------------------------|--------------------|
|
||||||
|
| HyperDHT secret stream | `server/server.js` + `client/connection.js` |
|
||||||
|
| protomux-rpc methods | `shared/protocol.js` + `server/handlers/monitor.js` |
|
||||||
|
| Capability invites (`pd1.`) | `shared/crypto-auth.js` |
|
||||||
|
| Roles viewer/operator/admin | `shared/protocol.js` `MethodRoles` + `server/core/acl.js` |
|
||||||
|
| Peer policy / revoke | `server/core/peer-policy.js` |
|
||||||
|
| Connection manager / multi-peer | `client/manager.js` |
|
||||||
|
| Job tray | `server/services/jobs.js` + desktop actions |
|
||||||
|
| Domain service | **Metrics pipeline** (`collector` → `store` → `anomaly` → pushes) |
|
||||||
|
| Optional HTTP surface | `server/rest/*` (Netdata v1/v2/v3) |
|
||||||
|
|
||||||
## System context
|
## System context
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart LR
|
flowchart LR
|
||||||
UI[Pear desktop / scripts] -->|HyperDHT Noise| SRV[Node server]
|
UI[PearData desktop] -->|HyperDHT Noise + RPC| AG[PearMonitor agent]
|
||||||
SRV --> STATE[Room / your domain]
|
SCRIPTS[curl / Grafana / Prometheus scrapers] -->|HTTP REST :19999| AG
|
||||||
UI -.->|bootstrap / punch| NET[HyperDHT network]
|
AG --> COL[Collector 1s]
|
||||||
SRV -.-> NET
|
COL --> STORE[Tiered ring buffers]
|
||||||
```
|
COL --> ANO[Anomaly engine]
|
||||||
|
AG -.->|bootstrap / punch| DHT[HyperDHT]
|
||||||
## Layered stack
|
UI -.-> DHT
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TB
|
|
||||||
subgraph Presentation
|
|
||||||
HTML[index.html + app.js + ui/styles.css]
|
|
||||||
PEAR[index.js pear-electron + bridge]
|
|
||||||
end
|
|
||||||
subgraph ClientCore
|
|
||||||
MGR[client/manager.js]
|
|
||||||
CON[client/connection.js]
|
|
||||||
ID[client/identity.js]
|
|
||||||
end
|
|
||||||
subgraph Wire
|
|
||||||
PROT[shared/protocol.js]
|
|
||||||
ENC[shared/encodings.js]
|
|
||||||
AUTH[shared/crypto-auth.js]
|
|
||||||
SCH[shared/schema.js]
|
|
||||||
end
|
|
||||||
subgraph ServerCore
|
|
||||||
BOOT[server/server.js]
|
|
||||||
SESS[server/rpc/session.js]
|
|
||||||
ACL[server/core/acl.js]
|
|
||||||
HAND[server/handlers/*]
|
|
||||||
SVC[server/services/*]
|
|
||||||
end
|
|
||||||
PEAR --> HTML
|
|
||||||
HTML --> MGR --> CON
|
|
||||||
CON --> PROT
|
|
||||||
CON --> AUTH
|
|
||||||
SESS --> PROT
|
|
||||||
SESS --> ACL
|
|
||||||
SESS --> SCH
|
|
||||||
HAND --> SVC
|
|
||||||
BOOT --> SESS
|
|
||||||
CON <-->|secret stream| SESS
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Process model
|
## Process model
|
||||||
|
|
||||||
| Process | Entry | Responsibility |
|
| Process | Entry | Responsibility |
|
||||||
|---------|-------|----------------|
|
|---------|-------|----------------|
|
||||||
| **Server** | `server/server.js` or `bin/peardata-server.mjs` | HyperDHT listen, RPC, domain state |
|
| **Agent** | `server/server.js` / `bin/peardata-server.mjs` | Collect, store, P2P RPC, optional REST |
|
||||||
| **Desktop** | `index.js` → Pear Runtime | Window + HTML UI; dials servers as a client |
|
| **Desktop** | `index.js` → Pear Runtime | Fleet UI, multi-peer dial, live charts |
|
||||||
| **Scripts** | `scripts/*` | mint-invite, healthcheck, soak (use client stack) |
|
| **Scripts** | `scripts/*` | mint-invite, healthcheck, soak |
|
||||||
|
|
||||||
Server and desktop are **independent**. You can run many clients against one server, or headless scripts with no UI.
|
Many desktops may dial one agent; one desktop may dial many agents.
|
||||||
|
|
||||||
## Server boot
|
## Planes
|
||||||
|
|
||||||
1. `loadOrCreateKeyPair()` → persist `SERVER_SEED` / `SERVER_PUBLIC_KEY` in `.env`
|
### 1. Control / metadata (RPC)
|
||||||
2. `initAuthKeys()` → HKDF MAC key for capabilities
|
|
||||||
3. `loadPeerPolicy()` → roles / revocations / spent JTIs from `PEARDATA_DATA_DIR`
|
|
||||||
4. `dht.createServer().listen(keyPair)`
|
|
||||||
5. On connection → revoke check → `PeerSession` → `registerAllHandlers` → peer registry
|
|
||||||
6. Banner logs public key + secure/insecure mode
|
|
||||||
7. `graceful-goodbye` / SIGINT / SIGTERM drain peers and destroy DHT
|
|
||||||
|
|
||||||
## Session middleware (every non-hot RPC)
|
Handshake, node info, chart/context catalog, alert config, invites, jobs, ACL.
|
||||||
|
|
||||||
|
### 2. High-frequency metrics
|
||||||
|
|
||||||
|
- **Ingest:** collector emits sample batches every `PEARDATA_SAMPLE_MS` (default 1000).
|
||||||
|
- **Store:** tier0 (1s) + tier1 (downsampled averages).
|
||||||
|
- **Push:** `push:metrics` to subscribed peers (protomux-rpc events).
|
||||||
|
- **Pull:** `queryData` / REST `/api/v3/data` for history windows.
|
||||||
|
|
||||||
|
### 3. Anomaly / health
|
||||||
|
|
||||||
|
Threshold engine evaluates each batch; transitions emit `push:anomaly` / `push:alert`; health snapshot on an interval via `push:health`.
|
||||||
|
|
||||||
|
### 4. Local REST (optional)
|
||||||
|
|
||||||
|
Netdata-compatible HTTP on `127.0.0.1:19999` by default — for local tooling without P2P. Disable with `PEARDATA_REST=0`.
|
||||||
|
|
||||||
|
## Layered stack
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart TD
|
flowchart TB
|
||||||
IN[method + args] --> RL{Rate limit}
|
subgraph Presentation
|
||||||
RL -->|deny| E1[RATE_LIMIT_EXCEEDED]
|
HTML[index.html + app.js]
|
||||||
RL -->|ok| ACL{roleAllows MethodRoles}
|
PEAR[index.js pear-electron]
|
||||||
ACL -->|deny| E2[PERMISSION_DENIED + audit]
|
end
|
||||||
ACL -->|ok| VAL{validateMethodArgs}
|
subgraph ClientCore
|
||||||
VAL -->|fail| E3[INVALID_ARGS]
|
MGR[client/manager.js]
|
||||||
VAL -->|ok| H[Handler]
|
CON[client/connection.js]
|
||||||
H --> OK[Result + optional audit]
|
end
|
||||||
|
subgraph Wire
|
||||||
|
PROT[shared/protocol.js]
|
||||||
|
MET[shared/metrics.js]
|
||||||
|
SCH[shared/schema.js]
|
||||||
|
AUTH[shared/crypto-auth.js]
|
||||||
|
end
|
||||||
|
subgraph Agent
|
||||||
|
BOOT[server/server.js]
|
||||||
|
PIPE[server/pipeline.js]
|
||||||
|
COL[collector]
|
||||||
|
STORE[store]
|
||||||
|
ANO[anomaly]
|
||||||
|
HAND[handlers/monitor.js]
|
||||||
|
REST[rest/http-server.js]
|
||||||
|
end
|
||||||
|
PEAR --> HTML --> MGR --> CON
|
||||||
|
CON <-->|Noise + protomux-rpc| HAND
|
||||||
|
BOOT --> PIPE --> COL --> STORE
|
||||||
|
COL --> ANO
|
||||||
|
REST --> STORE
|
||||||
|
HAND --> STORE
|
||||||
```
|
```
|
||||||
|
|
||||||
**Hot path** (`session.respond(method, handler, { hot: true })` or stream method names):
|
## Agent boot sequence
|
||||||
|
|
||||||
- Still rate-limited and ACL-checked
|
1. Load/create `SERVER_SEED` / public key → `initAuthKeys`
|
||||||
- Skips full schema validation / success audit (for high-frequency streams)
|
2. Load peer policy from `PEARDATA_DATA_DIR`
|
||||||
|
3. `startPipeline()` — collector + store + anomaly fan-out
|
||||||
|
4. `startRestServer()` — unless disabled
|
||||||
|
5. HyperDHT `createServer().listen(keyPair)`
|
||||||
|
6. On connection → revoke check → `PeerSession` → register monitor handlers
|
||||||
|
7. Banner prints pubkey + REST URL
|
||||||
|
|
||||||
## Client connection states
|
## Session middleware
|
||||||
|
|
||||||
```
|
Same as the template: rate limit → ACL (`MethodRoles`) → schema validate → handler.
|
||||||
idle → dialing → handshaking → ready
|
Hot methods (`queryData`, `subscribeMetrics`, `ping`, …) skip heavy audit.
|
||||||
↘ closed → (manager reconnect timer)
|
|
||||||
```
|
|
||||||
|
|
||||||
| State | Meaning |
|
|
||||||
|-------|---------|
|
|
||||||
| `idle` | Constructed, not dialing |
|
|
||||||
| `dialing` | `dht.connect(serverPk)` in flight |
|
|
||||||
| `handshaking` | Stream open; `handshake` RPC |
|
|
||||||
| `ready` | Authenticated; RPCs allowed |
|
|
||||||
| `closed` | Torn down |
|
|
||||||
|
|
||||||
`ConnectionManager` tracks multiple peers, active selection, and reconnect (max `PEARDATA_MAX_RECONNECT`).
|
|
||||||
|
|
||||||
## Identity planes
|
## Identity planes
|
||||||
|
|
||||||
| Plane | Storage | Purpose |
|
| Plane | Storage | Purpose |
|
||||||
|-------|---------|---------|
|
|-------|---------|---------|
|
||||||
| **Server keypair** | `.env` (`SERVER_SEED`) | DHT listen address + HMAC root |
|
| Agent keypair | `.env` `SERVER_SEED` | DHT address + HMAC root |
|
||||||
| **Client keypair** | `~/.config/peardata/identity.json` | Stable peerId for AuthZ / revoke |
|
| Client keypair | `~/.config/peardata/identity.json` | Stable peerId |
|
||||||
| **Capabilities** | Issued as tokens / `pd1.` invites | Role grants with optional expiry & peer bind |
|
| Capabilities | `pd1.` invites / raw tokens | Role grants |
|
||||||
| **Peer policy** | `data/peer-policy.json` | Registered roles, revocations, spent JTIs |
|
| Peer policy | `data/peer-policy.json` | Roles, revokes, spent JTIs |
|
||||||
| **Audit** | `data/audit.log` | Mutating RPC trail |
|
| Audit | `data/audit.log` | Mutating RPC trail |
|
||||||
|
|
||||||
|
## Parent peer (future)
|
||||||
|
|
||||||
|
A heavier agent may subscribe to child agents over P2P, downsample into its own store, and expose fleet REST — still no central SaaS. Design leaves room via `stream_path` REST stub and multi-peer desktop manager.
|
||||||
|
|
||||||
## Module map
|
## Module map
|
||||||
|
|
||||||
### `shared/`
|
### Keep from template
|
||||||
|
|
||||||
| File | Role |
|
`server/core/*`, `server/rpc/session.js`, `client/*`, `shared/crypto-auth.js`, `shared/encodings.js`, Pear titlebar patterns.
|
||||||
|------|------|
|
|
||||||
| `protocol.js` | `PROTOCOL`, roles, `MethodRoles`, `Methods`, `Pushes` |
|
|
||||||
| `encodings.js` | compact-encoding JSON for protomux-rpc |
|
|
||||||
| `crypto-auth.js` | MAC key, capabilities, admin proof, invites |
|
|
||||||
| `schema.js` | Lightweight request validation |
|
|
||||||
|
|
||||||
### `server/`
|
### PearData domain
|
||||||
|
|
||||||
| Path | Role |
|
| Path | Role |
|
||||||
|------|------|
|
|------|------|
|
||||||
| `server.js` | Boot + DHT accept loop |
|
| `shared/metrics.js` | Chart/context catalog |
|
||||||
| `core/keys.js` | Seed load / generate |
|
| `shared/data-model.js` | Typed shapes |
|
||||||
| `core/auth-keys.js` | Process-wide MAC key |
|
| `server/services/collector.js` | System sampling |
|
||||||
| `core/acl.js` | Role resolution + assert |
|
| `server/services/store.js` | Tiered buffers + query |
|
||||||
| `core/peer-policy.js` | File-backed policy |
|
| `server/services/anomaly.js` | Thresholds |
|
||||||
| `core/peer-registry.js` | Live sessions |
|
| `server/services/alerts.js` | Alert CRUD helpers |
|
||||||
| `core/audit.js` | Audit log writer |
|
| `server/services/subscriptions.js` | Push fan-out |
|
||||||
| `rpc/session.js` | ProtomuxRPC + middleware |
|
| `server/services/jobs.js` | On-demand jobs |
|
||||||
| `rpc/register.js` | Wire handlers per session |
|
| `server/handlers/monitor.js` | RPC surface |
|
||||||
| `handlers/demo.js` | **Replace** — domain RPCs |
|
| `server/rest/*` | Netdata HTTP API |
|
||||||
| `services/room.js` | **Replace** — domain state |
|
| `server/pipeline.js` | Wire collector→store→push |
|
||||||
| `utils/logger.js` | Structured / pretty logs |
|
|
||||||
| `utils/rateLimiter.js` | Per-peer RPM |
|
|
||||||
|
|
||||||
### `client/`
|
|
||||||
|
|
||||||
| File | Role |
|
|
||||||
|------|------|
|
|
||||||
| `identity.js` | Persistent client seed |
|
|
||||||
| `connection.js` | Single peer RPC client |
|
|
||||||
| `manager.js` | Multi-peer + reconnect |
|
|
||||||
| `errors.js` | Error normalization |
|
|
||||||
| `index.js` | Public re-exports |
|
|
||||||
|
|
||||||
### Desktop shell
|
|
||||||
|
|
||||||
| File | Role |
|
|
||||||
|------|------|
|
|
||||||
| `index.js` | Pear Runtime + Bridge |
|
|
||||||
| `index.html` | Titlebar (`pear-ctrl`) + layout |
|
|
||||||
| `app.js` | UI → manager |
|
|
||||||
| `ui/styles.css` | Drag regions + theme |
|
|
||||||
|
|
||||||
See [DESKTOP.md](./DESKTOP.md).
|
|
||||||
|
|
||||||
## What to keep vs replace
|
|
||||||
|
|
||||||
| Keep | Replace when productizing |
|
|
||||||
|------|---------------------------|
|
|
||||||
| `shared/*` wire + crypto | Method names / schema for your domain |
|
|
||||||
| `server/rpc/session.js` | Rarely — middleware is generic |
|
|
||||||
| `server/core/*` | Peer policy storage backend if needed |
|
|
||||||
| `client/connection.js` + `manager.js` | UI-specific multi-peer UX |
|
|
||||||
| Titlebar / `pear-ctrl` patterns | Visual design only — keep drag + controls |
|
|
||||||
| `server/handlers/demo.js` + `services/room.js` | **Your product** |
|
|
||||||
|
|
||||||
## Related docs
|
## Related docs
|
||||||
|
|
||||||
- [PROTOCOL.md](./PROTOCOL.md)
|
- [PROTOCOL.md](./PROTOCOL.md) — RPC methods & pushes
|
||||||
- [SECURITY.md](./SECURITY.md)
|
- [DATA-MODEL.md](./DATA-MODEL.md) — metrics / anomalies / health
|
||||||
- [DESKTOP.md](./DESKTOP.md)
|
- [REST-API.md](./REST-API.md) — `/api/v1|v2|v3`
|
||||||
- [CONFIGURATION.md](./CONFIGURATION.md)
|
- [TECH-CHOICES.md](./TECH-CHOICES.md) — collector & charts
|
||||||
- [EXTENDING.md](./EXTENDING.md)
|
- [ROADMAP.md](./ROADMAP.md) — phases
|
||||||
|
- [SECURITY.md](./SECURITY.md) — threat model
|
||||||
|
|||||||
+1
-1
@@ -20,7 +20,7 @@
|
|||||||
|
|
||||||
**lint-docs**
|
**lint-docs**
|
||||||
|
|
||||||
- Asserts required documentation files exist (README, architecture, protocol, getting started, security, desktop, configuration, LICENSE, etc.)
|
- Asserts required documentation files exist (README, architecture, protocol, data model, REST API, roadmap, tech choices, getting started, security, desktop, configuration, LICENSE, etc.)
|
||||||
|
|
||||||
### Release job details (`release.yml`)
|
### Release job details (`release.yml`)
|
||||||
|
|
||||||
|
|||||||
+54
-84
@@ -1,6 +1,6 @@
|
|||||||
# Configuration reference
|
# Configuration reference
|
||||||
|
|
||||||
All knobs can be set via environment variables. The server loads `.env` through `dotenv` on boot (`server/core/keys.js`). Copy `.env.example` to get started.
|
All knobs can be set via environment variables. The agent loads `.env` through `dotenv` on boot (`server/core/keys.js`). Copy `.env.example` to get started.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
@@ -8,7 +8,7 @@ cp .env.example .env
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Server identity
|
## Agent identity
|
||||||
|
|
||||||
| Variable | Default | Description |
|
| Variable | Default | Description |
|
||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
@@ -16,8 +16,6 @@ cp .env.example .env
|
|||||||
| `SERVER_PUBLIC_KEY` | *derived* | 32-byte public key as **64 hex**. Clients dial this. Auto-synced to `.env` when seed loads. |
|
| `SERVER_PUBLIC_KEY` | *derived* | 32-byte public key as **64 hex**. Clients dial this. Auto-synced to `.env` when seed loads. |
|
||||||
| `SERVER_KEY` | — | Alias accepted for `SERVER_SEED` (legacy). Prefer `SERVER_SEED`. |
|
| `SERVER_KEY` | — | Alias accepted for `SERVER_SEED` (legacy). Prefer `SERVER_SEED`. |
|
||||||
|
|
||||||
On first boot without a seed, the server appends both values to `.env`.
|
|
||||||
|
|
||||||
Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -26,10 +24,10 @@ Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
|||||||
|
|
||||||
| Variable | Default | Description |
|
| Variable | Default | Description |
|
||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
| `PEARDATA_DEFAULT_ROLE` | `viewer` | Baseline role for unknown peers: `viewer` \| `operator` \| `admin` |
|
| `PEARDATA_DEFAULT_ROLE` | `viewer` | Baseline role: `viewer` \| `operator` \| `admin` |
|
||||||
| `PEARDATA_ADMIN_KEYS` | empty | Comma-separated peer public keys always elevated to **admin** |
|
| `PEARDATA_ADMIN_KEYS` | empty | Comma-separated peer pubs always elevated to **admin** |
|
||||||
| `PEARDATA_ALLOWLIST` | empty | If **non-empty**, only listed peer pubs (plus already-registered policy peers) may connect |
|
| `PEARDATA_ALLOWLIST` | empty | If **non-empty**, only listed peers may connect |
|
||||||
| `PEARDATA_INSECURE_OPEN_ADMIN` | off | `1` / `true` / `yes` → every peer is admin. **Dev only.** |
|
| `PEARDATA_INSECURE_OPEN_ADMIN` | off | `1` → every peer is admin. **Dev only.** |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -38,20 +36,41 @@ Treat `SERVER_SEED` like a root password. Prefer `pd1.` invites for operators.
|
|||||||
| Variable | Default | Description |
|
| Variable | Default | Description |
|
||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
| `PEARDATA_DATA_DIR` | `./data` | Peer policy JSON + `audit.log` |
|
| `PEARDATA_DATA_DIR` | `./data` | Peer policy JSON + `audit.log` |
|
||||||
| `PEARDATA_HOME` | OS home | Root for client identity path construction (`client/identity.js`) |
|
| `PEARDATA_HOME` | OS home | Root for client identity (`~/.config/peardata/identity.json`) |
|
||||||
| `PEARDATA_MAX_MESSAGES` | `500` | In-memory demo room ring buffer size |
|
|
||||||
| `PEARDATA_RATE_LIMIT_RPM` | `120` | Per-peer RPC requests per minute |
|
| `PEARDATA_RATE_LIMIT_RPM` | `120` | Per-peer RPC requests per minute |
|
||||||
| `PEARDATA_MAX_RECONNECT` | `20` | Client manager reconnect attempts per peer |
|
| `PEARDATA_MAX_RECONNECT` | `20` | Client manager reconnect attempts per peer |
|
||||||
|
|
||||||
### Data directory layout
|
|
||||||
|
|
||||||
```
|
```
|
||||||
data/
|
data/
|
||||||
├── peer-policy.json # registered peers, roles, revocations, spent JTIs
|
├── peer-policy.json
|
||||||
└── audit.log # JSON lines for mutating RPCs + failures
|
└── audit.log
|
||||||
```
|
```
|
||||||
|
|
||||||
Recommended permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
Permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Metrics pipeline
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `PEARDATA_SAMPLE_MS` | `1000` | Collector interval |
|
||||||
|
| `PEARDATA_TIER0_POINTS` | `3600` | High-res ring size (~1h @ 1s) |
|
||||||
|
| `PEARDATA_TIER1_POINTS` | `1440` | Downsampled ring size |
|
||||||
|
| `PEARDATA_TIER1_EVERY` | `60` | Samples per tier1 average |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## REST API (Netdata-style)
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `PEARDATA_REST` | on | Set `0` / `off` to disable HTTP API |
|
||||||
|
| `PEARDATA_REST_HOST` | `127.0.0.1` | Bind address (`0.0.0.0` exposes LAN — firewall!) |
|
||||||
|
| `PEARDATA_REST_PORT` | `19999` | HTTP port (Netdata classic) |
|
||||||
|
| `PEARDATA_REST_CORS` | `*` | `Access-Control-Allow-Origin` |
|
||||||
|
|
||||||
|
See [REST-API.md](./REST-API.md).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -60,87 +79,38 @@ Recommended permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
|||||||
| Variable | Default | Description |
|
| Variable | Default | Description |
|
||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
| `LOG_LEVEL` | `info` | `debug` \| `info` \| `warn` \| `error` |
|
| `LOG_LEVEL` | `info` | `debug` \| `info` \| `warn` \| `error` |
|
||||||
| `LOG_JSON` | off | `1` → structured JSON logs (good for journald) |
|
| `LOG_JSON` | off | `1` → JSON lines |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Healthcheck & soak
|
## Healthcheck / soak / tests
|
||||||
|
|
||||||
| Variable | Default | Description |
|
| Variable | Default | Description |
|
||||||
|----------|---------|-------------|
|
|----------|---------|-------------|
|
||||||
| `PEARDATA_HEALTH_KEY` | — | Public key for remote health dial (falls back to `SERVER_PUBLIC_KEY`) |
|
| `PEARDATA_HEALTH_KEY` | `SERVER_PUBLIC_KEY` | Key for remote ping |
|
||||||
| `HEALTHCHECK_TIMEOUT_MS` | `8000` | Healthcheck hard timeout (ms) |
|
| `HEALTHCHECK_TIMEOUT_MS` | `8000` | Dial timeout |
|
||||||
| `SOAK_DURATION_MS` | `60000` | Soak test run length |
|
| `SOAK_DURATION_MS` | `60000` | Soak length |
|
||||||
| `SOAK_INTERVAL_MS` | `500` | Delay between soak posts |
|
| `SOAK_INTERVAL_MS` | `500` | Soak RPC interval |
|
||||||
|
| `SKIP_INTEGRATION` | off | `1` skips DHT integration test |
|
||||||
```bash
|
|
||||||
# Liveness only (no dial) when no key is set
|
|
||||||
npm run healthcheck
|
|
||||||
|
|
||||||
# Full dial + ping (server must be running)
|
|
||||||
SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run healthcheck
|
|
||||||
|
|
||||||
# Load soak (admin seed recommended so postMessage is allowed)
|
|
||||||
SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run soak
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
| Variable | Default | Description |
|
|
||||||
|----------|---------|-------------|
|
|
||||||
| `SKIP_INTEGRATION` | unset | `1` skips live HyperDHT integration test |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Pear GUI (package.json)
|
|
||||||
|
|
||||||
Not environment variables — set under `package.json` → `pear.gui`. See [DESKTOP.md](./DESKTOP.md).
|
|
||||||
|
|
||||||
| Field | Template |
|
|
||||||
|-------|----------|
|
|
||||||
| `width` × `height` | 1100 × 780 |
|
|
||||||
| `minWidth` × `minHeight` | 720 × 480 |
|
|
||||||
| `resizable` / `movable` | `true` |
|
|
||||||
| `backgroundColor` | `#0b1020` |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## systemd
|
|
||||||
|
|
||||||
`deploy/peardata.service` expects:
|
|
||||||
|
|
||||||
| Path | Purpose |
|
|
||||||
|------|---------|
|
|
||||||
| `/opt/peardata` | WorkingDirectory |
|
|
||||||
| `/opt/peardata/.env` | `EnvironmentFile` |
|
|
||||||
| `/opt/peardata/data` | Writable data dir |
|
|
||||||
|
|
||||||
Edit unit paths before enabling. See [GETTING-STARTED.md](./GETTING-STARTED.md#systemd).
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## npm scripts
|
## npm scripts
|
||||||
|
|
||||||
| Script | Command | Purpose |
|
| Script | Purpose |
|
||||||
|--------|---------|---------|
|
|--------|---------|
|
||||||
| `npm start` / `npm run dev` | `pear run -d .` | Pear desktop UI |
|
| `npm start` / `npm run dev` | Pear desktop |
|
||||||
| `npm run start:server` / `server` | `node server/server.js` | P2P server |
|
| `npm run start:server` | Agent (P2P + REST) |
|
||||||
| `npm run start:server:bin` | `node bin/peardata-server.mjs` | Alternate server entry |
|
| `npm run start:server:bin` | `bin/peardata-server.mjs` |
|
||||||
| `npm test` | brittle suite | Unit + integration |
|
| `npm test` | brittle suite |
|
||||||
| `npm run test:integration` | integration only | Live DHT test |
|
| `npm run mint-invite -- [role] [ttlMs]` | Offline `pd1.` invite |
|
||||||
| `npm run healthcheck` | dial or liveness | Process / network check |
|
| `npm run healthcheck` | Liveness |
|
||||||
| `npm run soak` | long connect loop | Stability exercise |
|
| `npm run soak` | Load exercise |
|
||||||
| `npm run mint-invite -- [role] [ttlMs]` | mint `pd1.` invite | Offline invite tooling |
|
|
||||||
| `npm run rename -- <slug> <Product>` | rebrand tree | New product from template |
|
|
||||||
| `bash scripts/release.sh` | source tarball | Local release artifacts |
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Security notes
|
## systemd
|
||||||
|
|
||||||
- Never ship `SERVER_SEED` in client bundles or public repos.
|
Unit: `deploy/peardata.service`
|
||||||
- Prefer invites over seed distribution.
|
WorkingDirectory: `/opt/peardata`
|
||||||
- Rotate seed = new public key → all clients re-dial and re-invite.
|
EnvironmentFile: `/opt/peardata/.env`
|
||||||
- See [SECURITY.md](./SECURITY.md) for the full checklist.
|
|
||||||
|
|||||||
@@ -0,0 +1,141 @@
|
|||||||
|
# Data model
|
||||||
|
|
||||||
|
Canonical shapes for metrics, anomalies, alerts, health, and jobs.
|
||||||
|
Implemented in `shared/data-model.js`, `shared/metrics.js`, and agent services.
|
||||||
|
|
||||||
|
## NodeInfo
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"nodeId": "a1b2c3…",
|
||||||
|
"hostname": "homelab-1",
|
||||||
|
"publicKeyHex": "…64 hex…",
|
||||||
|
"platform": "linux",
|
||||||
|
"arch": "x64",
|
||||||
|
"release": "6.x",
|
||||||
|
"cpus": 8,
|
||||||
|
"totalMemMiB": 32000,
|
||||||
|
"agentVersion": "0.1.0",
|
||||||
|
"protocolVersion": 1,
|
||||||
|
"startedAt": 1710000000000,
|
||||||
|
"charts": ["system.cpu", "system.ram", "…"],
|
||||||
|
"sampleIntervalMs": 1000,
|
||||||
|
"sampleCount": 42
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Metric contexts & charts
|
||||||
|
|
||||||
|
Netdata-inspired IDs:
|
||||||
|
|
||||||
|
| Context | Chart id | Units | Dimensions (MVP) |
|
||||||
|
|---------|----------|-------|------------------|
|
||||||
|
| `system.cpu` | `system.cpu` | percentage | user, system, nice, iowait, irq, softirq, idle |
|
||||||
|
| `system.ram` | `system.ram` | MiB | used, cached, buffers, free |
|
||||||
|
| `mem.available` | `mem.available` | MiB | avail |
|
||||||
|
| `system.load` | `system.load` | load | load1, load5, load15 |
|
||||||
|
| `system.io` | `system.io` | KiB/s | reads, writes |
|
||||||
|
| `system.net` | `system.net` | kilobits/s | received, sent |
|
||||||
|
| `system.processes` | `system.processes` | processes | running, blocked, total |
|
||||||
|
| `system.uptime` | `system.uptime` | seconds | uptime |
|
||||||
|
|
||||||
|
Chart summary objects mirror Netdata’s `/api/v1/charts` fields (`id`, `context`, `units`, `dimensions`, `update_every`, `first_entry`, `last_entry`, …).
|
||||||
|
|
||||||
|
## MetricSample (live push)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"chart": "system.cpu",
|
||||||
|
"context": "system.cpu",
|
||||||
|
"ts": 1710000000123,
|
||||||
|
"values": { "user": 12.4, "system": 3.1, "idle": 84.0 }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## QueryResult (history)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"chart": "system.cpu",
|
||||||
|
"context": "system.cpu",
|
||||||
|
"labels": ["time", "user", "system", "nice", "iowait", "irq", "softirq", "idle"],
|
||||||
|
"data": [[1710000000, 12.4, 3.1, 0, 0, 0, 0, 84.0]],
|
||||||
|
"view_update_every": 1,
|
||||||
|
"after": 1709999940,
|
||||||
|
"before": 1710000000,
|
||||||
|
"points": 60,
|
||||||
|
"format": "json"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`time` is unix seconds. Nulls allowed for missing dimensions.
|
||||||
|
|
||||||
|
## Storage tiers
|
||||||
|
|
||||||
|
| Tier | Resolution | Default retention | Env |
|
||||||
|
|------|------------|-------------------|-----|
|
||||||
|
| 0 | 1s | 3600 points (~1h) | `PEARDATA_TIER0_POINTS` |
|
||||||
|
| 1 | avg every N samples (default 60 → ~1m) | 1440 points (~24h) | `PEARDATA_TIER1_POINTS`, `PEARDATA_TIER1_EVERY` |
|
||||||
|
|
||||||
|
## AnomalyEvent
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "cpu_user_high:1710000000123",
|
||||||
|
"chart": "system.cpu",
|
||||||
|
"context": "system.cpu",
|
||||||
|
"dimension": "user",
|
||||||
|
"severity": "warning",
|
||||||
|
"score": 0.6,
|
||||||
|
"value": 88.2,
|
||||||
|
"threshold": 80,
|
||||||
|
"comparator": ">",
|
||||||
|
"message": "CPU user time high: user=88.2 > 80",
|
||||||
|
"ts": 1710000000123,
|
||||||
|
"cleared": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## AlertConfig / AlertState
|
||||||
|
|
||||||
|
Config fields: `id`, `chart`, `dimension`, `warn`, `crit`, `comparator`, `lookbackSec`, `enabled`, `info`.
|
||||||
|
|
||||||
|
State adds: `status` (`CLEAR`|`WARNING`|`CRITICAL`|`UNDEFINED`), `value`, `lastStatusChange`.
|
||||||
|
|
||||||
|
## HealthSnapshot
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"score": 1,
|
||||||
|
"checks": [{ "id": "cpu_user_high", "ok": true, "detail": "…" }],
|
||||||
|
"ts": 1710000000123
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Aggregate: any CRITICAL → `critical`; else any WARNING → `degraded`; else `ok`.
|
||||||
|
|
||||||
|
## JobRecord
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "uuid",
|
||||||
|
"name": "snapshot",
|
||||||
|
"status": "done",
|
||||||
|
"startedAt": 1710000000000,
|
||||||
|
"finishedAt": 1710000000500,
|
||||||
|
"result": { "ok": true }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## REST ↔ RPC parity
|
||||||
|
|
||||||
|
| Concept | RPC | REST |
|
||||||
|
|---------|-----|------|
|
||||||
|
| Charts | `listCharts` | `GET /api/v1/charts` |
|
||||||
|
| Data | `queryData` | `GET /api/v3/data` |
|
||||||
|
| Contexts | `listContexts` | `GET /api/v3/contexts` |
|
||||||
|
| Nodes | `getNodeInfo` | `GET /api/v3/nodes` |
|
||||||
|
| Alerts | `listAlerts` | `GET /api/v3/alerts` |
|
||||||
|
| Export | `getAllMetrics` | `GET /api/v3/allmetrics` |
|
||||||
|
| Health | `getHealth` | `GET /api/v3/health` |
|
||||||
+3
-3
@@ -8,7 +8,7 @@ The client shell is a **Pear desktop application** built with `pear-electron` +
|
|||||||
|------|------|
|
|------|------|
|
||||||
| `index.js` | Pear process entry — starts `pear-electron` Runtime + `pear-bridge` |
|
| `index.js` | Pear process entry — starts `pear-electron` Runtime + `pear-bridge` |
|
||||||
| `index.html` | GUI main (`pear.gui.main`) — titlebar + panels |
|
| `index.html` | GUI main (`pear.gui.main`) — titlebar + panels |
|
||||||
| `app.js` | UI logic (connect, room, presence, invites) |
|
| `app.js` | UI logic (connect, fleet, live charts, invites) |
|
||||||
| `ui/styles.css` | Layout, theme, **titlebar drag regions** |
|
| `ui/styles.css` | Layout, theme, **titlebar drag regions** |
|
||||||
| `client/*` | HyperDHT connection stack used by the UI |
|
| `client/*` | HyperDHT connection stack used by the UI |
|
||||||
|
|
||||||
@@ -25,8 +25,8 @@ Requires the [Pear](https://docs.pears.com) CLI installed and bootstrapped (`pea
|
|||||||
| Field | Template default | Purpose |
|
| Field | Template default | Purpose |
|
||||||
|-------|------------------|---------|
|
|-------|------------------|---------|
|
||||||
| `main` | `index.html` | HTML entry |
|
| `main` | `index.html` | HTML entry |
|
||||||
| `width` / `height` | `1100` / `780` | Initial size |
|
| `width` / `height` | `1280` / `860` | Initial size |
|
||||||
| `minWidth` / `minHeight` | `720` / `480` | Resize floor |
|
| `minWidth` / `minHeight` | `900` / `560` | Resize floor |
|
||||||
| `resizable` | `true` | Edge/corner resize |
|
| `resizable` | `true` | Edge/corner resize |
|
||||||
| `movable` | `true` | Allow OS move (with drag region) |
|
| `movable` | `true` | Allow OS move (with drag region) |
|
||||||
| `minimizable` / `maximizable` / `closable` | `true` | Window buttons |
|
| `minimizable` / `maximizable` / `closable` | `true` | Window buttons |
|
||||||
|
|||||||
+48
-119
@@ -1,159 +1,88 @@
|
|||||||
# Extending the template
|
# Extending PearData
|
||||||
|
|
||||||
## 1. Rebrand
|
## Add a chart / context
|
||||||
|
|
||||||
```bash
|
1. Define the chart in `shared/metrics.js` (`CHART_DEFS`).
|
||||||
npm run rename -- notes-mesh NotesMesh
|
2. Emit samples from `server/services/collector.js` (or a new collector module).
|
||||||
```
|
3. Store + REST/RPC pick it up automatically via `CHART_BY_ID`.
|
||||||
|
4. Document dimensions in [DATA-MODEL.md](./DATA-MODEL.md).
|
||||||
|
5. Optionally add a canvas panel in `index.html` + `app.js`.
|
||||||
|
|
||||||
This rewrites:
|
## Add an RPC method
|
||||||
|
|
||||||
- package name / product name
|
|
||||||
- protocol id (`notes-mesh/rpc`)
|
|
||||||
- env prefix (`NOTES_MESH_`)
|
|
||||||
- invite prefix (derived, e.g. `no1.`)
|
|
||||||
- binary + systemd unit filenames
|
|
||||||
|
|
||||||
Then:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm install
|
|
||||||
npm test
|
|
||||||
git diff # review
|
|
||||||
```
|
|
||||||
|
|
||||||
## 1b. Titlebar / window chrome
|
|
||||||
|
|
||||||
Keep these when restyling:
|
|
||||||
|
|
||||||
| Piece | Role |
|
|
||||||
|-------|------|
|
|
||||||
| `<pear-ctrl>` in `#titlebar` | Close / minimize / maximize (Pear runtime custom element) |
|
|
||||||
| `#titlebar { -webkit-app-region: drag }` | Drag the window |
|
|
||||||
| Interactive children `no-drag` | Buttons, chips, inputs stay clickable |
|
|
||||||
| `pear.gui.resizable: true` | Edge/corner resize |
|
|
||||||
| `pear.gui.minWidth` / `minHeight` | Floor size while resizing |
|
|
||||||
|
|
||||||
Do not remove `<pear-ctrl>` unless you intentionally want a frame without in-content controls (and understand platform differences). Details: [DESKTOP.md](./DESKTOP.md).
|
|
||||||
|
|
||||||
## 2. Add an RPC method
|
|
||||||
|
|
||||||
### `shared/protocol.js`
|
### `shared/protocol.js`
|
||||||
|
|
||||||
```js
|
```js
|
||||||
export const MethodRoles = Object.freeze({
|
export const MethodRoles = Object.freeze({
|
||||||
// ...
|
// ...
|
||||||
listNotes: Roles.viewer,
|
listContainers: Roles.viewer,
|
||||||
createNote: Roles.operator,
|
restartCollector: Roles.admin,
|
||||||
})
|
})
|
||||||
```
|
```
|
||||||
|
|
||||||
`Methods` is derived automatically from `MethodRoles` keys.
|
|
||||||
|
|
||||||
### `shared/schema.js`
|
### `shared/schema.js`
|
||||||
|
|
||||||
Validate `createNote` args (required fields, max lengths).
|
Validate args.
|
||||||
|
|
||||||
### `server/services/notes.js`
|
### `server/services/…` + `server/handlers/monitor.js`
|
||||||
|
|
||||||
Domain logic / storage (keep IO out of handlers when possible).
|
Register with `session.respond(...)`. Use `{ hot: true }` for high-frequency paths.
|
||||||
|
|
||||||
### `server/handlers/notes.js`
|
### Client
|
||||||
|
|
||||||
```js
|
```js
|
||||||
export function registerNoteHandlers(session) {
|
await manager.request(Methods.listContainers, {})
|
||||||
session.respond('listNotes', async () => ({ notes: [] }))
|
|
||||||
session.respond('createNote', async (args, s) => { /* ... */ })
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### `server/rpc/register.js`
|
### Docs + tests
|
||||||
|
|
||||||
Call `registerNoteHandlers(session)`.
|
Update [PROTOCOL.md](./PROTOCOL.md) and add a brittle test.
|
||||||
|
|
||||||
### Client / UI
|
## Add a REST route
|
||||||
|
|
||||||
|
Edit `server/rest/routes.js` — keep Netdata path naming when emulating Agent APIs (`/api/v3/...`).
|
||||||
|
|
||||||
|
## Add a job
|
||||||
|
|
||||||
|
Register in `server/services/jobs.js` `JOB_HANDLERS`.
|
||||||
|
|
||||||
```js
|
```js
|
||||||
await manager.request(Methods.createNote, { title: '…' })
|
retrainAnomaly: async () => { /* … */ return { ok: true } }
|
||||||
```
|
```
|
||||||
|
|
||||||
### Tests
|
Operators run via `runJob` or future REST function execute.
|
||||||
|
|
||||||
- Unit for pure helpers + schema
|
## Add a collector plugin (pattern)
|
||||||
- Integration for the happy path if AuthZ/wire matter
|
|
||||||
|
|
||||||
## 3. Add a push channel
|
```
|
||||||
|
server/services/collectors/
|
||||||
1. Add to `Pushes` in `shared/protocol.js`
|
system.js # default
|
||||||
2. Optionally map in `PushToType`
|
docker.js # Phase 3
|
||||||
3. `session.push(Pushes.foo, payload)` or broadcast via peer registry
|
peardock.js # bridge
|
||||||
4. Listen in UI: `manager.on('push', …)` or `conn.on(Pushes.foo, …)`
|
|
||||||
(`connection.js` auto-registers all `Pushes` values)
|
|
||||||
|
|
||||||
## 4. Persistence
|
|
||||||
|
|
||||||
Demo room is in-memory (`server/services/room.js`). Swap for:
|
|
||||||
|
|
||||||
| Store | Good for |
|
|
||||||
|-------|----------|
|
|
||||||
| **Corestore / Hypercore** | Append-only logs, P2P replication |
|
|
||||||
| **SQLite** | Structured queries |
|
|
||||||
| **JSON files** under `PEARDATA_DATA_DIR` | Small config (peer-policy already does this) |
|
|
||||||
|
|
||||||
Keep RPC handlers thin; put IO in `services/`.
|
|
||||||
|
|
||||||
## 5. Binary streams
|
|
||||||
|
|
||||||
Peardock-class apps use chunked binary RPC for uploads. Pattern:
|
|
||||||
|
|
||||||
- methods `binaryStreamOpen` / `Chunk` / `Close` marked `hot: true` in `session.respond`
|
|
||||||
- skip heavy schema/audit on the hot path
|
|
||||||
- still enforce ACL + rate limits
|
|
||||||
|
|
||||||
Stub hooks exist via `rateLimiter.isStreamMethod`.
|
|
||||||
|
|
||||||
## 6. Multi-server fleet
|
|
||||||
|
|
||||||
`client/manager.js` already holds many connections + active selection + reconnect:
|
|
||||||
|
|
||||||
```js
|
|
||||||
await manager.connect(keyOrInviteA, { autoReconnect: true })
|
|
||||||
await manager.connect(keyOrInviteB, { autoReconnect: true })
|
|
||||||
manager.setActive(keyA)
|
|
||||||
await manager.request(Methods.ping, {})
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Point UI at a peer list stored in `localStorage` or a file cache for a fuller fleet UX.
|
Have `pipeline.js` start each enabled collector; all emit `samples` batches into the same store.
|
||||||
|
|
||||||
## 7. Custom desktop UX
|
## Parent peer (fleet aggregator)
|
||||||
|
|
||||||
| Goal | Touch |
|
1. Parent dials child agents with `ConnectionManager`.
|
||||||
|------|-------|
|
2. Subscribes to `push:metrics` / periodically `queryData`.
|
||||||
| New screens | `index.html` + `app.js` + `ui/styles.css` |
|
3. Ingests into local store under namespaced chart ids (`childPk.system.cpu`) or labels.
|
||||||
| Window size | `package.json` → `pear.gui` |
|
4. Exposes `/api/v3/nodes` with multiple entries + `/api/v3/data` across nodes.
|
||||||
| Branding in titlebar | `.app-brand` markup/CSS |
|
|
||||||
| Persist UI prefs | replace demo `localStorage` keys |
|
|
||||||
|
|
||||||
Preserve drag / `pear-ctrl` behavior — [DESKTOP.md](./DESKTOP.md).
|
## PearDock / PearVirt hooks
|
||||||
|
|
||||||
## 8. Desktop packaging
|
- Prefer **RPC adapters** over scraping: call dock/virt methods, map to PearData contexts.
|
||||||
|
- Keep AGPL boundaries clean — depend on public APIs / your own MIT bridges.
|
||||||
|
|
||||||
This template ships the **Pear** GUI path (`pear run`). For Electron-forge / bare-standalone multi-arch releases, copy more complex packaging scripts from a production app (forge config, make scripts) once your protocol stabilizes.
|
## Desktop chrome
|
||||||
|
|
||||||
## 9. Checklist for a new product
|
Keep `<pear-ctrl>`, titlebar drag regions, and `pear.gui.resizable`. See [DESKTOP.md](./DESKTOP.md).
|
||||||
|
|
||||||
- [ ] `npm run rename -- …`
|
## Rebrand (fork)
|
||||||
- [ ] Replace demo handlers + services
|
|
||||||
- [ ] Update PROTOCOL.md method table
|
|
||||||
- [ ] Update SECURITY / CONFIG if new secrets
|
|
||||||
- [ ] Tests green (`npm test`)
|
|
||||||
- [ ] README product description
|
|
||||||
- [ ] CI still green
|
|
||||||
- [ ] First release tag ([RELEASE.md](./RELEASE.md))
|
|
||||||
|
|
||||||
## Related
|
```bash
|
||||||
|
npm run rename -- my-monitor MyMonitor
|
||||||
|
```
|
||||||
|
|
||||||
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
Review invite prefix / env prefix, then `npm test`.
|
||||||
- [PROTOCOL.md](./PROTOCOL.md)
|
|
||||||
- [TESTING.md](./TESTING.md)
|
|
||||||
- [DESKTOP.md](./DESKTOP.md)
|
|
||||||
|
|||||||
+63
-146
@@ -1,188 +1,105 @@
|
|||||||
# Getting started
|
# Getting started
|
||||||
|
|
||||||
## Prerequisites
|
## Requirements
|
||||||
|
|
||||||
| Tool | Required | Notes |
|
- Node.js **≥ 20**
|
||||||
|------|----------|--------|
|
- Pear runtime (for desktop): install via Holepunch/Pear docs
|
||||||
| **Node.js ≥ 20** | Yes | Server + tests |
|
- Linux recommended for full net/disk `/proc` collectors (macOS gets CPU/RAM/load)
|
||||||
| **npm** | Yes | Install deps |
|
|
||||||
| **[Pear](https://docs.pears.com) CLI** | For desktop UI | Run `pear` once to bootstrap the runtime |
|
|
||||||
| UDP / network | For real peers | HyperDHT hole-punching |
|
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone <your-fork-or-template-url> my-app
|
cd peardata
|
||||||
cd my-app
|
|
||||||
npm install
|
npm install
|
||||||
cp .env.example .env # optional; server auto-writes seed on first boot
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Run the server
|
Copy env template if needed:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env
|
||||||
|
```
|
||||||
|
|
||||||
|
## Run the agent
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm run start:server
|
npm run start:server
|
||||||
# alias: npm run server
|
|
||||||
```
|
```
|
||||||
|
|
||||||
On first boot the server appends to `.env`:
|
Banner shows:
|
||||||
|
|
||||||
```
|
- **publicKey** — dial this from the desktop (viewer)
|
||||||
SERVER_SEED=<64 hex secret>
|
- **rest** — e.g. `http://127.0.0.1:19999/api/v3/info`
|
||||||
SERVER_PUBLIC_KEY=<64 hex public>
|
- **admin** — use `SERVER_SEED` from `.env` or mint `pd1.` invites
|
||||||
|
|
||||||
|
Agent persists identity in `.env` on first boot.
|
||||||
|
|
||||||
|
### REST smoke tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -s http://127.0.0.1:19999/api/v3/info
|
||||||
|
curl -s http://127.0.0.1:19999/api/v1/charts | head
|
||||||
|
curl -s 'http://127.0.0.1:19999/api/v3/data?chart=system.cpu&after=-60&points=60'
|
||||||
|
curl -s 'http://127.0.0.1:19999/api/v3/allmetrics?format=prometheus' | head
|
||||||
```
|
```
|
||||||
|
|
||||||
**Treat `SERVER_SEED` like a root password.** Anyone with it can mint admin proofs and capabilities.
|
Disable REST: `PEARDATA_REST=0`.
|
||||||
|
|
||||||
Banner output shows the public key clients dial, for example:
|
## Mint an invite
|
||||||
|
|
||||||
```
|
```bash
|
||||||
Client → dial <SERVER_PUBLIC_KEY>
|
npm run mint-invite -- operator
|
||||||
|
# → pd1.…
|
||||||
```
|
```
|
||||||
|
|
||||||
Leave this process running while clients connect.
|
Share the invite string. Recipients paste it into the desktop connect box.
|
||||||
|
|
||||||
## Run the desktop UI
|
## Run the desktop
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm start
|
npm start
|
||||||
# or: npm run dev
|
# pear run -d .
|
||||||
# or: pear run -d .
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Desktop window chrome
|
1. Paste agent public key or `pd1.` invite
|
||||||
|
2. Optional: paste `SERVER_SEED` for admin
|
||||||
|
3. Connect → live overview + charts
|
||||||
|
4. Admin: **Mint invite**
|
||||||
|
|
||||||
The Pear UI uses a custom titlebar:
|
## systemd (Linux)
|
||||||
|
|
||||||
| Piece | Behavior |
|
|
||||||
|-------|----------|
|
|
||||||
| `<pear-ctrl>` | Platform window controls (macOS traffic lights / Windows & Linux min·max·close) |
|
|
||||||
| `#titlebar` | `-webkit-app-region: drag` — drag to move the window |
|
|
||||||
| Interactive children | `no-drag` so buttons and chips stay clickable |
|
|
||||||
| `pear.gui.resizable` | Edge/corner resize (`true` by default) |
|
|
||||||
| `minWidth` / `minHeight` | 720 × 480 floor |
|
|
||||||
|
|
||||||
Full details: [DESKTOP.md](./DESKTOP.md).
|
|
||||||
|
|
||||||
## Connect as admin (dev)
|
|
||||||
|
|
||||||
1. Start the Pear UI: `npm start`
|
|
||||||
2. Paste `SERVER_PUBLIC_KEY` into **Server / invite**
|
|
||||||
3. Paste `SERVER_SEED` into **Admin seed**
|
|
||||||
4. Optionally set a **Display name**
|
|
||||||
5. Click **Connect** → role badge should show `admin`
|
|
||||||
|
|
||||||
The seed never goes over the wire as plaintext — the client sends an HMAC **admin proof**.
|
|
||||||
|
|
||||||
## Connect via invite
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Persistent operator invite (default)
|
|
||||||
npm run mint-invite -- operator
|
|
||||||
|
|
||||||
# 7-day operator invite (ttl in ms)
|
|
||||||
npm run mint-invite -- operator 604800000
|
|
||||||
|
|
||||||
# Admin invite
|
|
||||||
npm run mint-invite -- admin
|
|
||||||
```
|
|
||||||
|
|
||||||
Stdout prints a `pd1.…` string. Paste it into the UI connect field (no seed needed).
|
|
||||||
|
|
||||||
You can also mint from a connected admin session with **Mint invite** in the UI.
|
|
||||||
|
|
||||||
## Viewer-only
|
|
||||||
|
|
||||||
Paste only the public key. You can:
|
|
||||||
|
|
||||||
- `listMessages`, `getPresence`, `getServerInfo`, `setDisplayName`, `ping`
|
|
||||||
|
|
||||||
You cannot:
|
|
||||||
|
|
||||||
- `postMessage` (needs `operator+`)
|
|
||||||
- `clearMessages`, `mintInvite`, `listPeers`, `revokePeer` (needs `admin`)
|
|
||||||
|
|
||||||
Unless you raise `PEARDATA_DEFAULT_ROLE` (not recommended for multi-user hosts).
|
|
||||||
|
|
||||||
## Quick verification
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm test
|
|
||||||
SKIP_INTEGRATION=1 npm test # unit only
|
|
||||||
|
|
||||||
# With server running:
|
|
||||||
export SERVER_PUBLIC_KEY=… # from .env
|
|
||||||
export SERVER_SEED=… # optional but enables admin dial
|
|
||||||
npm run healthcheck
|
|
||||||
npm run soak # optional load exercise
|
|
||||||
```
|
|
||||||
|
|
||||||
## Environment knobs (summary)
|
|
||||||
|
|
||||||
| Variable | Purpose |
|
|
||||||
|----------|---------|
|
|
||||||
| `SERVER_SEED` / `SERVER_PUBLIC_KEY` | Server identity |
|
|
||||||
| `PEARDATA_DEFAULT_ROLE` | Baseline role (`viewer` default) |
|
|
||||||
| `PEARDATA_ADMIN_KEYS` | Peer pubs always admin |
|
|
||||||
| `PEARDATA_INSECURE_OPEN_ADMIN` | Dev only — all peers admin |
|
|
||||||
| `PEARDATA_ALLOWLIST` | If set, only listed / registered peers |
|
|
||||||
| `PEARDATA_DATA_DIR` | Peer policy + audit log directory |
|
|
||||||
| `PEARDATA_RATE_LIMIT_RPM` | Per-peer RPC budget |
|
|
||||||
| `PEARDATA_MAX_MESSAGES` | Demo room history cap |
|
|
||||||
| `PEARDATA_MAX_RECONNECT` | Client reconnect tries |
|
|
||||||
| `LOG_LEVEL` / `LOG_JSON` | Logging |
|
|
||||||
|
|
||||||
Full table: [CONFIGURATION.md](./CONFIGURATION.md).
|
|
||||||
|
|
||||||
## Rebrand for a new product
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm run rename -- my-app MyApp
|
|
||||||
npm install
|
|
||||||
npm test
|
|
||||||
```
|
|
||||||
|
|
||||||
Rewrites package name, protocol id, env prefixes, invite prefix (`pd1.` → derived), product strings, and renames the server binary / systemd unit. Review `git diff` after.
|
|
||||||
|
|
||||||
## systemd
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Install tree to /opt/peardata (example)
|
|
||||||
sudo mkdir -p /opt/peardata
|
sudo mkdir -p /opt/peardata
|
||||||
sudo rsync -a --exclude node_modules --exclude .git ./ /opt/peardata/
|
sudo rsync -a ./ /opt/peardata/ --exclude node_modules
|
||||||
cd /opt/peardata && sudo npm install --omit=dev
|
cd /opt/peardata && sudo npm ci --omit=dev
|
||||||
|
|
||||||
sudo cp deploy/peardata.service /etc/systemd/system/
|
sudo cp deploy/peardata.service /etc/systemd/system/
|
||||||
# Edit WorkingDirectory, EnvironmentFile, ReadWritePaths if paths differ
|
|
||||||
sudo systemctl daemon-reload
|
sudo systemctl daemon-reload
|
||||||
sudo systemctl enable --now peardata
|
sudo systemctl enable --now peardata
|
||||||
sudo journalctl -u peardata -f
|
journalctl -u peardata -f
|
||||||
```
|
```
|
||||||
|
|
||||||
The unit sets `NoNewPrivileges`, `ProtectSystem=strict`, and writable paths for `data/` + `.env`.
|
Ensure `ReadWritePaths` can write `/opt/peardata/data` and `.env`.
|
||||||
|
|
||||||
|
## Roles cheat sheet
|
||||||
|
|
||||||
|
| Goal | Action |
|
||||||
|
|------|--------|
|
||||||
|
| Read-only monitoring | Dial pubkey only |
|
||||||
|
| Ack alerts / run jobs | `pd1.` operator invite |
|
||||||
|
| Mint invites / revoke | Admin seed or admin invite |
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
| Symptom | What to check |
|
| Symptom | Check |
|
||||||
|---------|----------------|
|
|---------|-------|
|
||||||
| `Connection timeout` | Server running? Correct 64-hex key? Firewall / UDP? |
|
| REST connection refused | Agent running? `PEARDATA_REST` not `0`? Port free? |
|
||||||
| `PERMISSION_DENIED` on send | Role is viewer — use invite or admin seed |
|
| Desktop can’t dial | Firewall / DHT; same machine should work; wait for punch |
|
||||||
| `Rate limit exceeded` | Raise `PEARDATA_RATE_LIMIT_RPM` or slow clients |
|
| Empty net/disk charts | Non-Linux host — expected until platform collectors land |
|
||||||
| Window won’t drag | Titlebar drag CSS; don’t cover bar with full-screen `no-drag` overlay |
|
| `PERMISSION_DENIED` | Need higher role invite or admin seed |
|
||||||
| Window won’t resize | `pear.gui.resizable` must be true; try edges not just corners |
|
| Integration tests hang | `SKIP_INTEGRATION=1 npm test` on restricted networks |
|
||||||
| No `<pear-ctrl>` buttons | Running under Pear (`pear run`)? Element is runtime-provided |
|
|
||||||
| Integration test fails in CI | Set `SKIP_INTEGRATION=1` or allow UDP |
|
|
||||||
| Seed regenerated every boot | `.env` not writable / wrong cwd |
|
|
||||||
|
|
||||||
## Next steps
|
## Next reading
|
||||||
|
|
||||||
| Doc | When |
|
- [REST-API.md](./REST-API.md)
|
||||||
|-----|------|
|
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
||||||
| [DESKTOP.md](./DESKTOP.md) | Titlebar, pear-ctrl, packaging |
|
- [ROADMAP.md](./ROADMAP.md)
|
||||||
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Stack & session pipeline |
|
- [CONFIGURATION.md](./CONFIGURATION.md)
|
||||||
| [PROTOCOL.md](./PROTOCOL.md) | Methods, pushes, versioning |
|
|
||||||
| [SECURITY.md](./SECURITY.md) | Production hardening |
|
|
||||||
| [CONFIGURATION.md](./CONFIGURATION.md) | Full env reference |
|
|
||||||
| [EXTENDING.md](./EXTENDING.md) | Replace the demo room |
|
|
||||||
| [TESTING.md](./TESTING.md) | Tests & soak |
|
|
||||||
| [CI.md](./CI.md) / [RELEASE.md](./RELEASE.md) | Pipelines & shipping |
|
|
||||||
|
|||||||
+104
-131
@@ -1,158 +1,131 @@
|
|||||||
# Protocol
|
# Protocol
|
||||||
|
|
||||||
## Constants
|
Wire contract for PearData P2P RPC. Source of truth: `shared/protocol.js`, `shared/schema.js`, `shared/metrics.js`.
|
||||||
|
|
||||||
|
## Identity
|
||||||
|
|
||||||
| Constant | Value |
|
| Constant | Value |
|
||||||
|----------|--------|
|
|----------|-------|
|
||||||
| `PROTOCOL` | `peardata/rpc` |
|
| Protocol id | `peardata/rpc` |
|
||||||
| `PROTOCOL_VERSION` | `1` |
|
| Protocol version | `1` |
|
||||||
| `APP_NAME` | `peardata` |
|
| Invite prefix | `pd1.` |
|
||||||
| `APP_VERSION` | `0.1.0` (keep in sync with package where useful) |
|
| Default role | `viewer` |
|
||||||
| Encoding | compact-encoding JSON (`shared/encodings.js`) |
|
|
||||||
| Schema | lightweight validators (`shared/schema.js`) `SCHEMA_VERSION=1` |
|
|
||||||
|
|
||||||
Bump `PROTOCOL_VERSION` on breaking request/response shapes. Additive methods may land without a bump if clients ignore unknown methods.
|
Bump `PROTOCOL_VERSION` on breaking argument/result shapes.
|
||||||
|
|
||||||
After `npm run rename`, `PROTOCOL` becomes `<slug>/rpc` and invite prefix is regenerated.
|
|
||||||
|
|
||||||
## Transport
|
|
||||||
|
|
||||||
1. Client opens HyperDHT secret stream to server public key (Noise, mutual key auth).
|
|
||||||
2. `ProtomuxRPC` is attached with `protocol: PROTOCOL` and shared encodings.
|
|
||||||
3. Client calls `handshake` before other RPCs (connection helper does this automatically).
|
|
||||||
4. Server may `push` events on named channels.
|
|
||||||
|
|
||||||
## Roles
|
## Roles
|
||||||
|
|
||||||
| Role | Intent |
|
| Role | Rank | Typical access |
|
||||||
|------|--------|
|
|------|------|----------------|
|
||||||
| `viewer` | Read-only |
|
| `viewer` | 1 | Read metrics, subscribe, list alerts |
|
||||||
| `operator` | Mutate domain data |
|
| `operator` | 2 | Ack/silence alerts, run jobs, set alert config |
|
||||||
| `admin` | Invite mint, clear, revoke, config |
|
| `admin` | 3 | Mint invites, revoke peers, export snapshot |
|
||||||
|
|
||||||
Hierarchy: `admin > operator > viewer` (`roleAllows`).
|
Auth modes at handshake: public key (viewer), capability token / `pd1.` invite, admin seed proof, allowlist.
|
||||||
|
|
||||||
Unknown methods default to **admin** required in `assertAllowed` if missing from `MethodRoles` — always register new methods.
|
|
||||||
|
|
||||||
## Methods
|
## Methods
|
||||||
|
|
||||||
| Method | Min role | Request args | Response (summary) |
|
### Session
|
||||||
|--------|----------|--------------|--------------------|
|
|
||||||
| `handshake` | viewer | `clientName`, `clientVersion`, optional `capability`, `adminProof` | role, auth, versions, features |
|
|
||||||
| `ping` | viewer | `{}` | `{ ok, pong, peerId }` |
|
|
||||||
| `getServerInfo` | viewer | `{}` | app, versions, host, peer count |
|
|
||||||
| `getAuthStatus` | viewer | `{}` | peerId, role, authMode, displayName |
|
|
||||||
| `listMessages` | viewer | `{ limit? }` | `{ messages: [...] }` |
|
|
||||||
| `getPresence` | viewer | `{}` | `{ peers: [...] }` |
|
|
||||||
| `postMessage` | operator | `{ text }` (1–2000 chars) | created message |
|
|
||||||
| `setDisplayName` | viewer | `{ name }` (1–40 chars) | updated label |
|
|
||||||
| `clearMessages` | admin | `{}` | success + system push |
|
|
||||||
| `mintInvite` | admin | `{ role?, ttlMs?, peerId?, alias? }` | `invite` (`pd1.…`), jti, exp |
|
|
||||||
| `listPeers` | admin | `{}` | live + policy peers |
|
|
||||||
| `revokePeer` | admin | `{ peerId }` (64 hex) | success; target dropped |
|
|
||||||
|
|
||||||
### Handshake request
|
| Method | Role | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| `handshake` | viewer | Negotiate version + elevate role |
|
||||||
|
| `ping` | viewer | Hot |
|
||||||
|
| `getServerInfo` | viewer | Agent metadata |
|
||||||
|
| `getAuthStatus` | viewer | Peer role / auth mode |
|
||||||
|
| `setDisplayName` | viewer | Label for audit / UI |
|
||||||
|
|
||||||
```json
|
### Node
|
||||||
{
|
|
||||||
"clientName": "peardata",
|
|
||||||
"clientVersion": "0.1.0",
|
|
||||||
"capability": "<optional HMAC token>",
|
|
||||||
"adminProof": { "nonce": "<hex>", "mac": "<hex>" }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Handshake response
|
| Method | Role | Result |
|
||||||
|
|--------|------|--------|
|
||||||
|
| `getNodeInfo` | viewer | Hostname, CPUs, charts, sample interval |
|
||||||
|
| `getHealth` | viewer | `{ status, score, checks }` |
|
||||||
|
|
||||||
```json
|
### Metrics discovery & query
|
||||||
{
|
|
||||||
"success": true,
|
|
||||||
"protocol": "peardata/rpc",
|
|
||||||
"protocolVersion": 1,
|
|
||||||
"schemaVersion": 1,
|
|
||||||
"role": "operator",
|
|
||||||
"peerId": "<64 hex>",
|
|
||||||
"serverTime": 0,
|
|
||||||
"auth": { "mode": "capability", "role": "operator" },
|
|
||||||
"features": { "hmacAuth": true, "invites": true, "room": true }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Auth modes (`auth.mode` / `session.authMode`)
|
| Method | Role | Args | Result |
|
||||||
|
|--------|------|------|--------|
|
||||||
|
| `listContexts` | viewer | — | Context catalog |
|
||||||
|
| `getContext` | viewer | `{ id }` | Charts in context |
|
||||||
|
| `listCharts` | viewer | — | Netdata-ish chart map |
|
||||||
|
| `getChart` | viewer | `{ id }` | Chart summary |
|
||||||
|
| `queryData` | viewer | `{ chart, after, before, points, group, tier }` | Time series |
|
||||||
|
| `getAllMetrics` | viewer | `{ format: json\|prometheus\|shell }` | Latest export |
|
||||||
|
|
||||||
| Mode | How obtained |
|
`after` / `before`: absolute unix seconds, or relative (negative = relative to `before`/`now`), Netdata-style.
|
||||||
|------|----------------|
|
|
||||||
| `viewer` | Default after connect with no grant |
|
|
||||||
| `capability` | Valid capability / invite |
|
|
||||||
| `seed` | Valid admin proof from `SERVER_SEED` |
|
|
||||||
| `allowlist` / registered | Elevated via policy / admin keys (implementation in ACL + policy) |
|
|
||||||
|
|
||||||
Exact labels depend on handshake path; UI shows `authMode` from `getAuthStatus`.
|
### Live subscriptions
|
||||||
|
|
||||||
## Pushes (server → client events)
|
| Method | Role | Args |
|
||||||
|
|--------|------|------|
|
||||||
|
| `subscribeMetrics` | viewer | `{ charts: string[]\|['*'], intervalMs }` |
|
||||||
|
| `unsubscribeMetrics` | viewer | — |
|
||||||
|
| `subscribeAnomalies` | viewer | — |
|
||||||
|
| `unsubscribeAnomalies` | viewer | — |
|
||||||
|
|
||||||
| Push | Payload |
|
### Anomalies & alerts
|
||||||
|
|
||||||
|
| Method | Role | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| `listAnomalies` | viewer | Recent events |
|
||||||
|
| `listAlerts` / `getAlert` | viewer | State + config |
|
||||||
|
| `setAlertConfig` | operator | Upsert threshold |
|
||||||
|
| `ackAlert` | operator | Clear until next breach |
|
||||||
|
| `silenceAlert` | operator | Disable temporarily |
|
||||||
|
|
||||||
|
### Jobs
|
||||||
|
|
||||||
|
| Method | Role | Known jobs |
|
||||||
|
|--------|------|------------|
|
||||||
|
| `listJobs` | viewer | — |
|
||||||
|
| `runJob` | operator | `collectOnce`, `snapshot`, `gcBuffers` |
|
||||||
|
| `cancelJob` | operator | By job id |
|
||||||
|
|
||||||
|
### Admin
|
||||||
|
|
||||||
|
| Method | Role |
|
||||||
|
|--------|------|
|
||||||
|
| `mintInvite` | admin |
|
||||||
|
| `listPeers` | admin |
|
||||||
|
| `revokePeer` | admin |
|
||||||
|
| `exportSnapshot` | admin |
|
||||||
|
|
||||||
|
## Pushes (server → client)
|
||||||
|
|
||||||
|
| Event | Payload |
|
||||||
|
|-------|---------|
|
||||||
|
| `push:metrics` | `{ samples: MetricSample[] }` |
|
||||||
|
| `push:anomaly` | `AnomalyEvent` |
|
||||||
|
| `push:alert` | Alert transition |
|
||||||
|
| `push:health` | `HealthSnapshot` |
|
||||||
|
| `push:job` | `JobRecord` |
|
||||||
|
| `push:system` | Generic notices |
|
||||||
|
|
||||||
|
## Errors
|
||||||
|
|
||||||
|
Handlers throw `Error` with `.code`:
|
||||||
|
|
||||||
|
| Code | Meaning |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| `push:message` | `{ id, peerId, displayName, text, ts }` |
|
| `PERMISSION_DENIED` | Role too low |
|
||||||
| `push:presence` | `{ peers: [...] }` |
|
| `RATE_LIMIT_EXCEEDED` | RPM exceeded |
|
||||||
| `push:system` | `{ type, ... }` e.g. `{ type: "cleared" }` |
|
| `INVALID_ARGS` | Schema failure |
|
||||||
|
| `NOT_CONNECTED` | Client-side |
|
||||||
Registered via `rpc.event` / `session.push`. Client `connection.js` binds all `Pushes` values.
|
| `CAPABILITY_*` | Invite/token issues |
|
||||||
|
|
||||||
## Invites
|
|
||||||
|
|
||||||
Envelope: `pd1.` + base64url(JSON):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"v": 1,
|
|
||||||
"publicKeyHex": "<server>",
|
|
||||||
"capability": "<token>",
|
|
||||||
"role": "operator",
|
|
||||||
"jti": "...",
|
|
||||||
"expiresAt": null
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Capability token: `base64url(payload).base64url(HMAC-SHA256)`.
|
|
||||||
|
|
||||||
Payload fields (canonical order for MAC): `v`, `role`, `peerId`, `exp`, `jti`, `iat`.
|
|
||||||
|
|
||||||
### Connection input classification
|
|
||||||
|
|
||||||
`classifyConnectionInput(string)` accepts:
|
|
||||||
|
|
||||||
| Input | Kind |
|
|
||||||
|-------|------|
|
|
||||||
| 64 hex chars | `publicKey` |
|
|
||||||
| `pd1.…` | `invite` (extracts key + capability) |
|
|
||||||
| other | error |
|
|
||||||
|
|
||||||
## Error codes
|
|
||||||
|
|
||||||
| Code | When |
|
|
||||||
|------|------|
|
|
||||||
| `RATE_LIMIT_EXCEEDED` | Peer over RPM budget |
|
|
||||||
| `PERMISSION_DENIED` | Role too low for method |
|
|
||||||
| `INVALID_ARGS` | Schema validation failed |
|
|
||||||
| `CONNECTION_TIMEOUT` | Client dial timeout |
|
|
||||||
| `RPC_ERROR` | Generic client-normalized failure |
|
|
||||||
| `UNKNOWN_ERROR` | Unclassified server handler error |
|
|
||||||
|
|
||||||
Clients should read `error.code` when present (`client/errors.js` preserves codes).
|
|
||||||
|
|
||||||
## Versioning policy
|
## Versioning policy
|
||||||
|
|
||||||
1. Document every method in this file.
|
1. Additive methods/fields: no version bump if old clients ignore unknowns.
|
||||||
2. Add `MethodRoles` entry before implementing handlers.
|
2. Rename/remove/change meaning: bump `PROTOCOL_VERSION`; reject or compat-negotiate in handshake.
|
||||||
3. Add `validateMethodArgs` case for mutating methods.
|
3. REST API versions (`v1`/`v2`/`v3`) are independent of RPC version but share the store.
|
||||||
4. Add brittle tests for pure helpers; integration test for critical paths.
|
|
||||||
5. Bump `PROTOCOL_VERSION` when existing response shapes break.
|
|
||||||
6. Bump `SCHEMA_VERSION` when validation semantics change meaningfully.
|
|
||||||
|
|
||||||
## Related
|
## Example session
|
||||||
|
|
||||||
- [ARCHITECTURE.md](./ARCHITECTURE.md)
|
```text
|
||||||
- [SECURITY.md](./SECURITY.md)
|
client → handshake { clientName, clientVersion, capability? }
|
||||||
- [EXTENDING.md](./EXTENDING.md)
|
server → { role, protocolVersion, auth }
|
||||||
- [TESTING.md](./TESTING.md)
|
client → subscribeMetrics { charts: ['*'], intervalMs: 1000 }
|
||||||
|
server → push:metrics { samples: [...] } # ~1 Hz
|
||||||
|
client → queryData { chart: 'system.cpu', after: -300, points: 300 }
|
||||||
|
```
|
||||||
|
|||||||
+14
-11
@@ -2,17 +2,20 @@
|
|||||||
|
|
||||||
| Doc | Audience | Contents |
|
| Doc | Audience | Contents |
|
||||||
|-----|----------|----------|
|
|-----|----------|----------|
|
||||||
| [GETTING-STARTED.md](./GETTING-STARTED.md) | New operators | Install, run, connect, systemd, troubleshooting |
|
| [GETTING-STARTED.md](./GETTING-STARTED.md) | Operators | Install, run agent, REST, desktop, systemd |
|
||||||
| [DESKTOP.md](./DESKTOP.md) | UI developers | Pear shell, `pear-ctrl`, drag/resize, identity |
|
| [ROADMAP.md](./ROADMAP.md) | Everyone | Phased MVP → advanced Netdata-class features |
|
||||||
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Engineers | Planes, boot, middleware, module map |
|
| [ARCHITECTURE.md](./ARCHITECTURE.md) | Engineers | Planes, PearDock mapping, module map |
|
||||||
| [PROTOCOL.md](./PROTOCOL.md) | Protocol owners | Methods, pushes, errors, versioning |
|
| [PROTOCOL.md](./PROTOCOL.md) | Protocol owners | RPC methods, pushes, versioning |
|
||||||
| [SECURITY.md](./SECURITY.md) | Operators / security | Trust model, checklist, crypto, incidents |
|
| [DATA-MODEL.md](./DATA-MODEL.md) | Engineers | Metrics, anomalies, alerts, jobs |
|
||||||
|
| [REST-API.md](./REST-API.md) | Integrators | Netdata-style `/api/v1\|v2\|v3` |
|
||||||
|
| [TECH-CHOICES.md](./TECH-CHOICES.md) | Engineers | Collector, charts, libraries |
|
||||||
|
| [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) | Engineers | HyperDB / Corestore / Autobase sync design |
|
||||||
|
| [DESKTOP.md](./DESKTOP.md) | UI developers | Pear shell, titlebar, identity |
|
||||||
|
| [SECURITY.md](./SECURITY.md) | Operators / security | Trust model, secrets, hardening |
|
||||||
| [CONFIGURATION.md](./CONFIGURATION.md) | Operators | Full env vars + npm scripts |
|
| [CONFIGURATION.md](./CONFIGURATION.md) | Operators | Full env vars + npm scripts |
|
||||||
| [TESTING.md](./TESTING.md) | Contributors | brittle suite, soak, manual QA |
|
| [TESTING.md](./TESTING.md) | Contributors | brittle, soak, manual QA |
|
||||||
| [CI.md](./CI.md) | Maintainers | GitHub/Gitea pipelines |
|
| [CI.md](./CI.md) | Maintainers | GitHub/Gitea pipelines |
|
||||||
| [RELEASE.md](./RELEASE.md) | Maintainers | Version, tag, tarball, rollback |
|
| [RELEASE.md](./RELEASE.md) | Maintainers | Version, tag, tarball |
|
||||||
| [EXTENDING.md](./EXTENDING.md) | Product builders | Rebrand, new RPCs, persistence, fleet |
|
| [EXTENDING.md](./EXTENDING.md) | Product builders | New charts, collectors, parents |
|
||||||
|
|
||||||
Start here if you are new: **[GETTING-STARTED.md](./GETTING-STARTED.md)**.
|
Start here: **[GETTING-STARTED.md](./GETTING-STARTED.md)** · Product overview: **[../README.md](../README.md)**.
|
||||||
|
|
||||||
Root overview: **[../README.md](../README.md)**.
|
|
||||||
|
|||||||
@@ -0,0 +1,147 @@
|
|||||||
|
# REST API (Netdata-compatible)
|
||||||
|
|
||||||
|
PearMonitor agents expose an optional HTTP API modeled on **Netdata Agent** endpoints (`/api/v1`, `/api/v2`, `/api/v3`).
|
||||||
|
|
||||||
|
Default bind: `http://127.0.0.1:19999` (Netdata’s classic port).
|
||||||
|
Disable: `PEARDATA_REST=0`.
|
||||||
|
Bind all interfaces (careful): `PEARDATA_REST_HOST=0.0.0.0`.
|
||||||
|
|
||||||
|
This is a **compatibility layer**, not a byte-for-byte Netdata clone. Core query/metadata paths are implemented for scripts, Grafana, and Prometheus scrapers.
|
||||||
|
|
||||||
|
## Quick examples
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Agent info
|
||||||
|
curl -s http://127.0.0.1:19999/api/v3/info | jq
|
||||||
|
|
||||||
|
# Chart catalog (v1 style)
|
||||||
|
curl -s http://127.0.0.1:19999/api/v1/charts | jq '.charts | keys'
|
||||||
|
|
||||||
|
# Last 60s of CPU (v3)
|
||||||
|
curl -s 'http://127.0.0.1:19999/api/v3/data?chart=system.cpu&after=-60&points=60' | jq
|
||||||
|
|
||||||
|
# Contexts
|
||||||
|
curl -s http://127.0.0.1:19999/api/v3/contexts | jq
|
||||||
|
|
||||||
|
# Prometheus export
|
||||||
|
curl -s 'http://127.0.0.1:19999/api/v3/allmetrics?format=prometheus'
|
||||||
|
|
||||||
|
# Health
|
||||||
|
curl -s http://127.0.0.1:19999/api/v3/health | jq
|
||||||
|
```
|
||||||
|
|
||||||
|
## Endpoint matrix
|
||||||
|
|
||||||
|
### Info & identity
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| GET | `/` or `/api` | Service index + P2P pubkey |
|
||||||
|
| GET | `/api/v1/info` | Agent info |
|
||||||
|
| GET | `/api/v2/info` | same |
|
||||||
|
| GET | `/api/v3/info` | **preferred** |
|
||||||
|
| GET | `/api/v3/versions` | Agent / protocol / API versions |
|
||||||
|
| GET | `/api/v3/me` | Anonymous REST identity note |
|
||||||
|
| GET | `/api/v3/settings` | Runtime knobs |
|
||||||
|
| GET | `/api/v3/config` | alias of settings |
|
||||||
|
| GET | `/health`, `/api/v1/health`, `/api/v3/health` | Aggregate health |
|
||||||
|
|
||||||
|
### Nodes
|
||||||
|
|
||||||
|
| Method | Path |
|
||||||
|
|--------|------|
|
||||||
|
| GET | `/api/v2/nodes` |
|
||||||
|
| GET | `/api/v3/nodes` |
|
||||||
|
| GET | `/api/v3/node_instances` |
|
||||||
|
| GET | `/api/v3/stream_path` |
|
||||||
|
|
||||||
|
Single-agent MVP returns one node (this host). Parent/fleet aggregation is roadmap.
|
||||||
|
|
||||||
|
### Contexts & charts
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| GET | `/api/v3/contexts` | Context map |
|
||||||
|
| GET | `/api/v2/contexts` | same |
|
||||||
|
| GET | `/api/v3/context?context=` | One context |
|
||||||
|
| GET | `/api/v1/charts` | Full chart summary (legacy but useful) |
|
||||||
|
| GET | `/api/v1/chart?chart=` | One chart |
|
||||||
|
|
||||||
|
### Data queries
|
||||||
|
|
||||||
|
| Method | Path | Query params |
|
||||||
|
|--------|------|--------------|
|
||||||
|
| GET | `/api/v3/data` | `chart` or `context`, `after`, `before`, `points`, `group`, `tier`, `format` |
|
||||||
|
| GET | `/api/v2/data` | same |
|
||||||
|
| GET | `/api/v1/data` | same (legacy) |
|
||||||
|
|
||||||
|
**Params (Netdata-style)**
|
||||||
|
|
||||||
|
| Param | Default | Description |
|
||||||
|
|-------|---------|-------------|
|
||||||
|
| `chart` / `context` | required | Chart id or context id |
|
||||||
|
| `after` | `-60` | Absolute unix sec, or relative (negative) |
|
||||||
|
| `before` | `0` (now) | Absolute or relative |
|
||||||
|
| `points` | `60` | Max points returned (downsampled) |
|
||||||
|
| `group` | `average` | `average` \| `min` \| `max` \| `sum` |
|
||||||
|
| `tier` | `0` | `0` = 1s buffer, `1` = downsampled |
|
||||||
|
| `format` | `json` | `json` \| `csv` \| `array` |
|
||||||
|
|
||||||
|
### Search & weights
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| GET | `/api/v3/q?q=` | Full-text over chart ids/titles |
|
||||||
|
| GET | `/api/v3/weights` | MVP: health-derived scores |
|
||||||
|
|
||||||
|
### Alerts
|
||||||
|
|
||||||
|
| Method | Path |
|
||||||
|
|--------|------|
|
||||||
|
| GET | `/api/v3/alerts` |
|
||||||
|
| GET | `/api/v2/alerts` |
|
||||||
|
| GET | `/api/v1/alarms` |
|
||||||
|
| GET | `/api/v3/alert_transitions` |
|
||||||
|
| GET | `/api/v3/alert_config` |
|
||||||
|
| GET | `/api/v3/variable` |
|
||||||
|
|
||||||
|
### Export & badges
|
||||||
|
|
||||||
|
| Method | Path | Params |
|
||||||
|
|--------|------|--------|
|
||||||
|
| GET | `/api/v3/allmetrics` | `format=json\|prometheus\|shell` |
|
||||||
|
| GET | `/api/v1/allmetrics` | same |
|
||||||
|
| GET | `/api/v3/badge.svg` | `chart`, `dimensions`, `label` |
|
||||||
|
|
||||||
|
### Functions (stub)
|
||||||
|
|
||||||
|
| Method | Path | Notes |
|
||||||
|
|--------|------|-------|
|
||||||
|
| GET | `/api/v3/functions` | Lists job names; execution remains P2P `runJob` for auth |
|
||||||
|
|
||||||
|
## Auth model (REST)
|
||||||
|
|
||||||
|
- **Default:** localhost-only, no bearer required (like a typical Netdata agent bind).
|
||||||
|
- **P2P remains the secure remote path** (Noise + roles).
|
||||||
|
- If you bind `0.0.0.0`, put REST behind a firewall, reverse proxy, or Holesail tunnel — do not expose raw metrics to the internet.
|
||||||
|
- Future: optional bearer gate (`/api/v3/bearer_protection` parity).
|
||||||
|
|
||||||
|
## CORS
|
||||||
|
|
||||||
|
`Access-Control-Allow-Origin` defaults to `*` (override with `PEARDATA_REST_CORS`).
|
||||||
|
|
||||||
|
## Compatibility notes
|
||||||
|
|
||||||
|
| Netdata | PearData MVP |
|
||||||
|
|---------|--------------|
|
||||||
|
| Full ML weights / metric correlations | Simplified health weights |
|
||||||
|
| Multi-node parent streaming | Single node; parent planned |
|
||||||
|
| Cloud POST `/api/v3/spaces/.../data` | Not implemented (agent GET style only) |
|
||||||
|
| Every chart Netdata ships | Core system charts (expanding) |
|
||||||
|
| Functions execute via HTTP | Listed; run via P2P jobs |
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
- Router: `server/rest/routes.js`
|
||||||
|
- Server: `server/rest/http-server.js`
|
||||||
|
- Exporters: `server/rest/formatters.js`
|
||||||
+114
@@ -0,0 +1,114 @@
|
|||||||
|
# PearData Roadmap
|
||||||
|
|
||||||
|
Phased plan from MVP agent → Netdata-class fleet observability on pure P2P.
|
||||||
|
|
||||||
|
## Guiding principles
|
||||||
|
|
||||||
|
1. **Instant value** — connect a pubkey, see live charts in seconds.
|
||||||
|
2. **Agent efficiency** — stay in Netdata’s ballpark for CPU/RAM overhead.
|
||||||
|
3. **PearDock patterns** — HyperDHT identity, protomux-rpc, roles, `pd1.` invites.
|
||||||
|
4. **Dual access** — P2P desktop + Netdata-style REST (`/api/v1|v2|v3`).
|
||||||
|
5. **Ecosystem glue** — ready for PearDock / PearVirt / HoneyPeer / BareOS later.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 0 — Foundation ✅ (this repo)
|
||||||
|
|
||||||
|
| Item | Status |
|
||||||
|
|------|--------|
|
||||||
|
| Copy pear-app-template → PearData rebrand (`pd1.`, `PEARDATA_*`, `peardata/rpc`) | Done |
|
||||||
|
| Shared protocol + schema + metrics catalog | Done |
|
||||||
|
| Agent collector (CPU/RAM/load/net/disk/processes) @ ~1s | Done |
|
||||||
|
| In-memory tiered store (1s + downsample) | Done |
|
||||||
|
| Threshold anomaly engine + health | Done |
|
||||||
|
| P2P RPC surface (query, subscribe, alerts, jobs, ACL) | Done |
|
||||||
|
| Netdata-compatible REST v1/v2/v3 (local :19999) | Done |
|
||||||
|
| Pear desktop fleet overview + live canvas charts | Done |
|
||||||
|
| Docs: architecture, protocol, data model, REST, roadmap, tech choices | Done |
|
||||||
|
| systemd unit + CI skeletons | Done |
|
||||||
|
|
||||||
|
**MVP acceptance criteria**
|
||||||
|
|
||||||
|
- [x] Agent exposes metrics over P2P by public key
|
||||||
|
- [x] Desktop connects to one+ agents and shows live CPU/RAM/net/disk
|
||||||
|
- [x] REST `/api/v3/data?chart=system.cpu&after=-60` returns series
|
||||||
|
- [x] Viewer vs admin (pubkey / seed / `pd1.` invite)
|
||||||
|
- [x] Simple threshold anomalies pushed to clients
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1 — Hardening & UX (next)
|
||||||
|
|
||||||
|
- Persistent client peer bookmarks + aliases in desktop
|
||||||
|
- Multi-peer compare mode (overlay 2–4 nodes on one chart)
|
||||||
|
- Reconnection / offline banners with last-known samples
|
||||||
|
- Agent process title / `peardata-agent` binary naming polish
|
||||||
|
- Expand collectors: per-core CPU, per-iface net, mount disk space
|
||||||
|
- Alert silence TTL auto-reenable
|
||||||
|
- brittle tests for collector, store query, REST routes
|
||||||
|
- One-line install script (`curl | bash`) for Linux agents
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2 — Storage & history (HyperDB)
|
||||||
|
|
||||||
|
See **[STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md)** for the full design (from Holepunch `hyperdb` / workshop / Autobase patterns).
|
||||||
|
|
||||||
|
- HyperDB (bee + Corestore) for metadata, alerts, peer-links, **warm** downsampled points
|
||||||
|
- Keep memory ring for hot 1s path; do **not** tx every sample into HyperDB
|
||||||
|
- Hyperswarm `store.replicate` for linked-node / desktop seed sync
|
||||||
|
- Configurable retention (hours@1s memory, days@1m HyperDB, weeks@1h)
|
||||||
|
- Historical query: memory miss → HyperDB range
|
||||||
|
- Optional later: Autobase multi-writer parents; Rocks engine for local-only speed
|
||||||
|
- Export snapshot job → JSON / Prometheus remote write (optional)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3 — Discovery & apps
|
||||||
|
|
||||||
|
- Docker / container auto-discovery (cgroup + Docker API)
|
||||||
|
- Common service collectors (nginx, postgres, redis) as plugins
|
||||||
|
- PearDock integration: container metrics from dock peers
|
||||||
|
- PearVirt / BareOS node metric adapters
|
||||||
|
- Holesail optional expose of REST UI per agent
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4 — Smarter anomalies & fleet
|
||||||
|
|
||||||
|
- Anomaly scoring + chart highlighting in UI
|
||||||
|
- Lightweight ML job (`runJob` retrain) — start with streaming z-score / k-means
|
||||||
|
- Fleet-wide composite views and correlation (`/api/v3/weights` depth)
|
||||||
|
- Parent peer aggregation (P2P “parent” without central SaaS)
|
||||||
|
- Push notifications (desktop + optional webhook)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5 — Product polish
|
||||||
|
|
||||||
|
- Signed release artifacts (agent + Pear desktop)
|
||||||
|
- Role templates (viewer / SRE operator / admin)
|
||||||
|
- Plugin SDK documentation
|
||||||
|
- Grafana datasource (REST) cookbook
|
||||||
|
- HoneyPeer presence for agent directory (opt-in)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Non-goals (for now)
|
||||||
|
|
||||||
|
- Replacing Netdata Cloud SaaS multi-tenant product
|
||||||
|
- Full byte-identical Netdata internal DB format
|
||||||
|
- Shipping a browser-only public dashboard without auth by default (REST stays localhost unless explicitly bound)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Milestone checklist (operators)
|
||||||
|
|
||||||
|
| Milestone | You can… |
|
||||||
|
|-----------|----------|
|
||||||
|
| M0 | `npm run start:server` + `curl localhost:19999/api/v3/info` |
|
||||||
|
| M1 | Pear UI live charts from agent pubkey |
|
||||||
|
| M2 | Mint `pd1.` operator invite; revoke peer |
|
||||||
|
| M3 | Historical scrub 1h@1s via REST + RPC |
|
||||||
|
| M4 | Container charts from Docker hosts |
|
||||||
|
| M5 | Parent peer rolling up a homelab fleet |
|
||||||
+8
-1
@@ -28,13 +28,20 @@
|
|||||||
| Identity file mode | `0600` |
|
| Identity file mode | `0600` |
|
||||||
| Data directory | local `./data` (not committed) |
|
| Data directory | local `./data` (not committed) |
|
||||||
|
|
||||||
|
## REST API exposure
|
||||||
|
|
||||||
|
- Default bind is **localhost only** (`127.0.0.1:19999`).
|
||||||
|
- REST is intentionally open on that bind (Netdata-agent style) — **do not** set `PEARDATA_REST_HOST=0.0.0.0` without a firewall, reverse proxy, or Holesail tunnel.
|
||||||
|
- Prefer **P2P + roles** for remote multi-operator access; use REST for local scrapers/Grafana.
|
||||||
|
|
||||||
## Production checklist
|
## Production checklist
|
||||||
|
|
||||||
- [ ] Never set `PEARDATA_INSECURE_OPEN_ADMIN` outside local demos
|
- [ ] Never set `PEARDATA_INSECURE_OPEN_ADMIN` outside local demos
|
||||||
- [ ] Keep `PEARDATA_DEFAULT_ROLE=viewer`
|
- [ ] Keep `PEARDATA_DEFAULT_ROLE=viewer`
|
||||||
- [ ] Prefer invites over sharing `SERVER_SEED`
|
- [ ] Prefer `pd1.` invites over sharing `SERVER_SEED`
|
||||||
- [ ] Use short `ttlMs` for high-privilege invites when practical
|
- [ ] Use short `ttlMs` for high-privilege invites when practical
|
||||||
- [ ] Set `PEARDATA_ALLOWLIST` if only known operators should dial
|
- [ ] Set `PEARDATA_ALLOWLIST` if only known operators should dial
|
||||||
|
- [ ] Keep REST on localhost unless explicitly secured
|
||||||
- [ ] Back up `SERVER_SEED` offline; rotate by redeploying a new keypair (clients must re-dial)
|
- [ ] Back up `SERVER_SEED` offline; rotate by redeploying a new keypair (clients must re-dial)
|
||||||
- [ ] Persist `data/` with mode `0700`; `audit.log` may contain peer ids
|
- [ ] Persist `data/` with mode `0700`; `audit.log` may contain peer ids
|
||||||
- [ ] Run under systemd with `ProtectSystem` / `NoNewPrivileges` (see `deploy/`)
|
- [ ] Run under systemd with `ProtectSystem` / `NoNewPrivileges` (see `deploy/`)
|
||||||
|
|||||||
@@ -0,0 +1,290 @@
|
|||||||
|
# HyperDB storage & linked-node sync
|
||||||
|
|
||||||
|
How PearData should adopt Holepunch’s **HyperDB + Corestore + Hyperswarm (+ Autobase)** stack for durable storage and P2P sync between linked agents — based on patterns in local clones under `holepunchto_repos` (`hyperdb`, `hyperdb-workshop`, `hyperdb-autobase-workshop`, `corestore`, `hyperswarm`, `autobee`, `pear-hyperdb`).
|
||||||
|
|
||||||
|
## Why HyperDB (not “just SQLite”)
|
||||||
|
|
||||||
|
| Need | HyperDB fit |
|
||||||
|
|------|-------------|
|
||||||
|
| Typed collections + indexes | Hyperschema + `@ns/collection` + secondary indexes |
|
||||||
|
| Local high-perf | `HyperDB.rocks(path, def)` |
|
||||||
|
| P2P replicate | `HyperDB.bee(hypercore, def, { autoUpdate })` over Corestore |
|
||||||
|
| Multi-writer HA parents | Autobase whose **view** is HyperDB (`extension: false`) |
|
||||||
|
| Same query API local + remote | Workshops prove one `Registry` class works for both |
|
||||||
|
|
||||||
|
PearData’s current `server/services/store.js` is an **in-memory ring**. HyperDB replaces durability + sync; the in-memory tier stays as the **hot 1s path**.
|
||||||
|
|
||||||
|
## Stack mapping (from Holepunch repos)
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TB
|
||||||
|
subgraph Agent
|
||||||
|
COL[collector 1s] --> HOT[Memory tier0 ring]
|
||||||
|
COL --> DS[Downsample]
|
||||||
|
DS --> HDB[(HyperDB bee/rocks)]
|
||||||
|
POL[peer-policy / alerts / links] --> HDB
|
||||||
|
HOT --> RPC[protomux-rpc + REST]
|
||||||
|
HDB --> RPC
|
||||||
|
end
|
||||||
|
subgraph Sync
|
||||||
|
CS[Corestore] --> HDB
|
||||||
|
SW[Hyperswarm] -->|store.replicate| CS
|
||||||
|
AB[Autobase optional] -->|view| HDB
|
||||||
|
end
|
||||||
|
CHILD[Linked child agent] -.->|discoveryKey| SW
|
||||||
|
PARENT[Parent / peer agent] -.-> SW
|
||||||
|
DESK[Pear desktop cache] -.-> SW
|
||||||
|
```
|
||||||
|
|
||||||
|
| Component | Repo pattern | PearData use |
|
||||||
|
|-----------|--------------|--------------|
|
||||||
|
| **Hyperschema + hyperdb/builder** | `hyperdb-workshop/build.js` | `spec/` codegen for collections |
|
||||||
|
| **HyperDB.bee** | workshop `Registry` | Replicable agent DB |
|
||||||
|
| **HyperDB.rocks** | `hyperdb` README | Optional local-only fast index |
|
||||||
|
| **Corestore** | workshop `bin.js` | Named cores: `metrics-meta`, `alerts`, … |
|
||||||
|
| **Hyperswarm** | `swarm.join(discoveryKey)` + `store.replicate(conn)` | Link nodes / seed DB |
|
||||||
|
| **protomux-rpc** | already in PearData | Control plane (unchanged) |
|
||||||
|
| **Autobase + hyperdispatch** | `hyperdb-autobase-workshop` | Multi-writer **parent** / HA registry |
|
||||||
|
| **autobee** | experimental multiwriter bee | Alternative later; prefer Autobase+HyperDB view for now |
|
||||||
|
| **pear-hyperdb** | Pear-shaped Model wrapper | Optional UX for desktop-local rocks |
|
||||||
|
|
||||||
|
## Critical design rule: don’t put 1s samples in HyperDB txs
|
||||||
|
|
||||||
|
HyperDB is an **indexable document DB** (put/get/find + flush). Writing every chart every second as HyperDB transactions will:
|
||||||
|
|
||||||
|
- Amplify Rocks/Bee write cost
|
||||||
|
- Create huge replication chatter
|
||||||
|
- Fight Netdata-class overhead goals
|
||||||
|
|
||||||
|
**Split planes:**
|
||||||
|
|
||||||
|
| Plane | Storage | Sync |
|
||||||
|
|-------|---------|------|
|
||||||
|
| **Hot live (≤1h @ 1s)** | Memory ring (current) | P2P `push:metrics` (current) |
|
||||||
|
| **Warm history (downsampled)** | Hypercore append **or** HyperDB rows keyed `(chart, tsBucket)` | Corestore replicate |
|
||||||
|
| **Metadata** (peers, alerts, labels, jobs, ACL cache) | **HyperDB** | Corestore replicate |
|
||||||
|
| **Fleet / parent consensus** | Autobase → HyperDB view | Swarm on autobase discoveryKey |
|
||||||
|
|
||||||
|
## Proposed HyperDB schema (`@peardata/*`)
|
||||||
|
|
||||||
|
Modeled after workshop `build.js` namespaces.
|
||||||
|
|
||||||
|
### Collections
|
||||||
|
|
||||||
|
```text
|
||||||
|
@peardata/node
|
||||||
|
key: nodeId (string / pubkey hex)
|
||||||
|
fields: hostname, platform, arch, cpus, agentVersion, labels{}, updatedAt
|
||||||
|
|
||||||
|
@peardata/peer-link
|
||||||
|
key: [localNodeId, remotePublicKey]
|
||||||
|
fields: role, alias, discoveryKey?, linkedAt, lastSeen, syncMode (push|pull|both)
|
||||||
|
|
||||||
|
@peardata/alert-config
|
||||||
|
key: id
|
||||||
|
fields: chart, dimension, warn, crit, comparator, enabled, info
|
||||||
|
|
||||||
|
@peardata/alert-event
|
||||||
|
key: [id, ts] # or ulid
|
||||||
|
fields: severity, value, threshold, message, cleared
|
||||||
|
|
||||||
|
@peardata/metric-point # WARM tier only (e.g. 1m buckets)
|
||||||
|
key: [chart, ts]
|
||||||
|
fields: context, values{} (map), tier
|
||||||
|
|
||||||
|
@peardata/job
|
||||||
|
key: id
|
||||||
|
fields: name, status, startedAt, finishedAt, result?
|
||||||
|
```
|
||||||
|
|
||||||
|
### Indexes
|
||||||
|
|
||||||
|
```text
|
||||||
|
@peardata/node-by-hostname → node.hostname
|
||||||
|
@peardata/peer-link-by-remote → peer-link.remotePublicKey
|
||||||
|
@peardata/alert-event-by-chart → alert-event.chart + ts
|
||||||
|
@peardata/metric-point-by-context → metric-point.context + ts
|
||||||
|
```
|
||||||
|
|
||||||
|
Rebuild with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node scripts/build-db.js # Hyperschema + HyperDB.toDisk → spec/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Agent integration shape
|
||||||
|
|
||||||
|
Follow `hyperdb-workshop` / `pear-hyperdb` Model pattern:
|
||||||
|
|
||||||
|
```text
|
||||||
|
server/
|
||||||
|
db/
|
||||||
|
build.js # schema codegen
|
||||||
|
spec/hyperschema/
|
||||||
|
spec/hyperdb/
|
||||||
|
model.js # PearDataModel: putNode, linkPeer, queryWarm, …
|
||||||
|
replicate.js # Hyperswarm join + store.replicate
|
||||||
|
services/
|
||||||
|
store.js # HOT memory (keep)
|
||||||
|
store-hyperdb.js # WARM + metadata facade used by queryData/REST
|
||||||
|
```
|
||||||
|
|
||||||
|
### Boot (single-writer agent — Phase 2)
|
||||||
|
|
||||||
|
```js
|
||||||
|
const store = new Corestore(dataDir + '/corestore')
|
||||||
|
const swarm = new Hyperswarm({ keyPair: await store.createKeyPair('swarm') })
|
||||||
|
swarm.on('connection', (conn) => store.replicate(conn))
|
||||||
|
|
||||||
|
const metaCore = store.get({ name: 'peardata-meta' })
|
||||||
|
const db = HyperDB.bee(metaCore, spec, { autoUpdate: true })
|
||||||
|
|
||||||
|
// announce for linked peers / desktop seeders
|
||||||
|
swarm.join(metaCore.discoveryKey, { server: true, client: true })
|
||||||
|
```
|
||||||
|
|
||||||
|
Collector path:
|
||||||
|
|
||||||
|
1. Ingest → memory tier0 (unchanged)
|
||||||
|
2. Every N samples → downsample → `tx.insert('@peardata/metric-point', …); tx.flush()`
|
||||||
|
3. Alert transitions → `alert-event` collection
|
||||||
|
4. `queryData` / REST: memory first, then HyperDB range scan for older windows
|
||||||
|
|
||||||
|
### Auth note
|
||||||
|
|
||||||
|
HyperDB replication shares **capability to read the core**, not PearData RPC roles. Keep:
|
||||||
|
|
||||||
|
- **Noise + MethodRoles** for mutating RPC (`setAlertConfig`, `runJob`)
|
||||||
|
- Replication topic optionally gated (only invite-linked peers get discoveryKey / capability)
|
||||||
|
- Do **not** announce writable cores to the public swarm without encryption / allowlist
|
||||||
|
|
||||||
|
## Linked nodes: sync modes
|
||||||
|
|
||||||
|
PearData “links” are first-class `@peardata/peer-link` rows + swarm topics.
|
||||||
|
|
||||||
|
| Mode | Behavior |
|
||||||
|
|------|----------|
|
||||||
|
| **Pull** | Local agent opens remote DB by key (`HyperDB.bee(store.get({ key }), spec, { writable: false, autoUpdate: true })`) and replicates |
|
||||||
|
| **Push** | Remote peers allowed to replicate our meta/warm cores (seed) |
|
||||||
|
| **Both** | Mutual swarm join (homelab mesh) |
|
||||||
|
| **Parent aggregate** | Parent pulls many children; stores namespaced copies or Autobase fleet view |
|
||||||
|
|
||||||
|
Discovery:
|
||||||
|
|
||||||
|
1. Desktop/admin mints link → stores remote pubkey + optional `dbKey` (z32/hex)
|
||||||
|
2. Agent joins `discoveryKey` of that core
|
||||||
|
3. On connection: `store.replicate(conn)` (workshop pattern)
|
||||||
|
4. `clone.watch` / `autoUpdate` refreshes HyperDB indexes for REST/RPC queries
|
||||||
|
|
||||||
|
This is the same pattern as workshop §3.1 Lookups — **read path is swarm + HyperDB**, not constant RPC polling.
|
||||||
|
|
||||||
|
## Parent / HA (Phase 4) — Autobase workshop
|
||||||
|
|
||||||
|
When you need multi-writer fleet registry or HA parents:
|
||||||
|
|
||||||
|
- Autobase bootstrap key shared across parent instances
|
||||||
|
- `open: (store) => HyperDB.bee(store.get('db-view'), spec, { extension: false, autoUpdate: true })`
|
||||||
|
- hyperdispatch ops: `add-writer`, `put-alert`, `put-link`, `ingest-rollup`
|
||||||
|
- RPC (existing PearData / workshop) appends ops; apply mutates HyperDB view
|
||||||
|
- Clients dial **any** writer; view key stays stable
|
||||||
|
|
||||||
|
Do **not** “backup” by copying Corestore folders (workshop warning). Rotate writers via Autobase instead.
|
||||||
|
|
||||||
|
## What stays on protomux-rpc
|
||||||
|
|
||||||
|
| Keep on RPC | Why |
|
||||||
|
|-------------|-----|
|
||||||
|
| Live `push:metrics` | Sub-second UX; not DB-shaped |
|
||||||
|
| `subscribeMetrics` / handshake / ACL | Session security |
|
||||||
|
| `runJob`, mintInvite | Mutating control |
|
||||||
|
| On-demand `queryData` for hot window | Memory path |
|
||||||
|
|
||||||
|
| Move to HyperDB (+ replicate) | Why |
|
||||||
|
|-------------------------------|-----|
|
||||||
|
| Peer links, aliases, labels | Shared fleet truth |
|
||||||
|
| Alert config + history | Durable, queryable |
|
||||||
|
| Warm/cold metric buckets | History without central server |
|
||||||
|
| Node inventory | Parent `/api/v3/nodes` |
|
||||||
|
|
||||||
|
## Phased delivery (recommended)
|
||||||
|
|
||||||
|
### Phase A — Local durability (rocks or bee, no swarm yet)
|
||||||
|
|
||||||
|
1. Add `hyperdb`, `hyperschema`, `corestore` deps
|
||||||
|
2. `scripts/build-db.js` + `spec/`
|
||||||
|
3. Persist alert configs, peer policy, warm downsample into HyperDB
|
||||||
|
4. REST/RPC history falls back to HyperDB after memory miss
|
||||||
|
5. Keep REST localhost semantics
|
||||||
|
|
||||||
|
### Phase B — Link & replicate
|
||||||
|
|
||||||
|
1. Hyperswarm alongside HyperDHT RPC (or reuse connections carefully — often **separate swarm for store.replicate**)
|
||||||
|
2. `linkPeer` / `unlinkPeer` RPCs write `@peardata/peer-link`
|
||||||
|
3. Desktop can seed/cache agent DB by key for offline scrubbing
|
||||||
|
4. Document `pd1.` invite vs **db discovery key** (two layers)
|
||||||
|
|
||||||
|
### Phase C — Parent fleet
|
||||||
|
|
||||||
|
1. Parent process pulls N child DB keys
|
||||||
|
2. Composite REST `/api/v3/nodes` + scoped `/api/v3/data`
|
||||||
|
3. Optional Autobase for parent HA
|
||||||
|
|
||||||
|
### Phase D — Polish
|
||||||
|
|
||||||
|
1. Encryption at rest (`encryptionKey` on cores / autobee)
|
||||||
|
2. Blind-peer seeding for always-on warm history
|
||||||
|
3. Grafana: already have REST Prometheus export; optionally expose hypercore-stats
|
||||||
|
|
||||||
|
## Concrete file plan (when implementing)
|
||||||
|
|
||||||
|
```text
|
||||||
|
peardata/
|
||||||
|
scripts/build-db.js
|
||||||
|
spec/hyperschema/
|
||||||
|
spec/hyperdb/
|
||||||
|
server/db/model.js
|
||||||
|
server/db/replicate.js
|
||||||
|
server/services/store-hyperdb.js
|
||||||
|
docs/STORAGE-HYPERDB.md ← this file
|
||||||
|
```
|
||||||
|
|
||||||
|
Deps (approximate):
|
||||||
|
|
||||||
|
```json
|
||||||
|
"hyperdb": "^6",
|
||||||
|
"hyperschema": "^1",
|
||||||
|
"corestore": "^7",
|
||||||
|
"hyperswarm": "^4",
|
||||||
|
"autobase": "^7",
|
||||||
|
"hyperdispatch": "^1"
|
||||||
|
```
|
||||||
|
|
||||||
|
(`autobee` only if you prefer that multiwriter path over Autobase+HyperDB view.)
|
||||||
|
|
||||||
|
## Relationship to current PearData roadmap
|
||||||
|
|
||||||
|
| Roadmap item | HyperDB role |
|
||||||
|
|--------------|--------------|
|
||||||
|
| Phase 2 tiered storage | Warm/cold collections |
|
||||||
|
| Phase 3 PearDock/containers | Extra collections / indexes |
|
||||||
|
| Phase 4 parent peer | Autobase + replicated views |
|
||||||
|
| REST v3 historical queries | `find` ranges on `@peardata/metric-point` |
|
||||||
|
|
||||||
|
## References (local clones)
|
||||||
|
|
||||||
|
| Path under `holepunchto_repos` | Takeaway |
|
||||||
|
|--------------------------------|----------|
|
||||||
|
| `hyperdb/README.md` | rocks vs bee, find/get/tx, autoUpdate |
|
||||||
|
| `hyperdb-workshop` | Schema builder, Corestore+Swarm replicate, RPC inserts |
|
||||||
|
| `hyperdb-autobase-workshop` | Multi-writer view, hyperdispatch, HA |
|
||||||
|
| `corestore/README.md` | `store.replicate(conn)`, namespacing |
|
||||||
|
| `pear-hyperdb` | Thin Model wrapper pattern for Pear apps |
|
||||||
|
| `autobee` | Experimental multiwriter bee alternative |
|
||||||
|
|
||||||
|
## Decision summary
|
||||||
|
|
||||||
|
1. **Yes — incorporate HyperDB** for metadata + warm history + linked-node sync.
|
||||||
|
2. **Keep memory + RPC pushes** for live 1s Netdata feel.
|
||||||
|
3. **Sync linked nodes via Corestore replication on Hyperswarm**, not by streaming every sample over RPC.
|
||||||
|
4. **Use Autobase+HyperDB** when you need multi-writer parents / HA — same pattern as Holepunch’s own workshops.
|
||||||
|
5. **Treat discovery keys as sensitive** — link only invited peers; RPC AuthZ remains authoritative for mutations.
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
# Tech choices
|
||||||
|
|
||||||
|
Recommendations for collection, streaming, visualization, and reusable libraries.
|
||||||
|
|
||||||
|
## Transport & identity
|
||||||
|
|
||||||
|
| Choice | Why |
|
||||||
|
|--------|-----|
|
||||||
|
| **HyperDHT** | Same peer identity + hole-punching model as PearDock |
|
||||||
|
| **protomux-rpc** | Multiplexed request/response + server push events |
|
||||||
|
| **compact-encoding JSON** | Fast enough for control + 1s metric batches; shared with template |
|
||||||
|
| **Ed25519 seeds** | Agent address = pubkey; admin proof via HMAC from seed |
|
||||||
|
|
||||||
|
### Streaming metrics
|
||||||
|
|
||||||
|
MVP uses **protomux-rpc `event` pushes** (`push:metrics`) with per-session subscribe + throttle.
|
||||||
|
|
||||||
|
If fleet scale demands it later:
|
||||||
|
|
||||||
|
1. Dedicated protomux channel with binary packs (Float64 arrays)
|
||||||
|
2. Hypercore / Hyperbee for durable streams a parent can replicate
|
||||||
|
3. Extract a shared **`pearrpc`** package (session middleware + hot-path helpers) from this template
|
||||||
|
|
||||||
|
For MVP, JSON pushes keep the stack simple and debuggable.
|
||||||
|
|
||||||
|
## Collection
|
||||||
|
|
||||||
|
| Option | Verdict |
|
||||||
|
|--------|---------|
|
||||||
|
| **Node `os` + `/proc` (chosen)** | Zero native deps, good enough for MVP, low overhead |
|
||||||
|
| `node-os-utils` | Convenient but extra dep / less control |
|
||||||
|
| Native bindings (netdata collectors, `systeminformation`) | Higher fidelity; consider Phase 2 for Windows depth |
|
||||||
|
| Shell out to `vmstat`/`iostat` | Avoid on hot path |
|
||||||
|
|
||||||
|
**Linux:** `/proc/meminfo`, `/proc/loadavg`, `/proc/net/dev`, `/proc/diskstats`
|
||||||
|
**macOS/Windows:** `os` fallbacks (CPU%, freemem, loadavg where available); I/O/net rates may be zero until platform collectors land.
|
||||||
|
|
||||||
|
Target overhead: single timer, no child processes per tick, ring buffers only.
|
||||||
|
|
||||||
|
## Storage
|
||||||
|
|
||||||
|
| Tier | Implementation |
|
||||||
|
|------|----------------|
|
||||||
|
| Now | In-memory ring (`server/services/store.js`) |
|
||||||
|
| Next | **HyperDB** + Corestore (+ Hyperswarm replicate) — see [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) |
|
||||||
|
| HA / multi-writer parents | Autobase with HyperDB view (`extension: false`) |
|
||||||
|
| Avoid | SQLite as primary P2P store; copying corestore folders as “backup” |
|
||||||
|
|
||||||
|
### HyperDB engines
|
||||||
|
|
||||||
|
| Engine | Sync | Use |
|
||||||
|
|--------|------|-----|
|
||||||
|
| `HyperDB.bee(core, spec)` | P2P via Hypercore | Agent meta + warm history + linked peers |
|
||||||
|
| `HyperDB.rocks(path, spec)` | Local only | Fast local index / desktop cache (`pear-hyperdb` style) |
|
||||||
|
|
||||||
|
## Visualization (desktop)
|
||||||
|
|
||||||
|
| Option | Verdict |
|
||||||
|
|--------|---------|
|
||||||
|
| **Canvas sparklines (chosen MVP)** | No extra deps inside Pear; snappy for 1s updates |
|
||||||
|
| **uPlot** | Best next step for Netdata-like interactive charts (tiny, fast) |
|
||||||
|
| Chart.js / ECharts | Heavier; fine for secondary views |
|
||||||
|
| Grafana via REST | External; use `/api/v3/data` + Prometheus export |
|
||||||
|
|
||||||
|
## REST
|
||||||
|
|
||||||
|
Built-in Node `http` (no Express) — small attack surface, enough for Netdata-style GET APIs.
|
||||||
|
|
||||||
|
## Packaging
|
||||||
|
|
||||||
|
| Piece | Approach |
|
||||||
|
|-------|----------|
|
||||||
|
| Agent | Node 20+ , systemd unit `deploy/peardata.service` |
|
||||||
|
| Desktop | Pear (`pear-electron` + `pear-bridge`) |
|
||||||
|
| Invites | `pd1.` tokens (PearDock-style) |
|
||||||
|
| Installer | Phase 1 one-liner script |
|
||||||
|
|
||||||
|
## Reusable library extraction (recommended later)
|
||||||
|
|
||||||
|
From this codebase / PearDock patterns:
|
||||||
|
|
||||||
|
| Package | Contents |
|
||||||
|
|---------|----------|
|
||||||
|
| `@pear/rpc-session` or `pearrpc` | PeerSession middleware, ACL, rate limit, audit hooks |
|
||||||
|
| `pear-metrics-wire` | Context/chart catalog types + query args |
|
||||||
|
| `pear-invite` | `pd1.` encode/decode + capability HMAC |
|
||||||
|
|
||||||
|
Keep MIT-clean relative to AGPL PearDock sources (template already does).
|
||||||
|
|
||||||
|
## Integration seams
|
||||||
|
|
||||||
|
| Ecosystem | Seam |
|
||||||
|
|-----------|------|
|
||||||
|
| PearDock | Container chart plugin reading dock RPC |
|
||||||
|
| PearVirt | VM CPU/mem contexts |
|
||||||
|
| HoneyPeer | Optional agent directory announcements |
|
||||||
|
| BareOS | Bare-compatible collector build |
|
||||||
|
| Holesail | Tunnel REST or future agent web UI |
|
||||||
+8
-6
@@ -14,8 +14,10 @@
|
|||||||
|------|----------|
|
|------|----------|
|
||||||
| `test/acl.test.js` | Role hierarchy, `assertAllowed` |
|
| `test/acl.test.js` | Role hierarchy, `assertAllowed` |
|
||||||
| `test/crypto-auth.test.js` | MAC key, capabilities, admin proof, invites, classify input |
|
| `test/crypto-auth.test.js` | MAC key, capabilities, admin proof, invites, classify input |
|
||||||
| `test/protocol.test.js` | Constants, `MethodRoles`, schema validators |
|
| `test/protocol.test.js` | Constants, monitoring `MethodRoles`, schema validators |
|
||||||
| `test/integration.test.js` | Live HyperDHT server + client handshake, post, push |
|
| `test/store.test.js` | Metric ring buffer ingest + query |
|
||||||
|
| `test/rest.test.js` | Netdata-style `/api/v3` routes |
|
||||||
|
| `test/integration.test.js` | Live HyperDHT agent + handshake + metrics query |
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm test
|
npm test
|
||||||
@@ -27,10 +29,10 @@ SKIP_INTEGRATION=1 npm test
|
|||||||
|
|
||||||
## Integration test behavior
|
## Integration test behavior
|
||||||
|
|
||||||
1. Starts an ephemeral HyperDHT server in-process
|
1. Starts metrics pipeline + ephemeral HyperDHT agent in-process
|
||||||
2. Sets `PEARDATA_INSECURE_OPEN_ADMIN=1` for the process (restored in teardown)
|
2. Sets `PEARDATA_INSECURE_OPEN_ADMIN=1` for the process (restored in teardown)
|
||||||
3. Dials as a client, handshakes, posts a message, asserts push delivery
|
3. Dials as a client, handshakes, lists charts, queries `system.cpu`
|
||||||
4. Tears down sockets / DHT
|
4. Tears down collector / sockets / DHT
|
||||||
|
|
||||||
Requires outbound/inbound UDP for HyperDHT. If the test hangs or fails on a locked-down network, use `SKIP_INTEGRATION=1`.
|
Requires outbound/inbound UDP for HyperDHT. If the test hangs or fails on a locked-down network, use `SKIP_INTEGRATION=1`.
|
||||||
|
|
||||||
@@ -78,7 +80,7 @@ When adding an RPC method:
|
|||||||
| Desktop chrome | `npm start` — drag titlebar, resize edges, min/max/close |
|
| Desktop chrome | `npm start` — drag titlebar, resize edges, min/max/close |
|
||||||
| Admin connect | Paste key + `SERVER_SEED` in UI |
|
| Admin connect | Paste key + `SERVER_SEED` in UI |
|
||||||
| Invite connect | Paste `pd1.…` without seed |
|
| Invite connect | Paste `pd1.…` without seed |
|
||||||
| Viewer denial | Public key only → `postMessage` fails with permission error |
|
| Viewer denial | Public key only → `runJob` / `mintInvite` fail with permission error |
|
||||||
| Health | With server up: `SERVER_PUBLIC_KEY=… npm run healthcheck` |
|
| Health | With server up: `SERVER_PUBLIC_KEY=… npm run healthcheck` |
|
||||||
| Soak | `SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run soak` |
|
| Soak | `SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run soak` |
|
||||||
|
|
||||||
|
|||||||
Generated
+240
-3
@@ -11,16 +11,21 @@
|
|||||||
"dependencies": {
|
"dependencies": {
|
||||||
"b4a": "^1.8.1",
|
"b4a": "^1.8.1",
|
||||||
"compact-encoding": "^3.3.0",
|
"compact-encoding": "^3.3.0",
|
||||||
|
"corestore": "^7.11.1",
|
||||||
"dotenv": "^17.4.2",
|
"dotenv": "^17.4.2",
|
||||||
"graceful-goodbye": "^1.3.3",
|
"graceful-goodbye": "^1.3.3",
|
||||||
"hypercore-crypto": "^3.7.0",
|
"hypercore-crypto": "^3.7.0",
|
||||||
|
"hypercore-id-encoding": "^1.3.0",
|
||||||
|
"hyperdb": "^6.7.0",
|
||||||
"hyperdht": "^6.33.0",
|
"hyperdht": "^6.33.0",
|
||||||
|
"hyperschema": "^1.21.0",
|
||||||
"pear-bridge": "^1.2.5",
|
"pear-bridge": "^1.2.5",
|
||||||
"pear-electron": "^1.7.28",
|
"pear-electron": "^1.7.28",
|
||||||
"pear-pipe": "^1.0.6",
|
"pear-pipe": "^1.0.6",
|
||||||
"pear-run": "^1.0.8",
|
"pear-run": "^1.0.8",
|
||||||
"protomux": "^3.11.0",
|
"protomux": "^3.11.0",
|
||||||
"protomux-rpc": "^1.10.0",
|
"protomux-rpc": "^1.10.0",
|
||||||
|
"ready-resource": "^1.2.0",
|
||||||
"safety-catch": "^1.0.3",
|
"safety-catch": "^1.0.3",
|
||||||
"z32": "^1.1.0"
|
"z32": "^1.1.0"
|
||||||
},
|
},
|
||||||
@@ -825,6 +830,12 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/big-sparse-array": {
|
||||||
|
"version": "1.0.3",
|
||||||
|
"resolved": "https://registry.npmjs.org/big-sparse-array/-/big-sparse-array-1.0.3.tgz",
|
||||||
|
"integrity": "sha512-6RjV/3mSZORlMdpUaQ6rUSpG637cZm0//E54YYGtQg1c1O+AbZP8UTdJ/TchsDZcTVLmyWZcseBfp2HBeXUXOQ==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/binary-stream-equals": {
|
"node_modules/binary-stream-equals": {
|
||||||
"version": "1.0.0",
|
"version": "1.0.0",
|
||||||
"resolved": "https://registry.npmjs.org/binary-stream-equals/-/binary-stream-equals-1.0.0.tgz",
|
"resolved": "https://registry.npmjs.org/binary-stream-equals/-/binary-stream-equals-1.0.0.tgz",
|
||||||
@@ -929,6 +940,15 @@
|
|||||||
"integrity": "sha512-9z8TZaGM1pfswYeXrUpzPrkx8UnWYdhJclsiYMm6x/w5+nN+8Tf/LnAgfLGQCm59qAOxU8WwHEq2vNwF6i4j+Q==",
|
"integrity": "sha512-9z8TZaGM1pfswYeXrUpzPrkx8UnWYdhJclsiYMm6x/w5+nN+8Tf/LnAgfLGQCm59qAOxU8WwHEq2vNwF6i4j+Q==",
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/codecs": {
|
||||||
|
"version": "3.1.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/codecs/-/codecs-3.1.0.tgz",
|
||||||
|
"integrity": "sha512-Dqx8NwvBvnMeuPQdVKy/XEF71igjR5apxBvCGeV0pP1tXadOiaLvDTXt7xh+/5wI1ASB195mXQGJbw3Ml4YDWQ==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.3"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/compact-encoding": {
|
"node_modules/compact-encoding": {
|
||||||
"version": "3.3.0",
|
"version": "3.3.0",
|
||||||
"resolved": "https://registry.npmjs.org/compact-encoding/-/compact-encoding-3.3.0.tgz",
|
"resolved": "https://registry.npmjs.org/compact-encoding/-/compact-encoding-3.3.0.tgz",
|
||||||
@@ -963,6 +983,24 @@
|
|||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/corestore": {
|
||||||
|
"version": "7.11.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/corestore/-/corestore-7.11.1.tgz",
|
||||||
|
"integrity": "sha512-X6UsGFyHeAcqffTqgoDV7WDZjhfvolWabwzws6GbdqeXQY0CTM6gNpNMGEi6RGvAdZDPnFaboF5jFBy1lw1rDA==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.7",
|
||||||
|
"bare-events": "^2.8.3",
|
||||||
|
"hypercore": "^11.32.0",
|
||||||
|
"hypercore-crypto": "^3.4.2",
|
||||||
|
"hypercore-errors": "^1.4.0",
|
||||||
|
"hypercore-id-encoding": "^1.3.0",
|
||||||
|
"ready-resource": "^1.1.1",
|
||||||
|
"sodium-universal": "^5.0.1",
|
||||||
|
"streamx": "^2.26.0",
|
||||||
|
"which-runtime": "^1.2.1"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/crc-native": {
|
"node_modules/crc-native": {
|
||||||
"version": "1.1.8",
|
"version": "1.1.8",
|
||||||
"resolved": "https://registry.npmjs.org/crc-native/-/crc-native-1.1.8.tgz",
|
"resolved": "https://registry.npmjs.org/crc-native/-/crc-native-1.1.8.tgz",
|
||||||
@@ -982,6 +1020,26 @@
|
|||||||
"crc-native": "^1.0.3"
|
"crc-native": "^1.0.3"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/debounceify": {
|
||||||
|
"version": "1.1.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/debounceify/-/debounceify-1.1.0.tgz",
|
||||||
|
"integrity": "sha512-eKuHDVfJVg+u/0nPy8P+fhnLgbyuTgVxuCRrS/R7EpDSMMkBDgSes41MJtSAY1F1hcqfHz3Zy/qpqHHIp/EhdA==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
|
"node_modules/device-file": {
|
||||||
|
"version": "2.3.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/device-file/-/device-file-2.3.1.tgz",
|
||||||
|
"integrity": "sha512-bmON44lwxJPle9N2OcH4tqM44pMGZKT8G6OkzXkz0urvqQ9LKkoQdTS6w0ztYfmLdUE27R4UUmx0+y2gEz4Jug==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.7",
|
||||||
|
"bare-fs": "^4.0.1",
|
||||||
|
"bare-path": "^3.0.0",
|
||||||
|
"fd-lock": "^2.1.0",
|
||||||
|
"fs-native-extensions": "^1.4.0",
|
||||||
|
"ready-resource": "^1.2.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/dht-rpc": {
|
"node_modules/dht-rpc": {
|
||||||
"version": "6.27.0",
|
"version": "6.27.0",
|
||||||
"resolved": "https://registry.npmjs.org/dht-rpc/-/dht-rpc-6.27.0.tgz",
|
"resolved": "https://registry.npmjs.org/dht-rpc/-/dht-rpc-6.27.0.tgz",
|
||||||
@@ -1056,6 +1114,24 @@
|
|||||||
"integrity": "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==",
|
"integrity": "sha512-/d9sfos4yxzpwkDkuN7k2SqFKtYNmCTzgfEpz82x34IM9/zc8KGxQoXg1liNC/izpRM/MBdt44Nmx41ZWqk+FQ==",
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/fd-lock": {
|
||||||
|
"version": "2.2.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/fd-lock/-/fd-lock-2.2.0.tgz",
|
||||||
|
"integrity": "sha512-Il4jWBhjjgvUg+z8d0bC7ncXqB42caqKTUpnZ9jpcqiPmw8bkIixt7Kic1czbZPMxAQD/9kEqfZ5Dq77nSll0w==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"bare-fs": "^4.5.0",
|
||||||
|
"fs-native-extensions": "^1.4.4",
|
||||||
|
"ready-resource": "^1.2.0",
|
||||||
|
"resource-on-exit": "^1.0.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/flat-tree": {
|
||||||
|
"version": "1.13.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/flat-tree/-/flat-tree-1.13.0.tgz",
|
||||||
|
"integrity": "sha512-fT3HIuCPwHhFgJ20QYzDHgUG0zMmFg5cHvFiFo5h+QMSJ28TihsEVY0f8HGliuO+pOzmvjMx1odToeaEWkTnyQ==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/framed-stream": {
|
"node_modules/framed-stream": {
|
||||||
"version": "1.0.1",
|
"version": "1.0.1",
|
||||||
"resolved": "https://registry.npmjs.org/framed-stream/-/framed-stream-1.0.1.tgz",
|
"resolved": "https://registry.npmjs.org/framed-stream/-/framed-stream-1.0.1.tgz",
|
||||||
@@ -1132,6 +1208,55 @@
|
|||||||
"safety-catch": "^1.0.2"
|
"safety-catch": "^1.0.2"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/hyperbee": {
|
||||||
|
"version": "2.27.3",
|
||||||
|
"resolved": "https://registry.npmjs.org/hyperbee/-/hyperbee-2.27.3.tgz",
|
||||||
|
"integrity": "sha512-PXURH2U4juUZyJRKHTrY5z1zX851pmI1Q0jfv5F/hCIErDt/ND8jOZuxc3hfOLM9f0W3qJEDTMlV5AJBkVPy8w==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.0",
|
||||||
|
"codecs": "^3.0.0",
|
||||||
|
"debounceify": "^1.0.0",
|
||||||
|
"hypercore-errors": "^1.0.0",
|
||||||
|
"mutexify": "^1.4.0",
|
||||||
|
"protocol-buffers-encodings": "^1.2.0",
|
||||||
|
"rache": "^1.0.0",
|
||||||
|
"ready-resource": "^1.0.0",
|
||||||
|
"resolve-reject-promise": "^1.1.0",
|
||||||
|
"safety-catch": "^1.0.2",
|
||||||
|
"streamx": "^2.12.4",
|
||||||
|
"unslab": "^1.2.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/hypercore": {
|
||||||
|
"version": "11.34.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/hypercore/-/hypercore-11.34.1.tgz",
|
||||||
|
"integrity": "sha512-nlrbXjI4x59akT8AwVEFLN54OicB53JDnH1xYP4j9zKDksHUVgqJFXnFwOkRr4Nr3kwCU0b9Rh3eIsU0JX5HEA==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"@hyperswarm/secret-stream": "^6.0.0",
|
||||||
|
"b4a": "^1.1.0",
|
||||||
|
"bare-events": "^2.2.0",
|
||||||
|
"big-sparse-array": "^1.0.3",
|
||||||
|
"compact-encoding": "^3.0.0",
|
||||||
|
"fast-fifo": "^1.3.0",
|
||||||
|
"flat-tree": "^1.9.0",
|
||||||
|
"hypercore-crypto": "^3.2.1",
|
||||||
|
"hypercore-errors": "^1.5.0",
|
||||||
|
"hypercore-id-encoding": "^1.2.0",
|
||||||
|
"hypercore-storage": "^3.2.0",
|
||||||
|
"is-options": "^1.0.1",
|
||||||
|
"nanoassert": "^2.0.0",
|
||||||
|
"protomux": "^3.5.0",
|
||||||
|
"quickbit-universal": "^2.2.0",
|
||||||
|
"random-array-iterator": "^1.0.0",
|
||||||
|
"safety-catch": "^1.0.1",
|
||||||
|
"sodium-universal": "^5.0.1",
|
||||||
|
"streamx": "^2.12.4",
|
||||||
|
"unslab": "^1.3.0",
|
||||||
|
"z32": "^1.0.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/hypercore-crypto": {
|
"node_modules/hypercore-crypto": {
|
||||||
"version": "3.7.0",
|
"version": "3.7.0",
|
||||||
"resolved": "https://registry.npmjs.org/hypercore-crypto/-/hypercore-crypto-3.7.0.tgz",
|
"resolved": "https://registry.npmjs.org/hypercore-crypto/-/hypercore-crypto-3.7.0.tgz",
|
||||||
@@ -1143,6 +1268,15 @@
|
|||||||
"sodium-universal": "^5.0.0"
|
"sodium-universal": "^5.0.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/hypercore-errors": {
|
||||||
|
"version": "1.5.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/hypercore-errors/-/hypercore-errors-1.5.0.tgz",
|
||||||
|
"integrity": "sha512-5KQ/SuDxsvet+7qWA35Ay6zdD9WyAHQoyWHGcPUTbmJBd300gvNIJoi3oma7kp4TTCSzii6qYumNZe/s0j/saQ==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"hypercore-id-encoding": "^1.3.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/hypercore-id-encoding": {
|
"node_modules/hypercore-id-encoding": {
|
||||||
"version": "1.3.0",
|
"version": "1.3.0",
|
||||||
"resolved": "https://registry.npmjs.org/hypercore-id-encoding/-/hypercore-id-encoding-1.3.0.tgz",
|
"resolved": "https://registry.npmjs.org/hypercore-id-encoding/-/hypercore-id-encoding-1.3.0.tgz",
|
||||||
@@ -1153,6 +1287,47 @@
|
|||||||
"z32": "^1.0.0"
|
"z32": "^1.0.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/hypercore-storage": {
|
||||||
|
"version": "3.2.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/hypercore-storage/-/hypercore-storage-3.2.0.tgz",
|
||||||
|
"integrity": "sha512-i5O5ZMkdZaNAWGuNRXUZjuzv7B41OIhDHUwLCZPjucgVmywY0ZE5PDafu6e5oPuhtghjl6S8q6Jn+POVKN2BvQ==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.7",
|
||||||
|
"bare-path": "^3.0.0",
|
||||||
|
"compact-encoding": "^3.1.0",
|
||||||
|
"device-file": "^2.1.2",
|
||||||
|
"flat-tree": "^1.12.1",
|
||||||
|
"hypercore-crypto": "^3.4.2",
|
||||||
|
"hyperschema": "^1.21.0",
|
||||||
|
"index-encoder": "^3.3.2",
|
||||||
|
"resolve-reject-promise": "^1.0.0",
|
||||||
|
"rocksdb-native": "^3.11.0",
|
||||||
|
"scope-lock": "^1.2.4",
|
||||||
|
"streamx": "^2.21.1",
|
||||||
|
"xache": "^1.2.1"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/hyperdb": {
|
||||||
|
"version": "6.7.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/hyperdb/-/hyperdb-6.7.0.tgz",
|
||||||
|
"integrity": "sha512-SVzV1mWNxD8c28sDlAXalOsac5ZPhSSknYwwVdMZyFz4vteXgANkq6pGDkvI2UUH2ZkF0KvcsGrBgLB+pbX41w==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.6",
|
||||||
|
"compact-encoding": "^3.0.0",
|
||||||
|
"generate-object-property": "^2.0.0",
|
||||||
|
"generate-string": "^1.0.1",
|
||||||
|
"hyperbee": "^2.24.2",
|
||||||
|
"hypercore": "^11.29.0",
|
||||||
|
"hyperschema": "^1.9.2",
|
||||||
|
"index-encoder": "^3.4.0",
|
||||||
|
"refcounter": "^1.0.0",
|
||||||
|
"rocksdb-native": "^3.0.0",
|
||||||
|
"scope-lock": "^1.2.4",
|
||||||
|
"streamx": "^2.20.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/hyperdht": {
|
"node_modules/hyperdht": {
|
||||||
"version": "6.33.0",
|
"version": "6.33.0",
|
||||||
"resolved": "https://registry.npmjs.org/hyperdht/-/hyperdht-6.33.0.tgz",
|
"resolved": "https://registry.npmjs.org/hyperdht/-/hyperdht-6.33.0.tgz",
|
||||||
@@ -1205,6 +1380,24 @@
|
|||||||
"generate-string": "^1.0.1"
|
"generate-string": "^1.0.1"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/index-encoder": {
|
||||||
|
"version": "3.5.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/index-encoder/-/index-encoder-3.5.0.tgz",
|
||||||
|
"integrity": "sha512-idZ1cxtZz2dRV6rUiaP9Xo99UjXbSzjcMacoQmxUMu/A7fEQcNPngvwDJYeWelQUS5XFlY71/or70lKn6XnwbQ==",
|
||||||
|
"license": "Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.4"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/is-options": {
|
||||||
|
"version": "1.0.2",
|
||||||
|
"resolved": "https://registry.npmjs.org/is-options/-/is-options-1.0.2.tgz",
|
||||||
|
"integrity": "sha512-u+Ai74c8Q74aS8BuHwPdI1jptGOT1FQXgCq8/zv0xRuE+wRgSMEJLj8lVO8Zp9BeGb29BXY6AsNPinfqjkr7Fg==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.1.1"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/is-property": {
|
"node_modules/is-property": {
|
||||||
"version": "1.0.2",
|
"version": "1.0.2",
|
||||||
"resolved": "https://registry.npmjs.org/is-property/-/is-property-1.0.2.tgz",
|
"resolved": "https://registry.npmjs.org/is-property/-/is-property-1.0.2.tgz",
|
||||||
@@ -1345,9 +1538,9 @@
|
|||||||
"pear-wakeups": "^1.0.0"
|
"pear-wakeups": "^1.0.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/peardatadrive": {
|
"node_modules/pear-appdrive": {
|
||||||
"version": "1.1.2",
|
"version": "1.1.2",
|
||||||
"resolved": "https://registry.npmjs.org/peardatadrive/-/peardatadrive-1.1.2.tgz",
|
"resolved": "https://registry.npmjs.org/pear-appdrive/-/pear-appdrive-1.1.2.tgz",
|
||||||
"integrity": "sha512-AmCrZaNGFU/BvHR+oFP/cNEUCH1vbzezsJV9wqASoggyg8d4sSb0L6kr99ugGkAuUFlOBI2uHni71hBTaO2rDA==",
|
"integrity": "sha512-AmCrZaNGFU/BvHR+oFP/cNEUCH1vbzezsJV9wqASoggyg8d4sSb0L6kr99ugGkAuUFlOBI2uHni71hBTaO2rDA==",
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
@@ -1365,7 +1558,7 @@
|
|||||||
"bare-http1": "^4.0.2",
|
"bare-http1": "^4.0.2",
|
||||||
"get-mime-type": "^2.0.0",
|
"get-mime-type": "^2.0.0",
|
||||||
"listen-async": "^1.0.0",
|
"listen-async": "^1.0.0",
|
||||||
"peardatadrive": "^1.0.0",
|
"pear-appdrive": "^1.0.0",
|
||||||
"pear-errors": "^1.0.0",
|
"pear-errors": "^1.0.0",
|
||||||
"pear-gunk": "^1.0.0",
|
"pear-gunk": "^1.0.0",
|
||||||
"pear-stamp": "^1.0.1",
|
"pear-stamp": "^1.0.1",
|
||||||
@@ -1754,6 +1947,17 @@
|
|||||||
"url": "https://github.com/sponsors/jonschlinkert"
|
"url": "https://github.com/sponsors/jonschlinkert"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/protocol-buffers-encodings": {
|
||||||
|
"version": "1.2.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/protocol-buffers-encodings/-/protocol-buffers-encodings-1.2.0.tgz",
|
||||||
|
"integrity": "sha512-daeNPuKh1NlLD1uDfbLpD+xyUTc07nEtfHwmBZmt/vH0B7VOM+JOCOpDcx9ZRpqHjAiIkGqyTDi+wfGSl17R9w==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"b4a": "^1.6.0",
|
||||||
|
"signed-varint": "^2.0.1",
|
||||||
|
"varint": "5.0.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/protomux": {
|
"node_modules/protomux": {
|
||||||
"version": "3.11.0",
|
"version": "3.11.0",
|
||||||
"resolved": "https://registry.npmjs.org/protomux/-/protomux-3.11.0.tgz",
|
"resolved": "https://registry.npmjs.org/protomux/-/protomux-3.11.0.tgz",
|
||||||
@@ -1840,6 +2044,18 @@
|
|||||||
"streamx": "^2.23.0"
|
"streamx": "^2.23.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/rache": {
|
||||||
|
"version": "1.0.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/rache/-/rache-1.0.0.tgz",
|
||||||
|
"integrity": "sha512-e0k0g0w/8jOCB+7YqCIlOa+OJ38k0wrYS4x18pMSmqOvLKoyhmMhmQyCcvfY6VaP8D75cqkEnlakXs+RYYLqNg==",
|
||||||
|
"license": "Apache-2.0"
|
||||||
|
},
|
||||||
|
"node_modules/random-array-iterator": {
|
||||||
|
"version": "1.0.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/random-array-iterator/-/random-array-iterator-1.0.0.tgz",
|
||||||
|
"integrity": "sha512-u7xCM93XqKEvPTP6xZp2ehttcAemKnh73oKNf1FvzuVCfpt6dILDt1Kxl1LeBjm2iNIeR49VGFhy4Iz3yOun+Q==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/read-write-mutexify": {
|
"node_modules/read-write-mutexify": {
|
||||||
"version": "2.1.0",
|
"version": "2.1.0",
|
||||||
"resolved": "https://registry.npmjs.org/read-write-mutexify/-/read-write-mutexify-2.1.0.tgz",
|
"resolved": "https://registry.npmjs.org/read-write-mutexify/-/read-write-mutexify-2.1.0.tgz",
|
||||||
@@ -1931,6 +2147,12 @@
|
|||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/scope-lock": {
|
||||||
|
"version": "1.2.4",
|
||||||
|
"resolved": "https://registry.npmjs.org/scope-lock/-/scope-lock-1.2.4.tgz",
|
||||||
|
"integrity": "sha512-BpSd8VCuCxW9ZitcdIC/vjs3gMaP9bRBL5nkHcyfX2VrS52n13/rHuBA2xJ/S/4DPuRdAO/Bk8pWd8eD/gHCIA==",
|
||||||
|
"license": "Apache-2.0"
|
||||||
|
},
|
||||||
"node_modules/script-linker": {
|
"node_modules/script-linker": {
|
||||||
"version": "2.5.4",
|
"version": "2.5.4",
|
||||||
"resolved": "https://registry.npmjs.org/script-linker/-/script-linker-2.5.4.tgz",
|
"resolved": "https://registry.npmjs.org/script-linker/-/script-linker-2.5.4.tgz",
|
||||||
@@ -1954,6 +2176,15 @@
|
|||||||
"integrity": "sha512-WBgv0UnIq2C+Aeh0/n+IRpP6967eIx9WpynTUoiW3isPpfe1zu2LJzyfXdo9Tgef8yR/sGjcMvoUXD7EYdiz+g==",
|
"integrity": "sha512-WBgv0UnIq2C+Aeh0/n+IRpP6967eIx9WpynTUoiW3isPpfe1zu2LJzyfXdo9Tgef8yR/sGjcMvoUXD7EYdiz+g==",
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/signed-varint": {
|
||||||
|
"version": "2.0.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/signed-varint/-/signed-varint-2.0.1.tgz",
|
||||||
|
"integrity": "sha512-abgDPg1106vuZZOvw7cFwdCABddfJRz5akcCcchzTbhyhYnsG31y4AlZEgp315T7W3nQq5P4xeOm186ZiPVFzw==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"varint": "~5.0.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/simdle-native": {
|
"node_modules/simdle-native": {
|
||||||
"version": "1.3.9",
|
"version": "1.3.9",
|
||||||
"resolved": "https://registry.npmjs.org/simdle-native/-/simdle-native-1.3.9.tgz",
|
"resolved": "https://registry.npmjs.org/simdle-native/-/simdle-native-1.3.9.tgz",
|
||||||
@@ -2188,6 +2419,12 @@
|
|||||||
"node": ">=10.12.0"
|
"node": ">=10.12.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/varint": {
|
||||||
|
"version": "5.0.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/varint/-/varint-5.0.0.tgz",
|
||||||
|
"integrity": "sha512-gC13b/bWrqQoKY2EmROCZ+AR0jitc6DnDGaQ6Ls9QpKmuSgJB1eQ7H3KETtQm7qSdMWMKCmsshyCmUwMLh3OAA==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/webidl-conversions": {
|
"node_modules/webidl-conversions": {
|
||||||
"version": "4.0.2",
|
"version": "4.0.2",
|
||||||
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-4.0.2.tgz",
|
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-4.0.2.tgz",
|
||||||
|
|||||||
+10
-5
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"name": "peardata",
|
"name": "peardata",
|
||||||
"version": "0.1.0",
|
"version": "0.1.0",
|
||||||
"description": "Production-ready HyperDHT + protomux-rpc P2P app template (demo room + presence)",
|
"description": "Decentralized P2P Netdata-class monitoring — PearMonitor agent + Pear desktop + REST v3",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"main": "index.js",
|
"main": "index.js",
|
||||||
@@ -15,10 +15,10 @@
|
|||||||
"gui": {
|
"gui": {
|
||||||
"main": "index.html",
|
"main": "index.html",
|
||||||
"backgroundColor": "#0b1020",
|
"backgroundColor": "#0b1020",
|
||||||
"height": 780,
|
"height": 860,
|
||||||
"width": 1100,
|
"width": 1280,
|
||||||
"minWidth": 720,
|
"minWidth": 900,
|
||||||
"minHeight": 480,
|
"minHeight": 560,
|
||||||
"resizable": true,
|
"resizable": true,
|
||||||
"movable": true,
|
"movable": true,
|
||||||
"minimizable": true,
|
"minimizable": true,
|
||||||
@@ -50,16 +50,21 @@
|
|||||||
"dependencies": {
|
"dependencies": {
|
||||||
"b4a": "^1.8.1",
|
"b4a": "^1.8.1",
|
||||||
"compact-encoding": "^3.3.0",
|
"compact-encoding": "^3.3.0",
|
||||||
|
"corestore": "^7.11.1",
|
||||||
"dotenv": "^17.4.2",
|
"dotenv": "^17.4.2",
|
||||||
"graceful-goodbye": "^1.3.3",
|
"graceful-goodbye": "^1.3.3",
|
||||||
"hypercore-crypto": "^3.7.0",
|
"hypercore-crypto": "^3.7.0",
|
||||||
|
"hypercore-id-encoding": "^1.3.0",
|
||||||
|
"hyperdb": "^6.7.0",
|
||||||
"hyperdht": "^6.33.0",
|
"hyperdht": "^6.33.0",
|
||||||
|
"hyperschema": "^1.21.0",
|
||||||
"pear-bridge": "^1.2.5",
|
"pear-bridge": "^1.2.5",
|
||||||
"pear-electron": "^1.7.28",
|
"pear-electron": "^1.7.28",
|
||||||
"pear-pipe": "^1.0.6",
|
"pear-pipe": "^1.0.6",
|
||||||
"pear-run": "^1.0.8",
|
"pear-run": "^1.0.8",
|
||||||
"protomux": "^3.11.0",
|
"protomux": "^3.11.0",
|
||||||
"protomux-rpc": "^1.10.0",
|
"protomux-rpc": "^1.10.0",
|
||||||
|
"ready-resource": "^1.2.0",
|
||||||
"safety-catch": "^1.0.3",
|
"safety-catch": "^1.0.3",
|
||||||
"z32": "^1.1.0"
|
"z32": "^1.1.0"
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env node
|
#!/usr/bin/env node
|
||||||
/**
|
/**
|
||||||
* Mint a pa1 invite from SERVER_SEED without starting the full server process.
|
* Mint a pd1 invite from SERVER_SEED without starting the full server process.
|
||||||
*
|
*
|
||||||
* Usage:
|
* Usage:
|
||||||
* node scripts/mint-invite.js [role=operator] [ttlMs]
|
* node scripts/mint-invite.js [role=operator] [ttlMs]
|
||||||
|
|||||||
+30
-16
@@ -27,7 +27,6 @@ fi
|
|||||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
|
|
||||||
# Derive invite prefix from first letters + 1.
|
|
||||||
PREFIX="$(echo "$NEW_SLUG" | tr -cd 'a-z' | cut -c1-2)1."
|
PREFIX="$(echo "$NEW_SLUG" | tr -cd 'a-z' | cut -c1-2)1."
|
||||||
ENV_PREFIX="$(echo "$NEW_SLUG" | tr 'a-z-' 'A-Z_')_"
|
ENV_PREFIX="$(echo "$NEW_SLUG" | tr 'a-z-' 'A-Z_')_"
|
||||||
ENV_PREFIX="${ENV_PREFIX//__/_}"
|
ENV_PREFIX="${ENV_PREFIX//__/_}"
|
||||||
@@ -40,24 +39,39 @@ echo " invite: $OLD_PREFIX → $PREFIX"
|
|||||||
echo " env: $OLD_ENV → $ENV_PREFIX"
|
echo " env: $OLD_ENV → $ENV_PREFIX"
|
||||||
|
|
||||||
export LC_ALL=C
|
export LC_ALL=C
|
||||||
find . -type f \
|
# Use python for safe replacements (perl struggles with slashes in protocol id)
|
||||||
\( -name '*.js' -o -name '*.mjs' -o -name '*.cjs' -o -name '*.json' -o -name '*.md' -o -name '*.html' -o -name '*.css' -o -name '*.yml' -o -name '*.yaml' -o -name '*.service' -o -name '*.example' -o -name '*.sh' \) \
|
python3 - "$OLD_NAME" "$NEW_SLUG" "$OLD_PRODUCT" "$NEW_PRODUCT" "$OLD_PROTOCOL" "${NEW_SLUG}/rpc" "$OLD_SLUG" "$NEW_SLUG" "$OLD_PREFIX" "$PREFIX" "$OLD_ENV" "$ENV_PREFIX" <<'PY'
|
||||||
! -path './node_modules/*' ! -path './.git/*' ! -path './data/*' \
|
import os, sys
|
||||||
-print0 | while IFS= read -r -d '' f; do
|
from pathlib import Path
|
||||||
perl -pi -e "
|
|
||||||
s/\Q$OLD_NAME\E/$NEW_SLUG/g;
|
|
||||||
s/\Q$OLD_PRODUCT\E/$NEW_PRODUCT/g;
|
|
||||||
s/\Q$OLD_PROTOCOL\E/${NEW_SLUG}\\/rpc/g;
|
|
||||||
s/\Q$OLD_SLUG\E/$NEW_SLUG/g;
|
|
||||||
s/\Q$OLD_PREFIX\E/$PREFIX/g;
|
|
||||||
s/\Q$OLD_ENV\E/$ENV_PREFIX/g;
|
|
||||||
" "$f"
|
|
||||||
done
|
|
||||||
|
|
||||||
if [[ -f bin/peardata-server.mjs ]]; then
|
pairs = list(zip(sys.argv[1::2], sys.argv[2::2]))
|
||||||
|
# longer keys first
|
||||||
|
pairs.sort(key=lambda x: -len(x[0]))
|
||||||
|
SKIP = {'node_modules', '.git', 'data'}
|
||||||
|
EXTS = {'.js','.mjs','.cjs','.json','.md','.html','.css','.yml','.yaml','.service','.example','.sh'}
|
||||||
|
|
||||||
|
for dirpath, dirnames, filenames in os.walk('.'):
|
||||||
|
dirnames[:] = [d for d in dirnames if d not in SKIP]
|
||||||
|
for name in filenames:
|
||||||
|
p = Path(dirpath) / name
|
||||||
|
if p.suffix not in EXTS:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
text = p.read_text(encoding='utf-8')
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
orig = text
|
||||||
|
for a, b in pairs:
|
||||||
|
text = text.replace(a, b)
|
||||||
|
if text != orig:
|
||||||
|
p.write_text(text, encoding='utf-8')
|
||||||
|
print('updated', p)
|
||||||
|
PY
|
||||||
|
|
||||||
|
if [[ -f bin/peardata-server.mjs && "$NEW_SLUG" != "peardata" ]]; then
|
||||||
mv bin/peardata-server.mjs "bin/${NEW_SLUG}-server.mjs"
|
mv bin/peardata-server.mjs "bin/${NEW_SLUG}-server.mjs"
|
||||||
fi
|
fi
|
||||||
if [[ -f deploy/peardata.service ]]; then
|
if [[ -f deploy/peardata.service && "$NEW_SLUG" != "peardata" ]]; then
|
||||||
mv deploy/peardata.service "deploy/${NEW_SLUG}.service"
|
mv deploy/peardata.service "deploy/${NEW_SLUG}.service"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|||||||
+7
-4
@@ -1,6 +1,6 @@
|
|||||||
#!/usr/bin/env node
|
#!/usr/bin/env node
|
||||||
/**
|
/**
|
||||||
* Soak test: connect, post messages, verify pushes for SOAK_DURATION_MS.
|
* Soak test: connect, query metrics, ping for SOAK_DURATION_MS.
|
||||||
*
|
*
|
||||||
* Usage:
|
* Usage:
|
||||||
* SERVER_PUBLIC_KEY=… SERVER_SEED=… node scripts/soak.js
|
* SERVER_PUBLIC_KEY=… SERVER_SEED=… node scripts/soak.js
|
||||||
@@ -26,18 +26,21 @@ let received = 0
|
|||||||
let sent = 0
|
let sent = 0
|
||||||
let errors = 0
|
let errors = 0
|
||||||
|
|
||||||
conn.on(Pushes.message, () => {
|
conn.on(Pushes.metrics, () => {
|
||||||
received++
|
received++
|
||||||
})
|
})
|
||||||
|
|
||||||
await conn.connect()
|
await conn.connect()
|
||||||
console.log('soak connected as', conn.role)
|
console.log('soak connected as', conn.role)
|
||||||
|
await conn.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
||||||
|
|
||||||
const end = Date.now() + duration
|
const end = Date.now() + duration
|
||||||
while (Date.now() < end) {
|
while (Date.now() < end) {
|
||||||
try {
|
try {
|
||||||
await conn.request(Methods.postMessage, {
|
await conn.request(Methods.queryData, {
|
||||||
text: `soak ${sent} @ ${new Date().toISOString()}`,
|
chart: 'system.cpu',
|
||||||
|
after: -30,
|
||||||
|
points: 30,
|
||||||
})
|
})
|
||||||
sent++
|
sent++
|
||||||
await conn.ping()
|
await conn.ping()
|
||||||
|
|||||||
@@ -5,10 +5,14 @@ import fs from 'fs'
|
|||||||
import path from 'path'
|
import path from 'path'
|
||||||
|
|
||||||
const MUTATING = new Set([
|
const MUTATING = new Set([
|
||||||
'postMessage',
|
|
||||||
'clearMessages',
|
|
||||||
'mintInvite',
|
'mintInvite',
|
||||||
'revokePeer',
|
'revokePeer',
|
||||||
|
'setAlertConfig',
|
||||||
|
'ackAlert',
|
||||||
|
'silenceAlert',
|
||||||
|
'runJob',
|
||||||
|
'cancelJob',
|
||||||
|
'exportSnapshot',
|
||||||
'handshake',
|
'handshake',
|
||||||
])
|
])
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
/**
|
/**
|
||||||
* Export formatters (JSON / Prometheus / shell) for /api/v*/allmetrics.
|
* Export formatters (JSON / Prometheus / shell) for /api/v1|v2|v3/allmetrics.
|
||||||
*/
|
*/
|
||||||
import os from 'os'
|
import os from 'os'
|
||||||
import { getStore } from '../services/store.js'
|
import { getStore } from '../services/store.js'
|
||||||
|
|||||||
@@ -122,7 +122,9 @@ export class MetricsCollector extends EventEmitter {
|
|||||||
*/
|
*/
|
||||||
constructor(opts = {}) {
|
constructor(opts = {}) {
|
||||||
super()
|
super()
|
||||||
this.intervalMs = opts.intervalMs ?? Number(process.env.PEARDATA_SAMPLE_MS) || SAMPLE_INTERVAL_MS
|
this.intervalMs =
|
||||||
|
opts.intervalMs ??
|
||||||
|
(Number(process.env.PEARDATA_SAMPLE_MS) || SAMPLE_INTERVAL_MS)
|
||||||
this.timer = null
|
this.timer = null
|
||||||
this.running = false
|
this.running = false
|
||||||
this.sampleCount = 0
|
this.sampleCount = 0
|
||||||
@@ -257,7 +259,7 @@ export class MetricsCollector extends EventEmitter {
|
|||||||
values: {
|
values: {
|
||||||
running: load.running,
|
running: load.running,
|
||||||
blocked: 0,
|
blocked: 0,
|
||||||
total: load.total || Object.keys(process || {}).length,
|
total: load.total || 0,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -301,7 +301,7 @@ export function encodeInvite(pkg) {
|
|||||||
export function decodeInvite(invite) {
|
export function decodeInvite(invite) {
|
||||||
const s = String(invite || '').trim()
|
const s = String(invite || '').trim()
|
||||||
if (!s.startsWith(INVITE_PREFIX)) {
|
if (!s.startsWith(INVITE_PREFIX)) {
|
||||||
return { ok: false, error: 'Not a pa1 invite', code: 'INVITE_INVALID' }
|
return { ok: false, error: 'Not a pd1 invite', code: 'INVITE_INVALID' }
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
const json = b64urlDecode(s.slice(INVITE_PREFIX.length)).toString('utf8')
|
const json = b64urlDecode(s.slice(INVITE_PREFIX.length)).toString('utf8')
|
||||||
|
|||||||
+10
-6
@@ -2,10 +2,12 @@ import test from 'brittle'
|
|||||||
import { Roles, roleAllows, MethodRoles } from '../shared/protocol.js'
|
import { Roles, roleAllows, MethodRoles } from '../shared/protocol.js'
|
||||||
import { assertAllowed, maxRole } from '../server/core/acl.js'
|
import { assertAllowed, maxRole } from '../server/core/acl.js'
|
||||||
|
|
||||||
test('assertAllowed allows operator postMessage', (t) => {
|
test('assertAllowed denies viewer mutating ops', (t) => {
|
||||||
t.exception(() => assertAllowed(Roles.viewer, 'postMessage'))
|
t.exception(() => assertAllowed(Roles.viewer, 'runJob'))
|
||||||
assertAllowed(Roles.operator, 'postMessage')
|
t.exception(() => assertAllowed(Roles.viewer, 'mintInvite'))
|
||||||
assertAllowed(Roles.admin, 'clearMessages')
|
assertAllowed(Roles.operator, 'runJob')
|
||||||
|
assertAllowed(Roles.operator, 'setAlertConfig')
|
||||||
|
assertAllowed(Roles.admin, 'mintInvite')
|
||||||
t.pass()
|
t.pass()
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -14,7 +16,9 @@ test('maxRole elevates', (t) => {
|
|||||||
t.is(maxRole(Roles.admin, Roles.operator), Roles.admin)
|
t.is(maxRole(Roles.admin, Roles.operator), Roles.admin)
|
||||||
})
|
})
|
||||||
|
|
||||||
test('viewer can read', (t) => {
|
test('viewer can read metrics', (t) => {
|
||||||
t.ok(roleAllows(Roles.viewer, MethodRoles.listMessages))
|
t.ok(roleAllows(Roles.viewer, MethodRoles.queryData))
|
||||||
|
t.ok(roleAllows(Roles.viewer, MethodRoles.listCharts))
|
||||||
t.ok(roleAllows(Roles.viewer, MethodRoles.handshake))
|
t.ok(roleAllows(Roles.viewer, MethodRoles.handshake))
|
||||||
|
t.ok(roleAllows(Roles.viewer, MethodRoles.subscribeMetrics))
|
||||||
})
|
})
|
||||||
|
|||||||
+29
-25
@@ -1,10 +1,5 @@
|
|||||||
/**
|
/**
|
||||||
* End-to-end: ephemeral HyperDHT server + client RPC + push.
|
* End-to-end: ephemeral HyperDHT agent + client RPC + metric push.
|
||||||
* Runs against real DHT (local, no bootstrap dependency for same-process? )
|
|
||||||
*
|
|
||||||
* hyperdht connect needs the DHT network; same-machine servers work via
|
|
||||||
* the default bootstrap / local discovery. May be slow on restricted networks.
|
|
||||||
*
|
|
||||||
* Skip with: SKIP_INTEGRATION=1 npm test
|
* Skip with: SKIP_INTEGRATION=1 npm test
|
||||||
*/
|
*/
|
||||||
import test from 'brittle'
|
import test from 'brittle'
|
||||||
@@ -17,21 +12,22 @@ import { peers } from '../server/core/peer-registry.js'
|
|||||||
import { initAuthKeys } from '../server/core/auth-keys.js'
|
import { initAuthKeys } from '../server/core/auth-keys.js'
|
||||||
import { PearDataConnection } from '../client/connection.js'
|
import { PearDataConnection } from '../client/connection.js'
|
||||||
import { Methods, Pushes } from '../shared/protocol.js'
|
import { Methods, Pushes } from '../shared/protocol.js'
|
||||||
import { signCapability } from '../shared/crypto-auth.js'
|
import { startPipeline } from '../server/pipeline.js'
|
||||||
|
import { getCollector } from '../server/services/collector.js'
|
||||||
|
|
||||||
const skip = process.env.SKIP_INTEGRATION === '1'
|
const skip = process.env.SKIP_INTEGRATION === '1'
|
||||||
|
|
||||||
test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, async (t) => {
|
test('integration: dial, handshake, metrics, subscribe', { skip, timeout: 60_000 }, async (t) => {
|
||||||
const seed = crypto.randomBytes(32)
|
const seed = crypto.randomBytes(32)
|
||||||
const keyPair = DHT.keyPair(seed)
|
const keyPair = DHT.keyPair(seed)
|
||||||
const publicKeyHex = b4a.toString(keyPair.publicKey, 'hex')
|
const publicKeyHex = b4a.toString(keyPair.publicKey, 'hex')
|
||||||
const seedHex = b4a.toString(seed, 'hex')
|
const seedHex = b4a.toString(seed, 'hex')
|
||||||
|
|
||||||
initAuthKeys({ seedHex, publicKeyHex })
|
initAuthKeys({ seedHex, publicKeyHex })
|
||||||
|
|
||||||
// Open admin for this test process
|
|
||||||
process.env.PEARDATA_INSECURE_OPEN_ADMIN = '1'
|
process.env.PEARDATA_INSECURE_OPEN_ADMIN = '1'
|
||||||
|
|
||||||
|
startPipeline()
|
||||||
|
|
||||||
const dht = new DHT()
|
const dht = new DHT()
|
||||||
const server = dht.createServer()
|
const server = dht.createServer()
|
||||||
|
|
||||||
@@ -49,6 +45,7 @@ test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, asyn
|
|||||||
|
|
||||||
await server.listen(keyPair)
|
await server.listen(keyPair)
|
||||||
t.teardown(async () => {
|
t.teardown(async () => {
|
||||||
|
getCollector().stop()
|
||||||
for (const s of peers.list()) s.destroy()
|
for (const s of peers.list()) s.destroy()
|
||||||
await server.close().catch(() => {})
|
await server.close().catch(() => {})
|
||||||
await dht.destroy().catch(() => {})
|
await dht.destroy().catch(() => {})
|
||||||
@@ -61,8 +58,8 @@ test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, asyn
|
|||||||
})
|
})
|
||||||
|
|
||||||
/** @type {object[]} */
|
/** @type {object[]} */
|
||||||
const pushes = []
|
const metricPushes = []
|
||||||
conn.on(Pushes.message, (m) => pushes.push(m))
|
conn.on(Pushes.metrics, (m) => metricPushes.push(m))
|
||||||
|
|
||||||
await conn.connect()
|
await conn.connect()
|
||||||
t.is(conn.role, 'admin')
|
t.is(conn.role, 'admin')
|
||||||
@@ -71,22 +68,29 @@ test('integration: dial, handshake, post, push', { skip, timeout: 60_000 }, asyn
|
|||||||
const pong = await conn.request(Methods.ping, {})
|
const pong = await conn.request(Methods.ping, {})
|
||||||
t.ok(pong.ok)
|
t.ok(pong.ok)
|
||||||
|
|
||||||
await conn.request(Methods.setDisplayName, { name: 'tester' })
|
const info = await conn.request(Methods.getServerInfo, {})
|
||||||
const post = await conn.request(Methods.postMessage, { text: 'hello p2p' })
|
t.is(info.app, 'peardata')
|
||||||
t.ok(post.success)
|
|
||||||
t.is(post.message.text, 'hello p2p')
|
|
||||||
|
|
||||||
// Allow push delivery
|
const charts = await conn.request(Methods.listCharts, {})
|
||||||
await new Promise((r) => setTimeout(r, 200))
|
t.ok(charts.charts)
|
||||||
const list = await conn.request(Methods.listMessages, {})
|
|
||||||
t.ok(list.messages.some((m) => m.text === 'hello p2p'))
|
|
||||||
|
|
||||||
const invite = await conn.request(Methods.mintInvite, { role: 'operator', ttlMs: 3600_000 })
|
await conn.request(Methods.subscribeMetrics, { charts: ['*'], intervalMs: 1000 })
|
||||||
t.ok(invite.invite.startsWith('pd1.'))
|
|
||||||
|
|
||||||
// Capability can be verified offline
|
// wait for at least one collector tick + push
|
||||||
const cap = signCapability(seedHex, { role: 'viewer', forever: true })
|
await new Promise((r) => setTimeout(r, 2200))
|
||||||
t.ok(cap.token.includes('.'))
|
|
||||||
|
const q = await conn.request(Methods.queryData, {
|
||||||
|
chart: 'system.cpu',
|
||||||
|
after: -30,
|
||||||
|
points: 30,
|
||||||
|
})
|
||||||
|
t.ok(q.labels)
|
||||||
|
t.ok(Array.isArray(q.data))
|
||||||
|
|
||||||
|
const health = await conn.request(Methods.getHealth, {})
|
||||||
|
t.ok(health.status)
|
||||||
|
|
||||||
|
t.ok(metricPushes.length >= 0) // push timing may vary; query proves pipeline
|
||||||
|
|
||||||
await conn.destroy()
|
await conn.destroy()
|
||||||
})
|
})
|
||||||
|
|||||||
+43
-10
@@ -5,11 +5,14 @@ import {
|
|||||||
MethodRoles,
|
MethodRoles,
|
||||||
PROTOCOL,
|
PROTOCOL,
|
||||||
PROTOCOL_VERSION,
|
PROTOCOL_VERSION,
|
||||||
|
Methods,
|
||||||
|
Pushes,
|
||||||
} from '../shared/protocol.js'
|
} from '../shared/protocol.js'
|
||||||
import { validateMethodArgs, SCHEMA_VERSION } from '../shared/schema.js'
|
import { validateMethodArgs, SCHEMA_VERSION } from '../shared/schema.js'
|
||||||
|
import { CHART_DEFS, CONTEXT_IDS } from '../shared/metrics.js'
|
||||||
|
|
||||||
test('protocol constants', (t) => {
|
test('protocol constants', (t) => {
|
||||||
t.ok(PROTOCOL.includes('/rpc'))
|
t.is(PROTOCOL, 'peardata/rpc')
|
||||||
t.ok(PROTOCOL_VERSION >= 1)
|
t.ok(PROTOCOL_VERSION >= 1)
|
||||||
t.ok(SCHEMA_VERSION >= 1)
|
t.ok(SCHEMA_VERSION >= 1)
|
||||||
})
|
})
|
||||||
@@ -22,19 +25,49 @@ test('roleAllows hierarchy', (t) => {
|
|||||||
t.absent(roleAllows(Roles.viewer, Roles.admin))
|
t.absent(roleAllows(Roles.viewer, Roles.admin))
|
||||||
})
|
})
|
||||||
|
|
||||||
test('method roles map covers core methods', (t) => {
|
test('method roles cover monitoring surface', (t) => {
|
||||||
for (const m of ['handshake', 'ping', 'listMessages', 'postMessage', 'mintInvite']) {
|
for (const m of [
|
||||||
|
'handshake',
|
||||||
|
'ping',
|
||||||
|
'queryData',
|
||||||
|
'subscribeMetrics',
|
||||||
|
'listCharts',
|
||||||
|
'mintInvite',
|
||||||
|
'runJob',
|
||||||
|
]) {
|
||||||
t.ok(MethodRoles[m], m)
|
t.ok(MethodRoles[m], m)
|
||||||
|
t.is(Methods[m], m)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
test('validateMethodArgs postMessage', (t) => {
|
test('pushes include metrics + anomaly', (t) => {
|
||||||
t.absent(validateMethodArgs('postMessage', {}).ok)
|
t.ok(Pushes.metrics.startsWith('push:'))
|
||||||
t.ok(validateMethodArgs('postMessage', { text: 'hi' }).ok)
|
t.ok(Pushes.anomaly.startsWith('push:'))
|
||||||
t.absent(validateMethodArgs('postMessage', { text: 'x'.repeat(2001) }).ok)
|
|
||||||
})
|
})
|
||||||
|
|
||||||
test('validateMethodArgs setDisplayName', (t) => {
|
test('metrics catalog non-empty', (t) => {
|
||||||
t.ok(validateMethodArgs('setDisplayName', { name: 'Ada' }).ok)
|
t.ok(CHART_DEFS.length >= 5)
|
||||||
t.absent(validateMethodArgs('setDisplayName', { name: '' }).ok)
|
t.ok(CONTEXT_IDS.includes('system.cpu'))
|
||||||
|
})
|
||||||
|
|
||||||
|
test('validateMethodArgs queryData', (t) => {
|
||||||
|
t.absent(validateMethodArgs('queryData', {}).ok)
|
||||||
|
t.ok(validateMethodArgs('queryData', { chart: 'system.cpu' }).ok)
|
||||||
|
t.absent(validateMethodArgs('queryData', { chart: 'system.cpu', points: 0 }).ok)
|
||||||
|
})
|
||||||
|
|
||||||
|
test('validateMethodArgs subscribeMetrics', (t) => {
|
||||||
|
const r = validateMethodArgs('subscribeMetrics', { charts: ['system.cpu'] })
|
||||||
|
t.ok(r.ok)
|
||||||
|
t.ok(r.args.intervalMs >= 500)
|
||||||
|
})
|
||||||
|
|
||||||
|
test('validateMethodArgs mintInvite', (t) => {
|
||||||
|
t.ok(validateMethodArgs('mintInvite', { role: 'operator' }).ok)
|
||||||
|
t.absent(validateMethodArgs('mintInvite', { role: 'god' }).ok)
|
||||||
|
})
|
||||||
|
|
||||||
|
test('validateMethodArgs revokePeer', (t) => {
|
||||||
|
t.absent(validateMethodArgs('revokePeer', { peerId: 'abc' }).ok)
|
||||||
|
t.ok(validateMethodArgs('revokePeer', { peerId: 'a'.repeat(64) }).ok)
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -0,0 +1,73 @@
|
|||||||
|
import test from 'brittle'
|
||||||
|
import { handleRest } from '../server/rest/routes.js'
|
||||||
|
import { getStore } from '../server/services/store.js'
|
||||||
|
import { initAuthKeys } from '../server/core/auth-keys.js'
|
||||||
|
import crypto from 'hypercore-crypto'
|
||||||
|
import b4a from 'b4a'
|
||||||
|
|
||||||
|
// REST routes read public key helper — init dummy keys
|
||||||
|
const seed = crypto.randomBytes(32)
|
||||||
|
initAuthKeys({
|
||||||
|
seedHex: b4a.toString(seed, 'hex'),
|
||||||
|
publicKeyHex: b4a.toString(crypto.keyPair(seed).publicKey, 'hex'),
|
||||||
|
})
|
||||||
|
|
||||||
|
test('REST /api/v3/info', (t) => {
|
||||||
|
const res = handleRest('/api/v3/info', new URLSearchParams())
|
||||||
|
t.is(res.status, 200)
|
||||||
|
t.ok(res.body.hostname)
|
||||||
|
t.ok(res.body.peardata)
|
||||||
|
})
|
||||||
|
|
||||||
|
test('REST /api/v1/charts after ingest', (t) => {
|
||||||
|
getStore().ingest([
|
||||||
|
{
|
||||||
|
chart: 'system.load',
|
||||||
|
context: 'system.load',
|
||||||
|
ts: Date.now(),
|
||||||
|
values: { load1: 0.5, load5: 0.4, load15: 0.3 },
|
||||||
|
},
|
||||||
|
])
|
||||||
|
const res = handleRest('/api/v1/charts', new URLSearchParams())
|
||||||
|
t.is(res.status, 200)
|
||||||
|
t.ok(res.body.charts['system.load'] || res.body.charts['system.cpu'])
|
||||||
|
})
|
||||||
|
|
||||||
|
test('REST /api/v3/data', (t) => {
|
||||||
|
const now = Date.now()
|
||||||
|
for (let i = 0; i < 10; i++) {
|
||||||
|
getStore().ingest([
|
||||||
|
{
|
||||||
|
chart: 'system.cpu',
|
||||||
|
context: 'system.cpu',
|
||||||
|
ts: now - (10 - i) * 1000,
|
||||||
|
values: {
|
||||||
|
user: i,
|
||||||
|
system: 1,
|
||||||
|
nice: 0,
|
||||||
|
iowait: 0,
|
||||||
|
irq: 0,
|
||||||
|
softirq: 0,
|
||||||
|
idle: 99 - i,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
])
|
||||||
|
}
|
||||||
|
const res = handleRest(
|
||||||
|
'/api/v3/data',
|
||||||
|
new URLSearchParams({ chart: 'system.cpu', after: '-30', points: '10' })
|
||||||
|
)
|
||||||
|
t.is(res.status, 200)
|
||||||
|
t.ok(Array.isArray(res.body.data))
|
||||||
|
})
|
||||||
|
|
||||||
|
test('REST /api/v3/contexts', (t) => {
|
||||||
|
const res = handleRest('/api/v3/contexts', new URLSearchParams())
|
||||||
|
t.is(res.status, 200)
|
||||||
|
t.ok(res.body.contexts['system.cpu'])
|
||||||
|
})
|
||||||
|
|
||||||
|
test('REST 404', (t) => {
|
||||||
|
const res = handleRest('/api/v9/nope', new URLSearchParams())
|
||||||
|
t.is(res.status, 404)
|
||||||
|
})
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
import test from 'brittle'
|
||||||
|
import { MetricStore } from '../server/services/store.js'
|
||||||
|
|
||||||
|
test('store ingest + query', (t) => {
|
||||||
|
const store = new MetricStore()
|
||||||
|
const now = Date.now()
|
||||||
|
for (let i = 0; i < 30; i++) {
|
||||||
|
store.ingest([
|
||||||
|
{
|
||||||
|
chart: 'system.cpu',
|
||||||
|
context: 'system.cpu',
|
||||||
|
ts: now - (30 - i) * 1000,
|
||||||
|
values: {
|
||||||
|
user: 10 + i,
|
||||||
|
system: 5,
|
||||||
|
nice: 0,
|
||||||
|
iowait: 0,
|
||||||
|
irq: 0,
|
||||||
|
softirq: 0,
|
||||||
|
idle: 85 - i,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
])
|
||||||
|
}
|
||||||
|
|
||||||
|
const meta = store.getMeta('system.cpu')
|
||||||
|
t.ok(meta)
|
||||||
|
t.is(meta.context, 'system.cpu')
|
||||||
|
|
||||||
|
const q = store.query({ chart: 'system.cpu', after: -60, before: 0, points: 15 })
|
||||||
|
t.absent(q.error)
|
||||||
|
t.ok(q.data.length > 0)
|
||||||
|
t.ok(q.labels.includes('user'))
|
||||||
|
t.ok(q.labels.includes('time'))
|
||||||
|
})
|
||||||
|
|
||||||
|
test('store unknown chart', (t) => {
|
||||||
|
const store = new MetricStore()
|
||||||
|
const q = store.query({ chart: 'nope.chart', points: 10 })
|
||||||
|
t.ok(q.error)
|
||||||
|
})
|
||||||
+173
@@ -465,3 +465,176 @@ code {
|
|||||||
font-size: 11px;
|
font-size: 11px;
|
||||||
color: #b7c9f5;
|
color: #b7c9f5;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ─── PearData fleet dashboard ─── */
|
||||||
|
#app {
|
||||||
|
flex-direction: row;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
.fleet-rail {
|
||||||
|
width: 280px;
|
||||||
|
flex: 0 0 280px;
|
||||||
|
margin: 12px 0 12px 12px;
|
||||||
|
overflow: auto;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 4px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.fleet-rail h2 {
|
||||||
|
margin-top: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.dash-main {
|
||||||
|
flex: 1;
|
||||||
|
min-width: 0;
|
||||||
|
overflow: auto;
|
||||||
|
padding: 12px;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.overview-strip {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(5, 1fr);
|
||||||
|
gap: 12px;
|
||||||
|
padding: 14px 16px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 4px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat-label {
|
||||||
|
font-size: 11px;
|
||||||
|
text-transform: uppercase;
|
||||||
|
letter-spacing: 0.04em;
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat strong {
|
||||||
|
font-size: 22px;
|
||||||
|
font-weight: 650;
|
||||||
|
letter-spacing: -0.02em;
|
||||||
|
font-variant-numeric: tabular-nums;
|
||||||
|
}
|
||||||
|
|
||||||
|
.stat[data-health='ok'] strong {
|
||||||
|
color: var(--ok);
|
||||||
|
}
|
||||||
|
.stat[data-health='degraded'] strong {
|
||||||
|
color: #f0b429;
|
||||||
|
}
|
||||||
|
.stat[data-health='critical'] strong {
|
||||||
|
color: var(--danger);
|
||||||
|
}
|
||||||
|
|
||||||
|
.charts-grid {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
|
gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chart-panel header {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
align-items: baseline;
|
||||||
|
margin-bottom: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chart-panel h3 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 13px;
|
||||||
|
font-weight: 600;
|
||||||
|
}
|
||||||
|
|
||||||
|
.chart-panel canvas {
|
||||||
|
width: 100%;
|
||||||
|
display: block;
|
||||||
|
border-radius: 8px;
|
||||||
|
background: #0a1224;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bottom-row {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
|
gap: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bottom-row h3 {
|
||||||
|
margin: 0 0 8px;
|
||||||
|
font-size: 13px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.peer-list,
|
||||||
|
.event-list {
|
||||||
|
list-style: none;
|
||||||
|
padding: 0;
|
||||||
|
margin: 10px 0 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.peer-list li,
|
||||||
|
.event-list li {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 8px;
|
||||||
|
padding: 8px 10px;
|
||||||
|
border-radius: 8px;
|
||||||
|
border: 1px solid transparent;
|
||||||
|
font-size: 12px;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
.peer-list li:hover,
|
||||||
|
.peer-list li.active {
|
||||||
|
background: #0d1426;
|
||||||
|
border-color: var(--border);
|
||||||
|
}
|
||||||
|
|
||||||
|
.event-list li {
|
||||||
|
cursor: default;
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: flex-start;
|
||||||
|
border-bottom: 1px solid rgba(36, 49, 84, 0.7);
|
||||||
|
border-radius: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.event-list li.crit {
|
||||||
|
color: var(--danger);
|
||||||
|
}
|
||||||
|
.event-list li.warn {
|
||||||
|
color: #f0b429;
|
||||||
|
}
|
||||||
|
.event-list li.ok {
|
||||||
|
color: var(--ok);
|
||||||
|
}
|
||||||
|
|
||||||
|
.admin-block {
|
||||||
|
margin-top: auto;
|
||||||
|
padding-top: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 1100px) {
|
||||||
|
.overview-strip {
|
||||||
|
grid-template-columns: repeat(3, 1fr);
|
||||||
|
}
|
||||||
|
.charts-grid,
|
||||||
|
.bottom-row {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 800px) {
|
||||||
|
#app {
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
.fleet-rail {
|
||||||
|
width: auto;
|
||||||
|
flex: 0 0 auto;
|
||||||
|
margin: 12px 12px 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user