Updates
CI / test (push) Successful in 58s
Release rolling / release (push) Successful in 7m10s

This commit is contained in:
Raven Scott
2026-07-18 19:16:58 -04:00
parent 1d46b7b3ad
commit 462cce4b5b
24 changed files with 1398 additions and 51 deletions
+17
View File
@@ -87,6 +87,23 @@ Permissions: directory `0700`. Do **not** commit `data/` or `.env`.
| `PEARDATA_POSTGRES_STATS_URL` | — | Optional HTTP key=value stats sidecar |
| `PEARDATA_EXPORT_DIR` | — | Write `snapshot-*.json` from export job/RPC |
| `PEARDATA_PUSHGATEWAY_URL` | — | POST Prometheus text (Pushgateway-compatible) |
| `PEARDATA_WEBHOOK_URL` | — | POST anomaly JSON on alert fire |
| `PEARDATA_WEBHOOK_SECRET` | — | HMAC key for `X-PearData-Signature` |
| `PEARDATA_WEBHOOK_SIGN` | on (if secret) | `0` disables HMAC signing |
| `PEARDATA_WEBHOOK_LEGACY_SECRET` | off | `1` also send plaintext `X-PearData-Secret` |
| `PEARDATA_NOTIFY_CLEARS` | off | `1` also webhook when anomaly clears |
| `PEARDATA_NOTIFY_MIN_SEVERITY` | `warning` | `warning` \| `critical` |
| `PEARDATA_ANOMALY_MODE` | `threshold` | `threshold` \| `zscore` \| `hybrid` |
| `PEARDATA_ANOMALY_WINDOW` | `120` | Rolling samples for z-score / retrain |
| `PEARDATA_ANOMALY_WARN_Z` | `2` | Warn at \|z\| ≥ this |
| `PEARDATA_ANOMALY_CRIT_Z` | `3` | Critical at \|z\| ≥ this |
| `PEARDATA_REST_TUNNEL` | off | `1` expose REST over HyperDHT (holesail-style) |
| `PEARDATA_REST_TUNNEL_SEED` | — | 64-hex seed for stable tunnel public key |
| `PEARDATA_PEARDOCK` | off | `1` enable PearDock bridge (remap remote docker charts) |
| `PEARDATA_PEARDOCK_PEERS` | empty | Comma-separated dock/agent public keys |
| `PEARDATA_PEARDOCK_PEERS_FILE` | — | One pubkey per line |
| `PEARDATA_PEARDOCK_SEED` | — | Optional admin seed for dial |
| `PEARDATA_PEARDOCK_POLL_MS` | `5000` | Bridge poll interval |
Storage: `$PEARDATA_DATA_DIR/corestore` (named core `peardata-meta`).
+29 -2
View File
@@ -47,10 +47,10 @@ Edit `server/rest/routes.js` — keep stable `/api/v3/...` path naming.
Register in `server/services/jobs.js` `JOB_HANDLERS`.
```js
retrainAnomaly: async () => { /* … */ return { ok: true } }
retrainAnomaly: async (args) => getAnomalyEngine().retrain(getStore(), args)
```
Operators run via `runJob` or future REST function execute.
Operators run via `runJob` (`retrainAnomaly` is registered).
## Add a collector plugin (pattern)
@@ -97,6 +97,33 @@ Have `pipeline.js` start each enabled collector; all emit `samples` batches into
- Job `prometheusPush``PEARDATA_PUSHGATEWAY_URL`
- Optional file write: `PEARDATA_EXPORT_DIR`
### Webhook notifications
1. Set `PEARDATA_WEBHOOK_URL=https://…` and optional `PEARDATA_WEBHOOK_SECRET`.
2. When a secret is set, requests include:
- `X-PearData-Timestamp`
- `X-PearData-Signature: sha256=<HMAC-SHA256(secret, `${ts}.${body}`)>`
3. Verify with `verifyWebhookSignature` from `server/services/notify.js`.
4. Desktop: toggle “Desktop notifications” in the Anomalies panel.
### z-score / retrain
1. `PEARDATA_ANOMALY_MODE=zscore|hybrid` for rolling z-score fires.
2. Job `retrainAnomaly` (operator) refits warn/crit from store history (`mean ± z·stdev`).
3. Helpers live in `server/services/zscore.js`.
### REST HyperDHT tunnel (holesail-style)
1. `PEARDATA_REST_TUNNEL=1` (+ optional `PEARDATA_REST_TUNNEL_SEED`).
2. Agent announces a HyperDHT key that proxies TCP → local REST.
3. Inspect: `GET /api/v3/tunnel` or `getServerInfo.restTunnel`.
### PearDock bridge
1. Set `PEARDATA_PEARDOCK=1` and `PEARDATA_PEARDOCK_PEERS=<dockOrAgentPk>,…`.
2. Bridge dials peers, reads `getAllMetrics`, remaps `docker.*``peardock.*`.
3. Does **not** vendor PearDock source (AGPL boundary).
## Parent peer (fleet aggregator)
Shipped opt-in spike:
+1
View File
@@ -64,6 +64,7 @@ Single-agent returns one node. With `PEARDATA_PARENT=1`, `/nodes` and `/fleet` i
| Method | Path | Notes |
|--------|------|-------|
| GET | `/api/v3/export` | JSON snapshot; `?format=prometheus` for text; `?write=1` writes `PEARDATA_EXPORT_DIR` |
| GET | `/api/v3/tunnel` | HyperDHT REST tunnel status / public key |
### Contexts & charts
+13 -7
View File
@@ -69,7 +69,7 @@ Template rebrand, protocol, collector, memory store, anomalies, REST, desktop MV
| Offline banner (last-known samples) | Done |
| Process top-N | Done (opt-in `PEARDATA_PROCESSES=1`) |
| Service plugins | nginx + redis + postgres done |
| PearDock bridge | Container metrics from dock peers |
| PearDock bridge | Done spike (`PEARDATA_PEARDOCK=1`, remap docker→peardock) |
| PearVirt / BareOS adapters | Thin translators into shared contexts |
| Holesail optional REST expose | Tunneled agent API |
@@ -89,10 +89,10 @@ Template rebrand, protocol, collector, memory store, anomalies, REST, desktop MV
| Parent peer aggregation | Done spike (`PEARDATA_PARENT=1`) |
| Fleet REST / RPC | Done (`/api/v3/fleet`, `getFleetHealth`) |
| Desktop fleet strip | Done (`getFleetHealth` + multi-dial peers) |
| Anomaly scoring + UI highlight | Beyond binary warn/crit |
| Streaming z-score / k-means job | `runJob` retrain stub exists |
| Anomaly scoring + UI highlight | Done (score + panel tint + threshold line) |
| Notifications | Done (desktop + `PEARDATA_WEBHOOK_URL`) |
| Streaming z-score / retrain job | Done (`PEARDATA_ANOMALY_MODE`, job `retrainAnomaly`) |
| `/api/v3/weights` depth | Real metric weights |
| Notifications | Desktop + optional webhook |
---
@@ -120,9 +120,15 @@ Template rebrand, protocol, collector, memory store, anomalies, REST, desktop MV
7. ~~Postgres / Redis plugins~~
8. ~~Desktop fleet strip~~
9. ~~Export snapshot / Prometheus push~~
10. **Anomaly scoring UI** — highlight warn/crit on charts
11. **Notifications**desktop + optional webhook
12. **PearDock bridge** — container metrics from dock peers
10. ~~Anomaly scoring UI~~
11. ~~Notifications~~ ✅ (desktop + webhook)
12. ~~PearDock bridge~~ ✅ (`PEARDATA_PEARDOCK=1`)
13. ~~Webhook HMAC signing~~ ✅ (`X-PearData-Signature`)
14. ~~z-score / retrain job~~
15. ~~REST HyperDHT tunnel~~ ✅ (`PEARDATA_REST_TUNNEL=1`)
16. **Autobase multi-writer parents** — HA fleet history
17. **Windows collector depth** — close `/proc`-only gaps
18. **Plugin SDK polish** — public collector registration API
---