Docs

2026-07-18 22:46:24 -04:00
parent 93a239e8ef
commit 7815037737
24 changed files with 576 additions and 16 deletions
@@ -81,7 +81,8 @@ peardata/
 │   ├── rest/                   # agent-compatible HTTP API
 │   └── core/ rpc/ utils/       # Auth, ACL, sessions (template)
 ├── client/                     # Multi-peer connection manager
-├── docs/                       # Full documentation set
+├── user-guide/                 # End-user desktop workflows
+├── docs/                       # Engineer / operator documentation
 ├── deploy/peardata.service     # systemd
 └── test/
 ```
@@ -121,12 +122,15 @@ peardata/

 | Doc | Contents |
 |-----|----------|
+| **[User guide](./user-guide/README.md)** | Desktop workflows — Charts, Metric Correlations, Fleet, Alerts |
 | [Getting started](./docs/GETTING-STARTED.md) | Install, agent, REST, desktop, systemd |
+| [Docs index](./docs/README.md) | Full engineer / operator doc set |
 | [Roadmap](./docs/ROADMAP.md) | MVP → advanced phases |
 | [Architecture](./docs/ARCHITECTURE.md) | PearDock mapping, planes, modules |
 | [Protocol](./docs/PROTOCOL.md) | RPC methods & pushes |
-| [Data model](./docs/DATA-MODEL.md) | Metrics, anomalies, health |
+| [Data model](./docs/DATA-MODEL.md) | Metrics, anomalies, health, weights |
 | [REST API](./docs/REST-API.md) | `/api/v1\|v2\|v3` |
+| [Dashboard](./docs/DASHBOARD.md) | Charts wall / Correlate product notes |
 | [Tech choices](./docs/TECH-CHOICES.md) | Collector, charts, Bare maps |
 | [HyperDB storage](./docs/STORAGE-HYPERDB.md) | Warm history, peer links, swarm sync |
 | [Release](./docs/RELEASE.md) | Binary host matrix + rolling CI |
@@ -149,6 +149,7 @@ A heavier agent may subscribe to child agents over P2P, downsample into its own
 | `server/services/collector.js` | System sampling |
 | `server/services/store.js` | Tiered buffers + query |
 | `server/services/anomaly.js` | Thresholds |
+| `server/services/weights.js` | Metric Correlations scoring (`volume` / `ks2` / …) |
 | `server/services/alerts.js` | Alert CRUD helpers |
 | `server/services/subscriptions.js` | Push fan-out |
 | `server/services/jobs.js` | On-demand jobs |
@@ -163,8 +164,9 @@ HyperDB design: [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md).

 ## Related docs

+- [User guide](../user-guide/README.md) — desktop workflows
 - [PROTOCOL.md](./PROTOCOL.md) — RPC methods & pushes
- [DATA-MODEL.md](./DATA-MODEL.md) — metrics / anomalies / health
+- [DATA-MODEL.md](./DATA-MODEL.md) — metrics / anomalies / health / weights
 - [REST-API.md](./REST-API.md) — `/api/v1|v2|v3`
 - [TECH-CHOICES.md](./TECH-CHOICES.md) — collector & charts
 - [ROADMAP.md](./ROADMAP.md) — phases
@@ -3,7 +3,7 @@
 Product plan for turning the Charts tab into a full host/fleet metrics wall:
 sectioned TOC, every collected chart, synchronized time, and investigation controls.

-Related: [ROADMAP.md](./ROADMAP.md) Phase 6 · [DESKTOP.md](./DESKTOP.md) · [DATA-MODEL.md](./DATA-MODEL.md)
+Related: [ROADMAP.md](./ROADMAP.md) Phase 6 · [DESKTOP.md](./DESKTOP.md) · [DATA-MODEL.md](./DATA-MODEL.md) · **[User guide](../user-guide/README.md)** (operator walkthroughs)

 ---

@@ -154,3 +154,5 @@ Do not name third-party products in code, commits, or user-facing copy.
 | Goal | Find metrics that **changed** vs a baseline around a highlight | Find charts **related** to a seed chart |
 | Engine | Server `weights.js` (`volume` / `ks2` / …) | Client taxonomy + Pearson |
 | UX | Toolbar Correlate → brush ≥15s → Find Correlations → filtered wall | Per-card ⇢ / dblclick → related panel |
+
+Full operator docs: [metric-correlations.md](../user-guide/metric-correlations.md) · [related-metrics.md](../user-guide/related-metrics.md) · [weights-api.md](../user-guide/weights-api.md).
@@ -146,13 +146,45 @@ Aggregate: any CRITICAL → `critical`; else any WARNING → `degraded`; else `o
 }
 ```

+## Weights / Metric Correlations
+
+Engine: `server/services/weights.js`. Not Pearson pairwise correlation — highlight-vs-baseline (or single-window) **scoring**.
+
+| Method | Windows | Score meaning |
+|--------|---------|---------------|
+| `volume` | highlight + baseline | Average change × time above/below baseline |
+| `ks2` | highlight + baseline | Two-sample KS on pairwise diffs |
+| `anomaly-rate` | highlight | Anomaly / alert activity in window |
+| `value` | highlight | Aggregated value (default `time_group=cv`) |
+| `alerts` | none | Legacy active-alert ranking (Related panel boost) |
+
+Default baseline when omitted: ends at highlight start, duration ≈ highlight × 4 (power-of-two snap). Reject if fewer than ~15 usable points.
+
+Response shape:
+
+```json
+{
+  "method": "volume",
+  "view": {
+    "highlight": { "after": 1710000000, "before": 1710000060 },
+    "baseline": { "after": 1710000000, "before": 1710000000 }
+  },
+  "results": [
+    { "id": "system.cpu", "weight": 0.91, "context": "system.cpu", "family": "cpu" }
+  ],
+  "stats": { "scored": 100, "elapsed_ms": 35 }
+}
+```
+
+Desktop UX: [user-guide/metric-correlations.md](../user-guide/metric-correlations.md). Related (Pearson/taxonomy) is separate: `shared/related-metrics.js`.
+
 ## REST ↔ RPC parity

 | Concept | RPC | REST |
 |---------|-----|------|
 | Charts | `listCharts` | `GET /api/v1/charts` |
 | Data | `queryData` | `GET /api/v3/data` |
-| Weights | `getWeights` | `GET /api/v3/weights` |
+| Weights | `getWeights` | `GET /api/v1\|v2\|v3/weights` |
 | Contexts | `listContexts` | `GET /api/v3/contexts` |
 | Nodes | `getNodeInfo` | `GET /api/v3/nodes` |
 | Alerts | `listAlerts` | `GET /api/v3/alerts` |
@@ -14,6 +14,7 @@ The client shell is a **Pear desktop application** built with `pear-electron` +
 | `ui/styles.css` | Layout, theme, **titlebar drag regions** |
 | `client/*` | HyperDHT connection stack used by the UI |
 | `docs/DASHBOARD.md` | Master Charts / metrics wall plan |
+| `user-guide/` | End-user workflows (Correlate, Related, Fleet, …) |

 ```bash
 npm start          # pear run -d .
@@ -188,8 +189,19 @@ Override root with `PEARDATA_HOME`. Writes are atomic (`tmp` → rename, mode `0

 This template ships the **Pear run** path only. For Electron-forge / multi-arch standalone binaries, adapt packaging from a fuller product (e.g. peardock’s forge + bare-standalone scripts) once the app stabilizes.

+## Charts investigation (summary)
+
+| Control | Doc |
+|---------|-----|
+| Metrics wall, time, board, gestures | [user-guide/charts.md](../user-guide/charts.md) · [DASHBOARD.md](./DASHBOARD.md) |
+| **Correlate** (highlight → Find Correlations) | [user-guide/metric-correlations.md](../user-guide/metric-correlations.md) |
+| **Related (⇢)** (taxonomy + Pearson) | [user-guide/related-metrics.md](../user-guide/related-metrics.md) |
+
+Implementation: `ui/dashboard.js`, `server/services/weights.js`, `shared/related-metrics.js`.
+
 ## Related

+- [User guide](../user-guide/README.md)
 - [GETTING-STARTED.md](./GETTING-STARTED.md)
 - [ARCHITECTURE.md](./ARCHITECTURE.md)
 - [CONFIGURATION.md](./CONFIGURATION.md)
@@ -144,6 +144,7 @@ From-source Node agents can still use `npm run start:server` under a custom unit

 ## Next reading

+- **[User guide](../user-guide/README.md)** — Charts, Metric Correlations, Fleet, Alerts
 - [REST-API.md](./REST-API.md)
 - [ARCHITECTURE.md](./ARCHITECTURE.md)
 - [ROADMAP.md](./ROADMAP.md)
@@ -51,10 +51,24 @@ Auth modes at handshake: public key (viewer), capability token / `pd1.` invite,
 | `listCharts` | viewer | — | agent-style chart map |
 | `getChart` | viewer | `{ id }` | Chart summary |
 | `queryData` | viewer | `{ chart, after, before, points, group, tier }` | Time series |
+| `getWeights` | viewer | see below | Metric Correlations / alert weights |
 | `getAllMetrics` | viewer | `{ format: json\|prometheus\|shell }` | Latest export |

 `after` / `before`: absolute unix seconds, or relative (negative = relative to `before`/`now`), agent-style.

+#### `getWeights`
+
+Scores charts for [Metric Correlations](../user-guide/metric-correlations.md) (or legacy alert weights when no highlight window).
+
+| Arg | Notes |
+|-----|--------|
+| `method` | `volume` \| `ks2` \| `anomaly-rate` \| `value` \| `alerts` |
+| `after` / `before` | Highlight window |
+| `baseline_after` / `baseline_before` | Optional baseline (default 4× preceding highlight) |
+| `points`, `time_group`, `contexts`, `charts`, `dimensions`, `limit`, `timeout` | Same as REST |
+
+REST parity: `GET /api/v*/weights`. Engine: `server/services/weights.js`.
+
 ### Live subscriptions

 | Method | Role | Args |
@@ -1,13 +1,22 @@
 # Documentation index

+## End users
+
+| Doc | Contents |
+|-----|----------|
+| **[User guide](../user-guide/README.md)** | Desktop workflows: Connect, Overview, Charts, Metric Correlations, Related, Alerts, Fleet, Settings |
+| [GETTING-STARTED.md](./GETTING-STARTED.md) | Install agent / desktop, first dial, systemd |
+
+## Engineers & operators
+
 | Doc | Audience | Contents |
 |-----|----------|----------|
-| [GETTING-STARTED.md](./GETTING-STARTED.md) | Operators | Install, run agent, REST, desktop, systemd |
 | [ROADMAP.md](./ROADMAP.md) | Everyone | Phased MVP → advanced real-time features |
 | [ARCHITECTURE.md](./ARCHITECTURE.md) | Engineers | Planes, PearDock mapping, module map |
 | [PROTOCOL.md](./PROTOCOL.md) | Protocol owners | RPC methods, pushes, versioning |
-| [DATA-MODEL.md](./DATA-MODEL.md) | Engineers | Metrics, anomalies, alerts, jobs |
+| [DATA-MODEL.md](./DATA-MODEL.md) | Engineers | Metrics, anomalies, alerts, jobs, weights |
 | [REST-API.md](./REST-API.md) | Integrators | agent-style `/api/v1\|v2\|v3` |
+| [DASHBOARD.md](./DASHBOARD.md) | UI developers | Charts wall / Correlate / Related product plan |
 | [TECH-CHOICES.md](./TECH-CHOICES.md) | Engineers | Collector, charts, libraries |
 | [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) | Engineers | HyperDB / Corestore / Autobase sync design |
 | [DESKTOP.md](./DESKTOP.md) | UI developers | Pear shell, titlebar, identity |
@@ -17,5 +26,6 @@
 | [CI.md](./CI.md) | Maintainers | GitHub/Gitea pipelines |
 | [RELEASE.md](./RELEASE.md) | Maintainers | Version, tag, tarball |
 | [EXTENDING.md](./EXTENDING.md) | Product builders | New charts, collectors, parents |
+| [SWAGGER_PLAN.md](./SWAGGER_PLAN.md) | Integrators | Endpoint parity tracker |

-Start here: **[GETTING-STARTED.md](./GETTING-STARTED.md)** · Product overview: **[../README.md](../README.md)**.
+Start here (ops): **[GETTING-STARTED.md](./GETTING-STARTED.md)** · Day-to-day UI: **[User guide](../user-guide/README.md)** · Product overview: **[../README.md](../README.md)**.
@@ -103,7 +103,7 @@ Single-agent returns one node. With `PEARDATA_PARENT=1`, `/nodes` and `/fleet` i
 | Method | Path | Notes |
 |--------|------|-------|
 | GET | `/api/v3/q?q=` | Full-text over chart ids/titles |
-| GET | `/api/v3/weights` | Per-chart anomaly weights (`?chart=&limit=`) |
+| GET | `/api/v1\|v2\|v3/weights` | Metric Correlations + alert weights — see [Weights](#weights--metric-correlations); user guide: [weights-api](../user-guide/weights-api.md) |

 ### Alerts

@@ -141,6 +141,8 @@ Single-agent returns one node. With `PEARDATA_PARENT=1`, `/nodes` and `/fleet` i

 `GET /api/v1/weights` · `/api/v2/weights` · `/api/v3/weights` (same engine; also RPC `getWeights`)

+Desktop walkthrough: [user-guide/metric-correlations.md](../user-guide/metric-correlations.md) · Parameter cheat sheet: [user-guide/weights-api.md](../user-guide/weights-api.md)
+
 | Param | Notes |
 |-------|--------|
 | `method` | `volume` (default with window), `ks2`, `anomaly-rate`, `value`, `alerts` |
@@ -155,6 +157,8 @@ Without a highlight window, `method` defaults to **alerts** (active anomaly/aler

 Response includes `method`, `view.highlight` / `view.baseline`, and `results[{ id, weight, context, family, info }]`. Higher `weight` = more changed / interesting.

+Engine: `server/services/weights.js`. Unit tests: `test/weights.test.js`.
+
 ## CORS

 `Access-Control-Allow-Origin` defaults to `*` (override with `PEARDATA_REST_CORS`).
@@ -194,6 +194,7 @@ time + play/pause; multi-dim series; Overview stays the compact home grid.

 | Doc | Role |
 |-----|------|
+| [User guide](../user-guide/README.md) | End-user desktop workflows (including Metric Correlations) |
 | [DASHBOARD.md](./DASHBOARD.md) | Master Charts / metrics wall plan |
 | [STORAGE-HYPERDB.md](./STORAGE-HYPERDB.md) | Warm store + swarm + Autobase |
 | [CI.md](./CI.md) / [RELEASE.md](./RELEASE.md) | Binary matrix + installer |
@@ -171,10 +171,10 @@ Legend:

 | Op | Path | PearData | Notes |
 |----|------|----------|-------|
-| GET | `/api/v1/weights` | **alias-miss** | Use v2/v3 |
+| GET | `/api/v1/weights` | **done** | Same engine as v2/v3 |
 | GET | `/api/v1/metric_correlations` | **skip** | Deprecated upstream |
-| GET | `/api/v2/weights` | **partial** | Health-derived scores, not full MC/Anomaly Weight API |
-| GET | `/api/v3/weights` | **partial** | Same |
+| GET | `/api/v2/weights` | **done** | MC: `volume` / `ks2` / `anomaly-rate` / `value`; no window → `alerts` |
+| GET | `/api/v3/weights` | **done** | Same; see [REST-API](./REST-API.md) + [user-guide/weights-api](../user-guide/weights-api.md). Multi-node `scope_nodes` / fleet aggregation still out of scope |

 ### B4. Nodes / info / health

@@ -265,7 +265,7 @@ Legend:
 | Prometheus scrape (`allmetrics`) | yes | **done** |
 | Badge SVG | yes | **partial** |
 | Context search (`/q`) | yes | **partial** |
-| Weights / anomaly scoring API | yes | **partial** (health MVP) |
+| Weights / Metric Correlations API | yes | **done** (single-host; no multi-node scope) |
 | v2/v3 `group_by` across nodes | yes (v2+) | **todo** |
 | HTTP function execute | yes | **todo** (P2P today) |
 | Full DynCFG | yes | **skip** (env + jobs) |
@@ -282,14 +282,14 @@ Legend:

 ### P1 — close swagger gaps that matter for tooling

-1. Add **v1 aliases**: `/api/v1/contexts`, `/api/v1/context`, `/api/v1/weights`, `/api/v1/variable` → existing handlers  
+1. Add **v1 aliases**: `/api/v1/contexts`, `/api/v1/context`, `/api/v1/variable` → existing handlers (`/api/v1/weights` already done)  
 2. Add **v2 aliases**: `alert_transitions`, `alert_config`, `node_instances`, `versions`  
 3. Extend `allmetrics`: accept `format=prometheus_all_hosts` as alias of `prometheus`; honor `filter`, `prefix`  
 4. Document response-field parity for Grafana/Prometheus dashboards in `REST-API.md`

 ### P2 — deepen metrics APIs

-1. Richer `/api/v3/weights` (per-chart anomaly scores already in engine)  
+1. ~~Richer `/api/v3/weights`~~ ✅ single-host MC engine (`volume` / `ks2` / `anomaly-rate` / `value`)  
 2. `/api/v1/alarm_count`, `/api/v1/alarms_values`, `/api/v1/alarm_log` thin wrappers  
 3. Optional PUT `/api/v3/settings` (localhost-only)  
 4. HTTP `function` bridge to `runJob` with strict localhost/admin gate  
@@ -362,4 +362,4 @@ curl -s 'http://127.0.0.1:19999/api/v3/allmetrics?format=prometheus' | head
 | Full swagger operation parity (72 ops) | **NO** (~38 matched; rest alias/skip/todo) |
 | Cloud / WebRTC / DynCFG | **N/A by design** |

-**Bottom line:** Every host metric chart PearData defines is collected. Every swagger endpoint required to *discover and scrape those metrics* is implemented on v1/v2/v3 as appropriate. Remaining work is API alias completeness, deeper alert/weights/function semantics, and optional extra collectors — not missing core system metrics.
+**Bottom line:** Every host metric chart PearData defines is collected. Every swagger endpoint required to *discover and scrape those metrics* is implemented on v1/v2/v3 as appropriate. Weights / Metric Correlations are **done** for single-host (`volume` / `ks2` / `anomaly-rate` / `value`). Remaining work is API alias completeness, deeper alert/function semantics, multi-node weight scope, and optional extra collectors — not missing core system metrics.
@@ -16,6 +16,7 @@
 | `test/crypto-auth.test.js` | MAC key, capabilities, admin proof, invites, classify input |
 | `test/protocol.test.js` | Constants, monitoring `MethodRoles`, schema validators |
 | `test/store.test.js` | Metric ring buffer ingest + query |
+| `test/weights.test.js` | Metric Correlations engine (`volume` / `ks2` / windows / errors) |
 | `test/hyperdb.test.js` | HyperDB model (node, links, warm points, alerts) |
 | `test/rest.test.js` | agent-style `/api/v3` routes |
 | `test/integration.test.js` | Live HyperDHT agent + handshake + metrics query |
@@ -84,6 +85,8 @@ When adding an RPC method:
 | Viewer denial | Public key only → `runJob` / `mintInvite` fail with permission error |
 | Health | With server up: `SERVER_PUBLIC_KEY=… npm run healthcheck` |
 | Soak | `SERVER_PUBLIC_KEY=… SERVER_SEED=… npm run soak` |
+| Metric Correlations | Charts → Correlate → brush ≥15s → Find Correlations → wall filters; Clear restores; Related ⇢ still works |
+| Weights REST | `curl -sG 'http://127.0.0.1:19999/api/v3/weights' --data-urlencode 'method=volume' --data-urlencode 'after=-60' --data-urlencode 'before=0'` |

 ## Soak test

@@ -0,0 +1,34 @@
+# PearData user guide
+
+End-user documentation for the **Pear desktop client** and day-to-day monitoring workflows.
+
+For install and agent ops, see [Getting started](../docs/GETTING-STARTED.md).  
+For engineers (protocol, REST deep-dive, architecture), see [docs/](../docs/README.md).
+
+---
+
+## Contents
+
+| Guide | What you’ll learn |
+|-------|-------------------|
+| [Overview](./overview.md) | What PearData is, planes (agent / desktop / REST), mental model |
+| [Connect](./connect.md) | Public key, invites, admin seed, restoring saved agents |
+| [Overview tab](./overview-tab.md) | KPIs, spark charts, fleet strip, spotlight |
+| [Charts](./charts.md) | Metrics wall, time bar, pins, board, gestures |
+| [Metric Correlations](./metric-correlations.md) | Highlight → Find Correlations → filtered wall (full walkthrough) |
+| [Related metrics](./related-metrics.md) | Per-chart ⇢ related (taxonomy + Pearson) — different from Correlate |
+| [Alerts](./alerts.md) | Anomaly list, Show / Correlate actions |
+| [Fleet](./fleet.md) | Multi-host roster, set active, reconnect, forget |
+| [Settings](./settings.md) | Theme, motion, spark depth, reconnect |
+| [Keyboard shortcuts](./keyboard.md) | Charts and shell shortcuts |
+| [Weights API](./weights-api.md) | REST / RPC for Metric Correlations from scripts |
+
+---
+
+## Quick path
+
+1. [Connect](./connect.md) an agent  
+2. Skim [Overview](./overview-tab.md)  
+3. Open **Charts** — [Charts guide](./charts.md)  
+4. Investigate with [Metric Correlations](./metric-correlations.md) or [Related](./related-metrics.md)  
+5. Manage hosts in [Fleet](./fleet.md)
@@ -0,0 +1,32 @@
+# Alerts
+
+The Alerts tab lists recent **anomaly** and threshold-style events from the active agent.
+
+## Event row
+
+Typical fields:
+
+- Timestamp  
+- Chart / context  
+- Severity or status  
+- Short message  
+
+## Actions
+
+| Action | Effect |
+|--------|--------|
+| **Show** | Switch to Charts, pause near the event time |
+| **Correlate** | Switch to Charts, brush a highlight around the event, run Metric Correlations with **`anomaly-rate`** |
+
+Full MC workflow: [Metric Correlations](./metric-correlations.md).
+
+## Empty list
+
+No events yet, or the anomaly engine has not raised anything. Confirm the agent has been collecting for a while and that health is not stuck in a failed state (Overview KPI / `GET /api/v3/health`).
+
+## REST / RPC
+
+- `GET /api/v3/alerts`  
+- RPC `listAlerts`  
+
+See [docs/REST-API.md](../docs/REST-API.md) and [docs/PROTOCOL.md](../docs/PROTOCOL.md).
@@ -0,0 +1,53 @@
+# Charts tab
+
+The Charts tab is the **metrics wall**: every chart the agent catalogs, grouped by taxonomy, with one shared time window.
+
+## Layout
+
+| Region | Role |
+|--------|------|
+| **Toolbar** | Live/Pause, Force, Board, Correlate, Reset, time presets, Group, Sort |
+| **Correlate bar** | Appears in Correlate mode — method, Find Correlations, Clear |
+| **Related / MC panels** | Ranked lists above the wall |
+| **TOC** | Search + jump to sections |
+| **Wall** | Scrollable cards (only this pane scrolls on Charts) |
+
+## Time controls
+
+| Control | Meaning |
+|---------|---------|
+| **1m … 6h** | Shared window length (live edge when playing) |
+| **Pause / Play** | Stop or resume live advance (`Space`) |
+| **Reset** | Back to live 5m |
+| **Group** | Downsample aggregation: average / min / max / sum |
+| **thin-history** (preset style) | Advisory — buffered history may be shorter than the window |
+
+Retention never hard-locks presets once an agent is connected; longer windows may look sparse until samples accumulate.
+
+## Gestures
+
+| Gesture | Action |
+|---------|--------|
+| Scroll | Browse the wall (normal) |
+| **Ctrl / ⌘ / Shift + scroll** | Zoom the shared window |
+| Horizontal drag | Pan (pause + scrub history) — not in Correlate mode |
+| **Correlate mode + drag** | Brush highlight band |
+| Click chart title | Expand min/avg/max stats |
+| ⇢ | [Related metrics](./related-metrics.md) |
+| ★ / ☆ | Pin / unpin |
+| Type button | Cycle line / area / stacked / bar / pie |
+| Resize handle | Card height (double-click resets) |
+
+## Board mode
+
+**Board** shows **pinned charts only**. Pin with ☆, reorder by drag in the Pinned section, then toggle Board. Clear Board to return to the full wall.
+
+## Force (wallboard)
+
+**Force** keeps the wall live even when you would otherwise pause — useful for always-on displays.
+
+## Correlate
+
+See the full guide: [Metric Correlations](./metric-correlations.md).
+
+Shortcut: **`c`** toggles Correlate mode; **Esc** clears correlation results (or Related).
@@ -0,0 +1,36 @@
+# Connect
+
+## Ways to connect
+
+| Method | Role | How |
+|--------|------|-----|
+| **64-hex public key** | Viewer | Paste on **Connect** → Connect |
+| **`pd1.` invite** | Granted role (viewer / operator / admin) | Mint with `npm run mint-invite -- <role>` on the agent host |
+| **Admin seed** | Admin | Paste `SERVER_SEED` from the agent `.env` (optional field on Connect) |
+
+Optional **alias** (e.g. `homelab-1`) labels the host in Fleet and the titlebar chip.
+
+## First connection checklist
+
+1. Agent is running (`npm run start:server` or systemd).  
+2. Note the printed **publicKey**.  
+3. Open the desktop (`npm start`).  
+4. **Connect** → paste key or invite → Connect.  
+5. Titlebar shows the active peer; Overview and Charts start filling.
+
+## Switching agents
+
+- **Fleet** → **Set active** on a live card, or **Reconnect** / **Charts** on a saved card.  
+- Overview fleet strip can also set active when multi-host is listed.  
+- Only the **active** agent feeds the Charts wall and Overview sparks.
+
+## Offline / reconnect
+
+Saved agents retry dial automatically (see Settings → max reconnect attempts).  
+Failed peers show in Fleet as **Failed**; use **Reconnect** or **Forget**.
+
+## Security notes
+
+- Viewer dials are read-only (metrics subscribe / query).  
+- Invites and admin seed are secrets — treat like passwords.  
+- REST binds to localhost by default; do not expose without a reverse proxy and auth.
@@ -0,0 +1,29 @@
+# Fleet
+
+Fleet is the multi-host roster: **saved** agents plus **live** connections.
+
+## Cards
+
+Each host shows:
+
+- Alias / public key short form  
+- Status: Live, Saved, Connecting, Failed  
+- Optional short metrics when live  
+- Actions: **Set active**, **Reconnect**, **Charts**, **Forget**
+
+## Set active
+
+Only the **active** agent drives Overview sparks, Charts wall, and Alerts. Use **Set active** (or Overview fleet strip) to switch.
+
+## Reconnect / Forget
+
+- **Reconnect** dials again (respects Settings → max reconnect attempts).  
+- **Forget** removes the saved peer from local config (does not stop the remote agent).
+
+## Empty state
+
+If nothing is saved yet, Fleet offers a CTA to **Connect**. After your first successful dial, peers persist when **Restore saved agents** is on (Settings).
+
+## Parent / multi-node (later)
+
+Single-host MC and Charts are first-class today. Fleet aggregation / parent-scope Metric Correlations may expand later; see [docs/ROADMAP.md](../docs/ROADMAP.md).
@@ -0,0 +1,12 @@
+# Keyboard shortcuts
+
+Shortcuts apply when the Charts (or shell) focus is not in a text field.
+
+| Key | Action |
+|-----|--------|
+| **Space** | Pause / play shared Charts time |
+| **c** | Toggle **Correlate** mode |
+| **Esc** | Clear Metric Correlation results, or dismiss Related panel |
+| **/** or focus search | Jump to Charts TOC search (when available) |
+
+Mouse / trackpad gestures for pan, zoom, and brush are documented in [Charts](./charts.md).
@@ -0,0 +1,72 @@
+# Metric Correlations
+
+**Metric Correlations** finds charts that **changed the most** between a **highlight** time range and a **baseline** (normally four times as long, immediately before the highlight). Results replace the Charts wall with a ranked, filtered set.
+
+This is **not** the same as [Related metrics](./related-metrics.md) (⇢), which finds charts similar to a seed by taxonomy / Pearson correlation.
+
+## When to use it
+
+- Something looks wrong in a spike — highlight that spike and find what else moved.  
+- From an [alert](./alerts.md), jump to **Correlate** around the event time.  
+- Compare methods: volume (default), statistical KS2, anomaly-rate, or value/CV (unstable metrics).
+
+## Walkthrough (desktop)
+
+1. Open **Charts**.  
+2. Click **Correlate** (or press **`c`**). The correlate bar appears.  
+3. Optionally set **Method** (`volume`, `ks2`, `anomaly-rate`, `value (cv)`).  
+4. On any chart, **drag horizontally** across the period of interest (≥ **15 seconds**).  
+   - A green band shows the highlight.  
+   - Hint text shows highlight length and the auto baseline (~4× before).  
+5. Click **Find Correlations**.  
+6. The wall filters to the top scored charts; each card shows a **weight %** chip. A results panel lists ranks — click a row to scroll to that card.  
+7. **Clear** (or **Esc**) restores the full wall and exits Correlate mode.
+
+### Requirements
+
+| Rule | Detail |
+|------|--------|
+| Highlight ≥ 15s | Shorter brushes are rejected |
+| Agent connected | Catalog + samples in memory/warm store |
+| Enough points | Engine needs usable samples in highlight (and baseline for `volume` / `ks2`) |
+
+## Methods
+
+| Method | Needs baseline? | Meaning |
+|--------|-----------------|---------|
+| **volume** (default) | Yes | How much the average moved × fraction of time the highlight sits above/below the baseline average |
+| **ks2** | Yes | Two-sample Kolmogorov–Smirnov on pairwise diffs of the series — statistical “distribution changed” |
+| **anomaly-rate** | No | How much anomaly / alert activity fell inside the highlight |
+| **value (cv)** | No | Coefficient of variation (or other `time_group`) over the highlight — “unstable” metrics |
+
+Baseline when omitted: ends at highlight start, duration = highlight × **4**, snapped to a power-of-two multiple of the highlight length (same rules as the agent weights engine).
+
+## From Alerts
+
+On the Alerts list, each event with a chart id offers:
+
+- **Show** — jump to Charts and pause near the event  
+- **Correlate** — open Charts, set highlight around the event (±60s by default), run **`anomaly-rate`**
+
+## Reading results
+
+- **Higher weight** = more changed / more interesting for the chosen method.  
+- Weights are spread onto (0, 1] for ranking; the chip shows percent.  
+- Empty results: try a wider highlight, another method, or wait for more history.
+
+## API / automation
+
+Scripts and integrators use the same engine:
+
+```http
+GET /api/v3/weights?method=volume&after=<unix>&before=<unix>&limit=50
+```
+
+RPC: `getWeights` with the same fields. Full parameter list: [Weights API](./weights-api.md) and [docs/REST-API.md](../docs/REST-API.md).
+
+## Tips
+
+- Zoom or pan the wall first so the brush sits on a meaningful window.  
+- Use **volume** for “what got busier/quieter”; **ks2** when shape/distribution matters; **anomaly-rate** right after an alert.  
+- Related (⇢) is still useful after MC to explore neighbors of a top result.  
+- Do not confuse **Board** (pins) with **Correlate** (scored filter).
@@ -0,0 +1,32 @@
+# Overview tab
+
+Compact home dashboard for the **active** agent.
+
+## KPIs
+
+Top strip shows live scalars (CPU, RAM, load, net, health). Health tint follows anomaly engine status (ok / degraded / critical).
+
+## Spark panels
+
+Six fixed panels (CPU, RAM, net, I/O, load, spotlight/explore):
+
+- Heights grow modestly with window size (`clamp`) so the pane prefers to fit without scrolling.  
+- Series depth follows the Charts timeframe when Charts has an active window; otherwise Settings → **Overview spark depth**.
+
+## Fleet strip
+
+When you have saved / linked hosts (or a parent fleet), a summary strip appears:
+
+- Per-host status and short metrics when available  
+- Click through to activate / connect  
+
+Full management lives on the [Fleet](./fleet.md) tab.
+
+## Spotlight / explore
+
+The explore panel tracks a configurable chart id (Settings → **Default spotlight chart**, default `system.io`). Change it from Overview controls when present, or Settings.
+
+## Tips
+
+- Overview is for **at-a-glance**; deep investigation belongs on **Charts**.  
+- If Overview feels cramped on a short display, the view may scroll — widen the window or collapse secondary chrome.
@@ -0,0 +1,45 @@
+# Overview
+
+PearData is **decentralized, P2P, real-time host monitoring** for the Pear / Holepunch ecosystem.
+
+## Three planes
+
+| Plane | Role |
+|-------|------|
+| **Agent** (`peardata-server`) | Collects metrics (~1s), stores hot/warm history, scores anomalies, serves P2P RPC + local REST |
+| **Desktop** (Pear / Electron) | Connects to one or more agents, shows Overview / Charts / Alerts / Fleet / Settings |
+| **REST** (`127.0.0.1:19999`) | Agent-compatible HTTP for scripts, Grafana, Prometheus |
+
+You dial agents by **public key** (viewer) or **`pd1.` invite** / admin seed for elevated roles.
+
+## Desktop tabs
+
+| Tab | Purpose |
+|-----|---------|
+| **Overview** | Compact home: KPIs + six spark panels + optional fleet strip |
+| **Charts** | Full metrics wall — every catalog chart, shared time, investigation |
+| **Alerts** | Recent anomaly / threshold events |
+| **Fleet** | Saved + live agents; set active, reconnect, open Charts |
+| **Connect** | Dial a new key or invite |
+| **Settings** | Theme, notifications, history depth, reconnect |
+
+## Investigation tools (don’t confuse them)
+
+| Tool | Where | Job |
+|------|-------|-----|
+| **Correlate** | Charts toolbar | “What **changed** around this highlight vs baseline?” |
+| **Related (⇢)** | Per chart card | “What charts are **related** to this one?” (context / Pearson) |
+| **Board** | Charts toolbar | Show **pinned** charts only |
+| **Force** | Charts toolbar | Keep live updates (wallboard) even if paused elsewhere |
+
+See [Metric Correlations](./metric-correlations.md) and [Related metrics](./related-metrics.md).
+
+## Persistence
+
+Desktop state lives under `~/.config/peardata` (or `PEARDATA_HOME`):
+
+- Saved peers / last active agent  
+- Settings (theme, chart prefs, reconnect)  
+- Metrics wall prefs (pins, card heights, chart types)
+
+Agents restore on launch when **Restore saved agents** is enabled (Settings → Connections).
@@ -0,0 +1,30 @@
+# Related metrics
+
+The **Related** helper answers: “Given this chart, what else should I look at?”
+
+It is **client-side** ranking (taxonomy + optional Pearson on loaded series + alert-weight boost). It does **not** replace [Metric Correlations](./metric-correlations.md).
+
+## How to open
+
+- Click **⇢** on a chart card, or  
+- **Double-click** the card (also expands stats)
+
+A panel lists ranked chart ids with short reasons (e.g. same context, high correlation). Click a row to scroll to that card. **Clear** or **Esc** dismisses.
+
+## Scoring (summary)
+
+| Signal | Effect |
+|--------|--------|
+| Same / related context, family, plugin, units | Taxonomy boost |
+| Pearson \|r\| on already-loaded wall series | Series similarity (needs enough points on both cards) |
+| Alert weights (`getWeights` without a highlight window) | Boost currently noisy charts |
+
+Pearson only runs on charts **already fetched** for the wall (visible / nearby). Taxonomy still ranks the full catalog.
+
+## Correlate vs Related
+
+| | Correlate | Related |
+|--|-----------|---------|
+| Question | What **changed** vs baseline? | What is **similar / nearby**? |
+| Where | Toolbar **Correlate** | Card **⇢** |
+| Engine | Server weights (`volume` / `ks2` / …) | Desktop `related-metrics.js` |
@@ -0,0 +1,33 @@
+# Settings
+
+Settings persist under `~/.config/peardata` (or `PEARDATA_HOME`).
+
+## Appearance
+
+| Setting | Notes |
+|---------|-------|
+| **Theme** | Light / dark / system |
+| **Motion** | Reduce or allow UI motion |
+
+## Charts / Overview
+
+| Setting | Notes |
+|---------|-------|
+| **Overview spark depth** | Sample count when Charts has no overriding window |
+| **Default spotlight chart** | Chart id for Overview explore (e.g. `system.io`) |
+| Chart type defaults / wall prefs | Also stored with pins and card heights |
+
+## Connections
+
+| Setting | Notes |
+|---------|-------|
+| **Restore saved agents** | Auto-dial peers on launch |
+| **Max reconnect attempts** | Cap for background dial retries |
+
+## Notifications
+
+Desktop notification toggles for alerts (when the runtime supports them).
+
+## Reset
+
+Clearing local prefs does not delete agent data on the server. Agent retention is controlled by agent env / warm store (see [docs/CONFIGURATION.md](../docs/CONFIGURATION.md)).
@@ -0,0 +1,67 @@
+# Weights API (Metric Correlations)
+
+Same scoring engine as desktop **Find Correlations**. Available over REST and P2P RPC.
+
+## REST
+
+```http
+GET /api/v1/weights
+GET /api/v2/weights
+GET /api/v3/weights
+```
+
+### Query parameters
+
+| Param | Default | Description |
+|-------|---------|-------------|
+| `method` | `volume` when windows set; else `alerts` | `volume` \| `ks2` \| `anomaly-rate` \| `value` \| `alerts` |
+| `after` | — | Highlight start (unix seconds, or relative) |
+| `before` | — | Highlight end |
+| `baseline_after` | auto | Baseline start (MC methods) |
+| `baseline_before` | auto | Baseline end |
+| `points` | `500` | Cap samples per series (max ~10000) |
+| `time_group` | `cv` for `value` | Aggregation for `value` method |
+| `contexts` | — | Filter contexts |
+| `chart` / `charts` | — | Filter chart ids |
+| `dimensions` | — | Dimension filter |
+| `limit` | — | Max results |
+| `timeout` | — | Scoring budget (ms) |
+
+Without highlight windows, the agent returns **legacy alert weights** (`method=alerts`) so Related-panel boosts keep working.
+
+### Example
+
+```bash
+# Highlight last 60s; baseline auto (≈4× preceding)
+curl -sG 'http://127.0.0.1:19999/api/v3/weights' \
+  --data-urlencode 'method=volume' \
+  --data-urlencode 'after=-60' \
+  --data-urlencode 'before=0' \
+  --data-urlencode 'limit=30'
+```
+
+### Response shape (conceptual)
+
+```json
+{
+  "method": "volume",
+  "view": {
+    "highlight": { "after": 0, "before": 0 },
+    "baseline": { "after": 0, "before": 0 }
+  },
+  "results": [
+    { "id": "system.cpu", "weight": 0.92, "context": "system.cpu", "family": "cpu" }
+  ],
+  "stats": { "scored": 120, "elapsed_ms": 40 }
+}
+```
+
+Higher `weight` ⇒ more changed for MC methods.
+
+## RPC
+
+Method: **`getWeights`**
+
+Same argument fields as the query table. Used by the desktop Charts tab and Related alert boost.
+
+See also [docs/REST-API.md](../docs/REST-API.md) and [docs/PROTOCOL.md](../docs/PROTOCOL.md).