Usage — total data dir, Corestore, memory rings, warm HyperDB sample
Hot memory — 15m → 24h (or custom points) Warm memory — downsampled ring + samples-per-bucket Disk history — auto-rotate by age (1 day → 1 year, forever, or custom days) Soft disk budget + max warm points Auto prune on a configurable hour interval Dry-run prune / Prune now / Save to agent (admin role)
This commit is contained in:
@@ -57,9 +57,15 @@ Permissions: directory `0700`. Do **not** commit `data/` or `.env`.
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `PEARDATA_SAMPLE_MS` | `1000` | Collector interval |
|
||||
| `PEARDATA_TIER0_POINTS` | `3600` | High-res ring size (~1h @ 1s) |
|
||||
| `PEARDATA_TIER1_POINTS` | `1440` | Downsampled ring size |
|
||||
| `PEARDATA_TIER0_POINTS` | `3600` | High-res ring size (~1h @ 1s); seeds Data Manager |
|
||||
| `PEARDATA_TIER1_POINTS` | `1440` | Downsampled ring size; seeds Data Manager |
|
||||
| `PEARDATA_TIER1_EVERY` | `60` | Samples per tier1 average (also HyperDB warm flush) |
|
||||
| `PEARDATA_WARM_RETENTION_MS` | `604800000` (7d) | Soft age limit for HyperDB warm points (seed) |
|
||||
| `PEARDATA_WARM_MAX_BYTES` | `0` | Soft corestore budget in bytes (`0` = unlimited) |
|
||||
| `PEARDATA_WARM_MAX_POINTS` | `0` | Cap on warm points (`0` = unlimited) |
|
||||
| `PEARDATA_AUTO_PRUNE` | on | `0` disables scheduled prune on boot |
|
||||
|
||||
Live retention is edited in the desktop **Settings → Data** tab (admin) and persisted to `$PEARDATA_DATA_DIR/retention.json`. Env values only seed the file on first boot.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+10
-1
@@ -108,9 +108,18 @@ Journal is **enabled by default** on Linux (`PEARDATA_JOURNAL=0` to disable). In
|
||||
| Method | Role | Known jobs |
|
||||
|--------|------|------------|
|
||||
| `listJobs` | viewer | — |
|
||||
| `runJob` | operator | `collectOnce`, `snapshot`, `gcBuffers` |
|
||||
| `runJob` | operator | `collectOnce`, `snapshot`, `gcBuffers` (prune) |
|
||||
| `cancelJob` | operator | By job id |
|
||||
|
||||
### Data Manager (retention / storage)
|
||||
|
||||
| Method | Role | Notes |
|
||||
|--------|------|-------|
|
||||
| `getStorageInfo` | viewer | Disk usage, memory rings, warm sample stats, effective retention |
|
||||
| `getRetentionConfig` | viewer | Current policy (`retention.json`) |
|
||||
| `setRetentionConfig` | admin | Update hot/warm rings, age rotate, disk budget, auto-prune |
|
||||
| `pruneHistory` | admin | `{ dryRun?, beforeMs? }` — delete warm points past retention |
|
||||
|
||||
### HyperDB / peer links
|
||||
|
||||
| Method | Role | Notes |
|
||||
|
||||
+1
-1
@@ -45,7 +45,7 @@ curl -s http://127.0.0.1:18888/api/v3/health | jq
|
||||
| GET | `/api/v3/db` | HyperDB public/discovery keys + collections |
|
||||
| GET | `/api/v3/versions` | Agent / protocol / API versions |
|
||||
| GET | `/api/v3/me` | Anonymous REST identity note |
|
||||
| GET | `/api/v3/settings` | Runtime knobs |
|
||||
| GET | `/api/v3/settings` | Runtime knobs + retention + storage usage summary |
|
||||
| GET | `/api/v3/config` | alias of settings |
|
||||
| GET | `/health`, `/api/v1/health`, `/api/v3/health` | Aggregate health |
|
||||
|
||||
|
||||
+2
-1
@@ -47,7 +47,8 @@ Template rebrand, protocol, collector, memory store, anomalies, REST, desktop MV
|
||||
| RPC peer-link + `getDbInfo` / REST `/api/v3/db` | Done |
|
||||
| Query fallback: memory miss / long window → HyperDB warm | Done |
|
||||
| Hyperswarm mesh (`PEARDATA_SWARM=1`) | Done (opt-in) |
|
||||
| Configurable retention knobs (tier sizes) | Done (`PEARDATA_TIER*`) |
|
||||
| Configurable retention knobs (tier sizes) | Done (`PEARDATA_TIER*` + Data Manager RPC/UI) |
|
||||
| Data Manager (usage, age rotate, auto-prune) | Done (`retention.json`, Settings → Data) |
|
||||
| Documented warm history across restart (operator M4) | Done (`store-hyperdb-fallback` + STORAGE doc) |
|
||||
| Export snapshot job → JSON / Prometheus push | Done (`exportSnapshot`, `prometheusPush`) |
|
||||
| Autobase multi-writer parents | Later (Phase 2c) |
|
||||
|
||||
@@ -129,6 +129,11 @@ Banner fields on agent start:
|
||||
| `PEARDATA_SWARM` | off | `1` enables Hyperswarm `store.replicate` |
|
||||
| `PEARDATA_TIER1_EVERY` | `60` | Samples per warm bucket (~60s @ 1Hz) |
|
||||
| `PEARDATA_TIER1_POINTS` | `1440` | In-memory tier1 ring (HyperDB keeps longer) |
|
||||
| `PEARDATA_WARM_RETENTION_MS` | `7d` | Seeds Data Manager age rotate |
|
||||
| `PEARDATA_WARM_MAX_BYTES` / `_POINTS` | `0` | Soft caps; `0` = unlimited |
|
||||
| `PEARDATA_AUTO_PRUNE` | on | Scheduled prune after boot |
|
||||
|
||||
**Data Manager:** desktop **Settings → Data** (admin) calls `setRetentionConfig` / `pruneHistory`. Policy file: `$PEARDATA_DATA_DIR/retention.json`. Auto-prune runs on an interval and deletes HyperDB `@peardata/metric-point` rows older than the configured window (and enforces optional point/disk caps). Note: Hypercore deletes are logical; reclaiming physical disk may require compaction over time.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user