Updates
This commit is contained in:
Raven Scott
@@ -1,94 +1,93 @@
|
||||
# API: hyper-p2p-temporal-index
|
||||
|
||||
**Protocol:** `hyper-p2p-temporal-index/v1`
|
||||
**Protocol:** `hyper-p2p-temporal-index/v1` (`TEMPORAL_PROTOCOL`)
|
||||
|
||||
**Export:** `HyperP2PTemporalIndex`
|
||||
|
||||
## Overview
|
||||
|
||||
HyperP2PTemporalIndex Novel temporal indexing primitive for Bare/Pear P2P. Features: - Multi-level hierarchical time bucketing (year/month/day/hour/minute) for efficient range queries
|
||||
`HyperP2PTemporalIndex` indexes time-series events in hierarchical UTC buckets (year → month → day → hour → minute), optional Ed25519 signing, TTL pruning, range and nearest queries, optional Hyperbee persistence, and gossip replication of inserts when `topic` or `swarm` is configured.
|
||||
|
||||
## Constructor
|
||||
|
||||
```js
|
||||
const mod = new HyperP2PTemporalIndex(opts)
|
||||
const HyperP2PTemporalIndex = require('hyper-p2p-temporal-index')
|
||||
const idx = new HyperP2PTemporalIndex(options)
|
||||
```
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `topic` | `string` | `Buffer` | `null` | Hyperswarm topic; required for P2P `ready()` |
|
||||
| `keyPair` | KeyPair | random | Ed25519 key pair |
|
||||
| `localId` | `string` \| `Buffer` | random 8 bytes | Metadata + vector clock id |
|
||||
| `maxEvents` | `number` | `100000` | Soft cap; triggers prune pressure |
|
||||
| `defaultTtlMs` | `number` | 30 days | Event expiry |
|
||||
| `pruneIntervalMs` | `number` | 5 min | Background prune timer |
|
||||
| `enableSigning` | `boolean` | `true` | Sign events on insert |
|
||||
| `keyPair` | `KeyPair` | random | Signing + P2P |
|
||||
| `hyperbee` | `Hyperbee` | `null` | Optional persistence |
|
||||
| `swarm` | object | `null` | Pre-attached swarm |
|
||||
| `vectorClock` | `HyperP2PVectorClock` | `null` | Optional; `tick` on insert |
|
||||
| `topic` | `string` | `null` | Auto `_initP2P` on construct |
|
||||
|
||||
## Methods
|
||||
|
||||
### `insertEvent(data, options = {})`
|
||||
### `async insertEvent(data, options = {}) → event`
|
||||
|
||||
- **Returns:** `Promise`
|
||||
- **Throws:** — (none documented in method body)
|
||||
- **options:** `timestamp`, `ttlMs`, `metadata`, `id`
|
||||
- **Returns:** signed event with `id`, `timestamp`, `data`, `metadata`, `vectorClock`, `expiresAt`
|
||||
- **Emits:** `insert`, `event`
|
||||
- Gossips `{ type: 'event', event }` when swarm attached
|
||||
|
||||
### `queryRange(startTime, endTime, options = {})`
|
||||
### `async queryRange(startTime, endTime, options = {}) → event[]`
|
||||
|
||||
- **Returns:** `Promise`
|
||||
- **Throws:** — (none documented in method body)
|
||||
Scans overlapping day buckets in `timeIndex`; verifies signatures when enabled; sorts by timestamp then vector clock.
|
||||
|
||||
### `queryNearest(targetTime, options = {})`
|
||||
- **options:** `limit` (default 1000), `verify` (default true)
|
||||
|
||||
- **Returns:** `Promise`
|
||||
- **Throws:** — (none documented in method body)
|
||||
### `async queryNearest(targetTime, options = {}) → event | null`
|
||||
|
||||
### `pruneExpired(force = false)`
|
||||
- **options.direction:** `'before' | 'after' | 'nearest'` (default `'before'`)
|
||||
|
||||
- **Returns:** `Promise`
|
||||
- **Throws:** — (none documented in method body)
|
||||
### `async pruneExpired(force = false) → number`
|
||||
|
||||
### `getStats(—)`
|
||||
|
||||
- **Returns:** `object`
|
||||
- **Throws:** — (none documented in method body)
|
||||
|
||||
### `close(—)`
|
||||
|
||||
- **Returns:** `Promise<void>`
|
||||
- **Throws:** — (none documented in method body)
|
||||
|
||||
### `getMetrics(—)`
|
||||
|
||||
- **Returns:** `value`
|
||||
- **Throws:** — (none documented in method body)
|
||||
|
||||
### `deriveTopic(timeBucket = null)`
|
||||
|
||||
- **Returns:** `value`
|
||||
- **Throws:** — (none documented in method body)
|
||||
Removes expired ids from `events`, `timeIndex`, `expiryQueue`. Emits `prune`.
|
||||
|
||||
### `setVectorClock(vc)`
|
||||
|
||||
- **Returns:** `value`
|
||||
- **Throws:** — (none documented in method body)
|
||||
Attach external vector clock module.
|
||||
|
||||
## Events
|
||||
### `deriveTopic(timeBucket?) → Buffer`
|
||||
|
||||
| Event | Payload |
|
||||
|-------|---------|
|
||||
| `close` | no payload |
|
||||
| `error` | err |
|
||||
| `event` | type |
|
||||
| `insert` | event |
|
||||
| `prune` | count |
|
||||
| `query` | type, count |
|
||||
SHA topic prefix + bucket for sharded swarm join.
|
||||
|
||||
## getStats()
|
||||
### `getMetrics()` / `getStats()` / `async close()`
|
||||
|
||||
Returns `{ ...this._stats }` — typically `ops`, `errors`, and module-specific counters (`created`, `relays`, `open`, `peers`, etc.).
|
||||
Library-only modules may include `mode: 'local'`.
|
||||
`getMetrics`: `{ inserts, queries, prunes, signed, eventCount, bucketCount }`.
|
||||
|
||||
## Errors
|
||||
## Event record
|
||||
|
||||
Stable message substrings: see [`../_shared/ERROR_CODES.md`](../../_shared/ERROR_CODES.md).
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `id` | Unique hex |
|
||||
| `timestamp` | Event time ms |
|
||||
| `data` | Payload |
|
||||
| `metadata` | Includes `localId` |
|
||||
| `vectorClock` | Counter or clock snapshot |
|
||||
| `signature` / `issuer` | When signing enabled |
|
||||
| `insertedAt` / `expiresAt` | Lifecycle |
|
||||
|
||||
## P2P
|
||||
## P2P wire
|
||||
|
||||
When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `hyper-p2p-temporal-index/v1`.
|
||||
| type | fields |
|
||||
|------|--------|
|
||||
| `event` | `event` (full record) |
|
||||
|
||||
## Events (EventEmitter)
|
||||
|
||||
`insert`, `event`, `query`, `prune`, `event-replicated`, `error`, `close`
|
||||
|
||||
## Buckets
|
||||
|
||||
Levels: `year`, `month`, `day`, `hour`, `minute` — all keys stored per insert for range scans.
|
||||
|
||||
## Testing
|
||||
|
||||
@@ -96,4 +95,4 @@ When `topic` is set, `ready()` joins Hyperswarm and opens Protomux `hyper-p2p-te
|
||||
npm install && npm test
|
||||
```
|
||||
|
||||
Integration: [`../../real_tests/integration/temporal-index-two-node.js`](../../../real_tests/integration/temporal-index-two-node.js)
|
||||
Example: [`../examples/basic.js`](../examples/basic.js).
|
||||
|
||||
@@ -1,45 +1,36 @@
|
||||
# Architecture: hyper-p2p-temporal-index
|
||||
|
||||
**Category:** Time & ordering
|
||||
**Category:** `time-ordering` · **Protocol:** `hyper-p2p-temporal-index/v1`
|
||||
|
||||
## Role
|
||||
|
||||
P2P-native time-series index: hierarchical buckets + optional causality + signed audit trail.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
App[Application] --> Mod[HyperP2PTemporalIndex]
|
||||
Mod --> Mux[Protomux hyper-p2p-temporal-index/v1]
|
||||
Mux --> Swarm[Hyperswarm]
|
||||
```
|
||||
|
||||
## Sequence (P2P)
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant App
|
||||
participant Mod as Module
|
||||
participant SW as Hyperswarm
|
||||
participant Peer
|
||||
App->>Mod: ready(topic)
|
||||
Mod->>SW: join(topic)
|
||||
SW->>Peer: connection
|
||||
Mod->>Peer: gossip / Protomux
|
||||
Peer-->>Mod: onmessage
|
||||
Mod-->>App: emit(event)
|
||||
App[Telemetry app] --> TI[TemporalIndex]
|
||||
TI --> Buckets[timeIndex Map]
|
||||
TI --> HB[Hyperbee optional]
|
||||
TI --> P2P[gossip event]
|
||||
```
|
||||
|
||||
## Wire messages
|
||||
|
||||
| type | fields | direction | behavior |
|
||||
|------|--------|-----------|----------|
|
||||
| `event` | event | gossip | Handled in onmessage / gossipSend |
|
||||
| `insert` | type | gossip | Handled in onmessage / gossipSend |
|
||||
| `range` | count, type | gossip | Handled in onmessage / gossipSend |
|
||||
| type | direction | fields | behavior |
|
||||
|------|-----------|--------|----------|
|
||||
| `event` | gossip | `event` | Upsert `events` map; `event-replicated` |
|
||||
|
||||
## State model
|
||||
|
||||
- In-memory `Map` / `Set` structures for hot path
|
||||
- Optional Hyperbee/Hypercore persistence when `storageDir` or `memoryOnly` is configured
|
||||
- `close()` tears down swarm, timers, and clears ephemeral state
|
||||
| Structure | Purpose |
|
||||
|-----------|---------|
|
||||
| `events` | id → event |
|
||||
| `timeIndex` | bucket → Set of ids |
|
||||
| `expiryQueue` | id → expiresAt |
|
||||
|
||||
Hyperbee keys: `temporal/{bucket}/{id}`.
|
||||
|
||||
## Composition
|
||||
|
||||
Composes with: `hyper-p2p-vector-clock`, `hyper-p2p-paradox-clock`.
|
||||
|
||||
- **`hyper-p2p-vector-clock`** — causal ordering in range results
|
||||
- **`hyper-p2p-distributed-event-bus`** — topic events into temporal store
|
||||
|
||||
Reference in New Issue
Block a user