Files

T

Raven ScottandCursor e43e2e83f1 Deepen indexes-search: APIs, tests, and category docs

Extend all eight index/search packages with practical helpers for app
development and fix semantic-vector getStats() to expose real metrics.

Per-module code:
- bloom-gossip: listKeys(), clear()
- inverted-index: searchAll(and|or), getDocTerms()
- fulltext-lite: search(and|or), suggest(prefix)
- trie-prefix: countPrefix(), autocomplete(limit)
- graph-index: removeEdge(), outDegree(), edgeCount()
- similarity-lsh: nearVector(), bucketCount()
- semantic-vector-index: getStats() delegates to getMetrics() + PROTOCOL export
- spatial-index: listLocalPoints(), pointCount(), protocol in getStats()

Tests added for new APIs across six packages; all indexes-search npm test
suites pass (8/8).

Docs:
- Rewrite indexes-search category README (module table, composition)
- Expand bloom-gossip and semantic-vector architecture notes
- modules/README.md: link indexes-search doc hub

Parent workspace docs/indexes-search/README.md and MODULE_DOC_PASS.md
updated separately (outside this git root).

Co-authored-by: Cursor <[email protected]>

2026-05-21 00:20:56 -04:00

2.1 KiB

Raw Blame History

Architecture: hyper-p2p-semantic-vector-index

Category: indexes-search
Protocol: hyper-p2p-semantic-vector-index/v1 (SEMANTIC_PROTOCOL)

Purpose

Semantic embedding store with cosine top-k search, optional Ed25519 signing, 8-bit quantization, Hyperbee persistence, TTL pruning, and P2P vector-insert gossip.

Components

HyperP2PSemanticVectorIndex
  ├── vectors: Map<id, entry>
  ├── tagIndex: Map<tag, Set<id>>
  ├── expiryQueue: Map<id, expiryTs>
  ├── optional hyperbee / swarm / vectorClock
  └── prune timer (bare-timers)

Wire messages

type	fields	direction	behavior
`vector-insert`	`id`, `entry` (metadata subset on gossip)	outbound on `insert` when `swarm` set; inbound via `_initP2P` / `receiveGossip`	Remote handler increments `gossipReceived`, emits `gossip-received`; full vector merge via `receiveGossip`

Gossip send payload (lightweight):

{ type: 'vector-insert', id, entry: { id, metadata, timestamp } }

receiveGossip accepts full entry.vector when present and verifies signature if enabled.

Insert pipeline

L2-normalize vector to options.dimension
Build entry + optional clockSnapshot
_signVector when enableSigning
Store in vectors, index tags, schedule expiry
_persistToHyperbee if configured
vectorClock.tick if provided
gossipSend when swarm wired

Search pipeline

Filter by tags, owner, timeRange → optional signature verify → cosine vs query → sort → top k.

P2P topic derivation

createP2PTopic(namespace) hashes SEMANTIC_PROTOCOL:namespace:localId.

Lifecycle

constructor → (optional _initP2P) → open() loads hyperbee
insert / search / pruneExpired → close()

Composition

hyper-p2p-similarity-lsh for candidate pre-filter
hyper-p2p-vector-clock (optional) for causal snapshots
hyper-spatial-index for geo metadata alongside embeddings

Limits

Linear scan search (no HNSW)
getStats() returns getMetrics() plus protocol (same counters as getMetrics())

2.1 KiB Raw Blame History