Files

T

Raven ScottandCursor e43e2e83f1 Deepen indexes-search: APIs, tests, and category docs

Extend all eight index/search packages with practical helpers for app
development and fix semantic-vector getStats() to expose real metrics.

Per-module code:
- bloom-gossip: listKeys(), clear()
- inverted-index: searchAll(and|or), getDocTerms()
- fulltext-lite: search(and|or), suggest(prefix)
- trie-prefix: countPrefix(), autocomplete(limit)
- graph-index: removeEdge(), outDegree(), edgeCount()
- similarity-lsh: nearVector(), bucketCount()
- semantic-vector-index: getStats() delegates to getMetrics() + PROTOCOL export
- spatial-index: listLocalPoints(), pointCount(), protocol in getStats()

Tests added for new APIs across six packages; all indexes-search npm test
suites pass (8/8).

Docs:
- Rewrite indexes-search category README (module table, composition)
- Expand bloom-gossip and semantic-vector architecture notes
- modules/README.md: link indexes-search doc hub

Parent workspace docs/indexes-search/README.md and MODULE_DOC_PASS.md
updated separately (outside this git root).

Co-authored-by: Cursor <[email protected]>

2026-05-21 00:20:56 -04:00

1.8 KiB

Raw Blame History

Architecture: hyper-p2p-bloom-gossip

Category: indexes-search
Protocol: bloom-gossip/v1

Purpose

Replicate “seen key” hints across a Hyperswarm topic using a compact bloom filter plus incremental bloom-add gossip, avoiding full filter snapshots on every insert.

Component diagram

Application
    │
    ▼
HyperP2PBloomGossip
    ├── Uint8Array bit field (_bits)
    ├── Set _keys (exact membership for stats)
    └── initModuleSwarm → Protomux JSON channel

Wire messages

type	direction	fields	behavior
`bloom-add`	outbound / inbound	`key`, `peer`, `at`	Set bloom bits for `key` if not already in `_keys`; emit `remote-add` on inbound

Ignored: messages with missing type, wrong type, or missing key.

State model

Field	Role
`_bits`	Packed bloom array, length `ceil(_size / 8)`
`_keys`	Exact keys seen (local + remote)
`_stats`	Counters: `added`, `queries`, `gossipIn`, `gossipOut`
`peerHex`	Local public key hex on gossip payloads

Introspection

listKeys() — exact key set (for debugging; Bloom mightContain remains probabilistic for unknown keys)
clear() — zero bits and keys
exportBits() / importBits() — hand off filter state to new peers

Lifecycle

Construct with optional topic and bits
await ready() — joins swarm when topic set
add / mightContain / exportBits
await close() — zero filter, tear down swarm

Composition

Uses ../../_shared/p2p-bare.js (initModuleSwarm, gossipSend) and ../../_shared/lib/errors.js (assertNonEmpty).

Trade-offs

Pros: Low bandwidth per new key; fast negative lookups
Cons: False positives; importBits does not sync _keys; no delete gossip

1.8 KiB Raw Blame History