Architecture: hyper-p2p-inverted-index

Category: indexes-search
Protocol: inverted-index/v1

Purpose

Distributed term → document inverted index with incremental gossip sync.

Dual index

_terms: Map<term, Set<docId>>
_docs:  Map<docId, Set<term>>

Keeps remove and re-index operations consistent.

Wire messages

type	op	fields	behavior
`term-index-sync`	`add`	`term`, `docId`, `peer`, `at`	Insert posting; emit `remote-index`
`term-index-sync`	`remove`	`term`, `docId`, `peer`, `at`	Delete posting; prune empty term; emit `remote-remove`

Gossip flow

index(doc, terms)
  → for new (term, doc) pairs → gossipSend add
removeDoc(doc)
  → for each term → gossipSend remove
_onGossip
  → merge into _terms / _docs

Swarm integration

initModuleSwarm from ../../_shared/p2p-bare.js registers Protomux JSON handler → _onGossip.

Lifecycle

Construct with topic
ready() — join topic
index / search / removeDoc
close() — teardown

Trade-offs

Gossip per term change (not batch snapshots)
No ranking or TF-IDF
Re-index replaces doc terms locally but only gossips new terms on index (not automatic diff remove for changed terms — re-call removeDoc or manage terms explicitly)

Composition

Pair with hyper-p2p-fulltext-lite locally for snippets; use Hyperbee at app layer for persistence.

1.4 KiB Raw Blame History