118 lines
3.0 KiB
Markdown
118 lines
3.0 KiB
Markdown
# API: hyper-p2p-inverted-index
|
|
|
|
**Export:** `{ HyperP2PInvertedIndex, PROTOCOL }`
|
|
**Protocol:** `inverted-index/v1`
|
|
|
|
## Overview
|
|
|
|
`HyperP2PInvertedIndex` maps normalized terms to document IDs and mirrors document → term sets locally. With a Hyperswarm `topic`, add/remove operations gossip as `term-index-sync` so peers converge on the same inverted postings without shipping full index snapshots.
|
|
|
|
## Constructor
|
|
|
|
```js
|
|
const { HyperP2PInvertedIndex } = require('hyper-p2p-inverted-index')
|
|
const idx = new HyperP2PInvertedIndex(opts)
|
|
```
|
|
|
|
| Option | Type | Default | Description |
|
|
|--------|------|---------|-------------|
|
|
| `topic` | `string` \| `Buffer` \| `null` | `null` | Swarm topic; gossip active after `ready()` |
|
|
| `keyPair` | `KeyPair` | new key pair | Swarm identity; `peerHex` on gossip |
|
|
|
|
## Term normalization
|
|
|
|
`normalizeTerm(term)` → lowercase trimmed string. Empty tokens dropped on `index`.
|
|
|
|
## Methods
|
|
|
|
### `index(docId, terms)`
|
|
|
|
- **Parameters:** `docId` (non-empty), `terms` (array of strings)
|
|
- **Returns:** `{ docId, terms: normalized[] }`
|
|
- **Gossip:** For each new term on this doc, sends `{ type: 'term-index-sync', op: 'add', term, docId, peer, at }`
|
|
- **Emits:** `index`
|
|
- **Increments:** `stats.indexed`
|
|
|
|
### `search(term)`
|
|
|
|
- **Returns:** sorted doc IDs for normalized term, or `[]`
|
|
- **Increments:** `stats.queries`
|
|
|
|
### `removeDoc(docId)`
|
|
|
|
Removes all term postings for document; gossips `op: 'remove'` per term. Emits `remove`. Returns `false` if unknown doc.
|
|
|
|
### `listTerms()`
|
|
|
|
Sorted list of indexed terms.
|
|
|
|
### `docCount()`
|
|
|
|
Number of documents in `_docs`.
|
|
|
|
### `getStats()`
|
|
|
|
```js
|
|
{
|
|
indexed, removed, queries, gossipIn, gossipOut,
|
|
terms, docs, protocol
|
|
}
|
|
```
|
|
|
|
### `ready()` / `close()`
|
|
|
|
Joins swarm when `topic` set. `close()` clears maps and destroys swarm.
|
|
|
|
## Gossip handler `_onGossip`
|
|
|
|
Accepts only `type === 'term-index-sync'`.
|
|
|
|
| `op` | Effect | Event |
|
|
|------|--------|-------|
|
|
| `add` | Add `docId` to term set; update `_docs` | `remote-index` |
|
|
| `remove` | Remove posting; trim `_docs` term set | `remote-remove` |
|
|
|
|
## Events
|
|
|
|
| Event | Payload |
|
|
|-------|---------|
|
|
| `index` | `{ docId, terms }` |
|
|
| `remove` | `{ docId }` |
|
|
| `remote-index` | `{ term, docId, peer }` |
|
|
| `remote-remove` | `{ term, docId, peer }` |
|
|
| `closed` | — |
|
|
|
|
## Wire messages (summary)
|
|
|
|
| type | fields |
|
|
|------|--------|
|
|
| `term-index-sync` | `op` (`add` \| `remove`), `term`, `docId`, `peer`, `at` |
|
|
|
|
## Errors
|
|
|
|
`assertNonEmpty` on `index` and `removeDoc`. Gossip ignored when `_peerMsgs` not ready.
|
|
|
|
## Usage pattern
|
|
|
|
```js
|
|
const idx = new HyperP2PInvertedIndex({ topic: process.argv[2] })
|
|
await idx.ready()
|
|
idx.index('doc-1', ['hyper', 'p2p', 'index'])
|
|
console.log(idx.search('p2p'))
|
|
await idx.close()
|
|
```
|
|
|
|
## Comparison to fulltext-lite
|
|
|
|
| Feature | inverted-index | fulltext-lite |
|
|
|---------|----------------|---------------|
|
|
| P2P sync | yes (`term-index-sync`) | no |
|
|
| Document body | terms only | full text + snippets |
|
|
| Query | single term | multi-term AND |
|
|
|
|
## Testing
|
|
|
|
```bash
|
|
npm install && npm test
|
|
```
|