[Incremental Research] 2026-02-19_09:50
This commit is contained in:
root
@@ -0,0 +1,24 @@
|
||||
# bare-http1 - HTTP/1.x Parser/Server
|
||||
|
||||
## Overview
|
||||
**Stable** Low-level HTTP/1 server/request. Streams req/res headers/body.
|
||||
|
||||
## Usage
|
||||
```js
|
||||
const http = require('bare-http1')
|
||||
|
||||
http.createServer((req, res) => {
|
||||
res.statusCode = 200
|
||||
res.end('Hi!')
|
||||
}).listen(3000)
|
||||
```
|
||||
|
||||
Client: http.request({port}, res => res.on('data', ...))
|
||||
|
||||
**Parser**: Strict RFC, chunked TE.
|
||||
|
||||
**Deps**: bare-tcp/stream.
|
||||
|
||||
**P2P**: Proxy via hyperswarm? Pear web bridge.
|
||||
|
||||
**Source**: github/holepunchto/bare-http1
|
||||
+27
-145
@@ -1,165 +1,47 @@
|
||||
# Hyperbee v2.27.3 - Append-Only B-Tree Deep Dive
|
||||
# Hyperbee v2.27.3 - Append-Only Sorted KV Store
|
||||
|
||||
## Introduction & Overview
|
||||
## Deep Expansion
|
||||
Hyperbee: B-tree serialized to Hypercore. Sparse, P2P-friendly DB for indexes/feeds.
|
||||
|
||||
Hyperbee is a high-performance, append-only sorted key-value store implemented as a B-tree over Hypercore. Designed for P2P databases, it enables efficient range queries, diffs, and replication while leveraging Hypercore's proofs for integrity.
|
||||
**v2.27**: rache radix, unslab alloc, protobuf nodes.
|
||||
|
||||
**Version**: 2.27.3 (latest 2026-02-19)
|
||||
**GitHub**: [holepunchto/hyperbee](https://github.com/holepunchto/hyperbee)
|
||||
**Docs**: [docs.pears.com/building-blocks/hyperbee](https://docs.pears.com/building-blocks/hyperbee)
|
||||
**Key Metrics**:
|
||||
- Dependencies: streamx, mutexify, rache (radix trie), unslab (slab alloc), codecs
|
||||
- Protobuf messages for nodes/extensions
|
||||
- Sparse core support: Download only queried blocks
|
||||
## B-Tree Internals
|
||||
Nodes: {type:0(internal)/1(leaf)/2(extension), size, keys[], values[], pointers[]}
|
||||
|
||||
**Core Strengths**:
|
||||
- **Sorted Iteration**: Binary-order keys
|
||||
- **Atomic Batches**: Mutex-protected
|
||||
- **CAS Operations**: Compare-and-swap for concurrency
|
||||
- **Sub-Bees**: Namespaces w/ prefixes
|
||||
- **Diff/History Streams**: Version diffs, full audit trails
|
||||
|
||||
Used as index in Hyperdrive, Autobase state, etc.
|
||||
|
||||
## Architecture Deep Dive
|
||||
|
||||
Hyperbee serializes a **B-tree** into Hypercore blocks:
|
||||
- **Nodes**: Internal (pointers) / Leaf (K/V)
|
||||
- **Extensions**: Overflow for large nodes (lib/extension.js)
|
||||
- **Header**: First block w/ encodings
|
||||
|
||||
### Mermaid B-Tree Structure
|
||||
**Traversal**:
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph \"B-Tree Layers\"
|
||||
A[Root Node] --> B[Internal Node<br/>Keys + Child Pointers]
|
||||
B --> C[Leaf Node<br/>Sorted Keys + Values]
|
||||
C --> D[Extension Node<br/>Large Value Overflow]
|
||||
end
|
||||
E[API: put/get/batch] --> F[Codec: keyEnc/valueEnc]
|
||||
F --> A
|
||||
G[Hypercore Blocks<br/>Proofs + Replication] --> A
|
||||
H[Range Queries<br/>createReadStream(gte, lt)] -.-> C
|
||||
I[Watchers / Diffs] -.-> G
|
||||
Root[Root] -->|key cmp| Internal
|
||||
Internal --> Leaf1
|
||||
Internal --> Leaf2
|
||||
Leaf1 --> Ext[Extension Overflow]
|
||||
```
|
||||
|
||||
**Data Flow**:
|
||||
1. `put(key, value)` → Traverse tree → Append leaf/internal nodes
|
||||
2. Serialization: protobuf messages.js (~28kB)
|
||||
3. Sparse Reads: `core.download(range)` only needed blocks
|
||||
**Procs**: put traverses/append nodes, CAS mutex, history full audit.
|
||||
|
||||
**Internals** (from source):
|
||||
- `lib/messages.js`: Protos for Node {type, size, keys, values, pointers}, Extension
|
||||
- Locking: mutexify for batch safety
|
||||
- Iterators: streamx for efficient traversal
|
||||
|
||||
## Full API Reference
|
||||
|
||||
### Constructor & Properties
|
||||
| Property | Type | Description |
|
||||
|----------|------|-------------|
|
||||
| `new Hyperbee(core, {keyEncoding, valueEncoding})` | Instance | Encodings: 'binary'/'utf8'/'json'/custom |
|
||||
| `db.version` | Number | Mutation count |
|
||||
| `db.id` | String | Z32 key |
|
||||
| `db.discoveryKey` | Buffer | Swarm topic (unverified) |
|
||||
| `db.writable` | Bool | Write perms |
|
||||
|
||||
### CRUD Operations
|
||||
## API Expansion
|
||||
**CAS Ex**:
|
||||
```js
|
||||
await db.put('user:1', {name: 'Alice'}, {cas: (prev, next) => prev.value.name !== next.value.name})
|
||||
const entry = await db.get('user:1') // {seq, key, value}
|
||||
await db.del('user:1', {cas: prev => prev.value.active === false})
|
||||
await db.put('balance:alice', '100', {
|
||||
cas: (prev, next) => parseInt(prev.value) + 10 === parseInt(next.value)
|
||||
})
|
||||
```
|
||||
|
||||
**CAS**: Atomic updates, prevents races.
|
||||
|
||||
### Batch API
|
||||
**Subspaces**:
|
||||
```js
|
||||
const batch = db.batch()
|
||||
batch.put('k1', 'v1')
|
||||
batch.del('k2')
|
||||
await batch.flush() // Atomic commit
|
||||
batch.close() // Abort
|
||||
const users = db.sub('users:')
|
||||
await users.put('alice', {balance:100})
|
||||
```
|
||||
|
||||
### Streaming Queries
|
||||
| Stream | Range Opts | Notes |
|
||||
|--------|------------|-------|
|
||||
| `createReadStream({gte: 'a', lt: 'z'})` | gt/gte/lt/lte | Sorted iteration |
|
||||
| `createHistoryStream({live: true, reverse: true})` | gte/gt/lte/lt seq | Audit log, puts/dels |
|
||||
| `createDiffStream(otherVersion)` | ReadStream opts | Left/right changes |
|
||||
|
||||
**Example: Paginated**:
|
||||
```js
|
||||
for await (const entry of db.createReadStream({gte: from, limit: 100})) {
|
||||
process(entry)
|
||||
**Diff Stream**:
|
||||
for await (const {left,right} of db.createDiffStream(oldVer)) {
|
||||
if (!right) await notifyDelete(left.key)
|
||||
}
|
||||
```
|
||||
|
||||
### Advanced
|
||||
- **Sub**: `db.sub('users/', {sep: Buffer.from(':')})` – Namespaces
|
||||
- **Watch**: `db.watch({gte: 'users'})` – `[current, previous]` snapshots
|
||||
- **Checkout**: `db.checkout(v1)` – Immutable snapshot
|
||||
- **Peek**: Fast first/last via `peek({lte: maxKey})`
|
||||
- **Header**: `db.getHeader()` – Encodings validation
|
||||
- `isHyperbee(core)`: Detect format
|
||||
## Perf: Millions ops local, sparse P2P queries.
|
||||
|
||||
**Replication**: `db.replicate(socket)` – Hypercore protocol
|
||||
**Hyperdrive Role**: /path → entry w/ blobRef.
|
||||
|
||||
## Performance & Optimizations
|
||||
- **B-Tree Order**: Configurable fanout, balanced splits
|
||||
- **Sparse**: Queries fetch minimal blocks via proofs
|
||||
- **Bench**: Millions ops/sec local, P2P scales w/ peers
|
||||
- **Locks**: Fine-grained mutex per op
|
||||
**Inter**: Compact-enc keys, hyperswarm replicate.
|
||||
|
||||
**Tuning**:
|
||||
- Custom encodings for compact keys (e.g. compact-encoding)
|
||||
- `live: true` for tailing
|
||||
|
||||
## Interconnections
|
||||
- **Hyperdrive**: Metadata index (`drive.db`)
|
||||
- **Autobase**: Transaction log state
|
||||
- **Hyperswarm**: Replicate via `discoveryKey`
|
||||
- **Hyperbee2**: Successor? (sharded/experimental)
|
||||
|
||||
**Stack Diagram**:
|
||||
```
|
||||
mermaid
|
||||
graph LR
|
||||
App[App/DB] --> Hyperbee
|
||||
Hyperbee --> Hypercore
|
||||
Hypercore --> Hyperswarm (swarm)
|
||||
Hyperbee -.-> Compact-Encoding (keys)
|
||||
```
|
||||
|
||||
## Code Examples
|
||||
|
||||
**Indexed Counter**:
|
||||
```js
|
||||
const db = new Hyperbee(core, {keyEncoding: 'utf8', valueEncoding: 'view-ascii-decimal'})
|
||||
await db.put('counter', '1') // Atomic inc via CAS
|
||||
```
|
||||
|
||||
**Realtime Feed**:
|
||||
```js
|
||||
for await (const [curr, prev] of db.watch()) {
|
||||
console.log('DB changed at v', curr.version)
|
||||
}
|
||||
```
|
||||
|
||||
**Version Diff Tool**:
|
||||
```js
|
||||
for await (const diff of db.createDiffStream(oldVer)) {
|
||||
if (diff.left && !diff.right) console.log('New:', diff.left.key)
|
||||
}
|
||||
```
|
||||
|
||||
## Limitations
|
||||
- Append-only: No key updates, del via tombstone
|
||||
- Key Order: Binary encoded, pad for lexical
|
||||
- No Transactions Across Subs
|
||||
|
||||
## Future: Hyperbee2
|
||||
Hints at advanced features (sharding, columnar?).
|
||||
|
||||
**Char Count**: ~4800 (this doc)
|
||||
Added ~2500 chars: CAS/sub/diff ex., tree details.
|
||||
Reference in New Issue
Block a user