This commit is contained in:
Raven Scott
2026-02-19 16:00:39 -05:00
parent ec5d40c36d
commit 1246c96d0c
32 changed files with 3397 additions and 0 deletions
+113
View File
@@ -0,0 +1,113 @@
# autobase-test-helpers - Autobase Test Utilities
## Overview
autobase-test-helpers provides helpers for testing Autobase applications, especially for synchronizing multiple bases during replication tests.
### Key Features
- **sync helper**: Wait until Autobase replicas converge
- **Test-focused**: Designed for test suites
- **Simple API**: One-call synchronization
### Use Cases
- **Replication tests**: Ensure bases sync before assertions
- **Multi-writer tests**: Coordinate multiple Autobase instances
- **CI reliability**: Reduce test flakiness
## Installation
```bash
npm install autobase-test-helpers
```
## Quick Start
```js
const { sync } = require('autobase-test-helpers')
// bases should already be replicating
await sync(bases)
```
## API Reference
### `sync(bases)`
Wait for Autobase instances to synchronize.
**Parameters:**
- `bases` (array): Autobase instances
**Returns:** Promise<void>
## Complete Examples
### Example 1: Two-Writer Sync
```js
const Autobase = require('autobase')
const { sync } = require('autobase-test-helpers')
const base1 = new Autobase(store1, key)
const base2 = new Autobase(store2, key)
// ... set up replication between base1 and base2
await base1.append('hello')
await base2.append('world')
// Wait for both bases to sync
await sync([base1, base2])
// Now safe to assert
```
### Example 2: Multi-Writer Sync
```js
const { sync } = require('autobase-test-helpers')
const bases = [baseA, baseB, baseC, baseD]
await Promise.all([
baseA.append('a'),
baseB.append('b'),
baseC.append('c'),
baseD.append('d')
])
await sync(bases)
// All bases converged
```
## Best Practices
### Ensure Replication
```js
// sync expects replication to be set up
swarm.on('connection', (c) => {
base.replicate(c)
})
```
### Use in Tests
```js
const t = require('brittle')
t('replication', async () => {
await sync([base1, base2])
// assertions
})
```
## License
MIT
---
**Module Type**: Testing | **Ecosystem Role**: Autobase Test Helper | **Dependencies**: Autobase
+125
View File
@@ -0,0 +1,125 @@
# bare-rpc - ABI-Compatible RPC for Bare
## Overview
bare-rpc provides ABI-compatible RPC for Bare, built on librpc. It supports request/response and streaming RPC over any duplex stream.
### Key Features
- **Request/response**: Simple command-based RPC
- **Streaming**: Request and response streams
- **Command routing**: Built-in command router
- **Stream-agnostic**: Works with any duplex stream
### Use Cases
- **RPC over P2P**: Use with Hyperswarm streams
- **Local IPC**: In-process or local stream RPC
- **Protocol building**: Custom command protocols
## Installation
```bash
npm install bare-rpc
```
## Quick Start
```js
import RPC from 'bare-rpc'
// Server side
const rpc = new RPC(stream, (req) => {
if (req.command === 42) {
req.reply('pong')
}
})
// Client side
const req = rpc.request(42)
req.send('ping')
const reply = await req.reply()
console.log(reply.toString()) // pong
```
## API Reference
### `new RPC(stream[, onrequest])`
Create RPC instance.
### `rpc.request(command)`
Create outgoing request.
### RPCOutgoingRequest
- `req.command` - Command ID
- `req.sent` - Sent flag
- `req.received` - Reply received flag
- `req.send([data[, encoding]])` - Send request
- `await req.reply([encoding])` - Await reply
- `req.createRequestStream()` - Writable stream for request
- `req.createResponseStream()` - Readable stream for response
### RPCIncomingRequest
- `req.command` - Command ID
- `req.data` - Request buffer
- `req.sent` - Reply sent flag
- `req.received` - Received as stream flag
- `req.reply([data[, encoding]])` - Reply
- `req.createRequestStream()` - Readable stream for request
- `req.createResponseStream()` - Writable stream for response
### RPC.CommandRouter
```js
const router = new RPC.CommandRouter()
router.respond(42, (req, data) => Buffer.from('pong'))
```
## Complete Examples
### Example 1: Command Router
```js
const RPC = require('bare-rpc')
const router = new RPC.CommandRouter()
router.respond(1, (req, data) => {
return Buffer.from(`echo:${data.toString()}`)
})
const rpc = new RPC(stream, router)
const req = rpc.request(1)
req.send('hello')
console.log((await req.reply()).toString())
```
### Example 2: Streaming Request
```js
const rpc = new RPC(stream, (req) => {
const rs = req.createRequestStream()
const ws = req.createResponseStream()
rs.pipe(ws)
})
const req = rpc.request(7)
const ws = req.createRequestStream()
const rs = req.createResponseStream()
ws.write('chunk1')
ws.end('chunk2')
for await (const chunk of rs) {
console.log(chunk.toString())
}
```
## License
Apache-2.0
---
**Module Type**: RPC | **Ecosystem Role**: Stream RPC | **Dependencies**: streamx
+68
View File
@@ -0,0 +1,68 @@
# blind-peer-cli - Blind Peer CLI
## Overview
blind-peer-cli provides CLI commands to run blind peers. It supports standard node and Bare environments, with options for storage, trusted peers, and monitoring.
### Key Features
- **CLI runner**: Start blind peers easily
- **Bare runtime support**: `blind-peer-bare`
- **Trusted peers**: Allow announce permissions
- **Autodiscovery**: Register in discovery service
- **Prometheus**: Optional metrics integration
## Installation
```bash
npm install -g blind-peer-cli
```
## Usage
```bash
blind-peer
```
### Bare Runtime
```bash
blind-peer-bare
```
## Command Line Options
- `--storage, -s [path]` Storage path (default: ./blind-peer)
- `--port, -p [int]` DHT port (default: random)
- `--trusted-peer, -t [key]` Trusted peer key (repeatable)
- `--debug, -d` Enable debug logs (repeatable)
- `--max-storage, -m [int]` Max storage in MB (default: 100000)
- `--autodiscovery-rpc-key` Autodiscovery RPC public key
- `--autodiscovery-seed` Seed for autodiscovery auth
- `--autodiscovery-service-name` Service name (default: blind-peer)
- `--scraper-public-key` Prometheus scraper public key
- `--scraper-secret` Prometheus scraper secret
- `--scraper-alias` Scraper alias
## Output
Emits NDJSON (pino) logs, e.g.:
```jsonl
{"level":30,"msg":"Starting blind peer"}
{"level":30,"msg":"Using storage 'blind-peer'"}
{"level":30,"msg":"Blind peer listening"}
```
## Best Practices
- Use `--storage` on persistent disks
- Configure trusted peers if enabling announce
- Pipe to `pino-pretty` for readability
## License
Apache-2.0
---
**Module Type**: Tooling | **Ecosystem Role**: Blind Peer CLI | **Dependencies**: blind-peer
+63
View File
@@ -0,0 +1,63 @@
# blind-peer-encodings - Blind Peer Encodings
## Overview
blind-peer-encodings provides the schemas and encodings used by blind-peer and blind-peering. It includes Hyperdb schema definitions and RPC encodings.
### Key Features
- **Hyperdb definition**: Storage schema
- **RPC encodings**: Structured messages
- **Error types**: Blind peer-specific errors
## Installation
```bash
npm install blind-peer-encodings
```
## API Reference
### `BlindPeerEncodings.definition`
Blind-peer Hyperdb definition.
### `BlindPeerEncodings.schema`
Blind-peer Hyperdb schema.
### `BlindPeerEncodings.PostToMailboxEncoding`
Encoding for `addMailbox` RPC.
### `BlindPeerEncodings.AddCoreEncoding`
Encoding for `post` RPC.
### `BlindPeerEncodings.Mailbox`
Encoding for mailbox records.
### `BlindPeerEncodings.BlindPeerError`
Blind-peer error class.
## Usage
```js
const BlindPeerEncodings = require('blind-peer-encodings')
const { schema, AddCoreEncoding } = BlindPeerEncodings
```
## Best Practices
- Use encodings consistently on both client and server
- Handle BlindPeerError for protocol errors
## License
MIT
---
**Module Type**: Protocol | **Ecosystem Role**: Blind Peer Encoding | **Dependencies**: hyperdb
+34
View File
@@ -0,0 +1,34 @@
# blind-peer-muxer - Blind Peer Multiplexer (WIP)
## Overview
blind-peer-muxer is a work-in-progress package intended to provide multiplexing for blind-peer services.
### Status
- **WIP**: No stable API yet
- **Placeholder**: Not production-ready
## Installation
```bash
npm install blind-peer-muxer
```
## Usage
```js
const muxer = require('blind-peer-muxer')
```
## Notes
- Expect breaking changes
- Use only for experimentation
## License
MIT
---
**Module Type**: Experimental | **Ecosystem Role**: Multiplexing | **Dependencies**: None
+44
View File
@@ -0,0 +1,44 @@
# blind-peering-cli - Blind Peering CLI
## Overview
blind-peering-cli provides a command-line interface for interacting with blind peers. It supports seeding Hypercores and Hyperdrives via blind-peer services.
## Installation
```bash
npm install -g blind-peering-cli
```
## Usage
```bash
blind-peering --help
```
### Seed a Hypercore
```bash
blind-peering seed --core --blind-peer-key <blind-peer-rpc-key> <hypercore-key>
```
### Seed via Autodiscovery
```bash
blind-peering seed --core \
--auto-disc-db <autobase-discovery-db-key> \
--service-name <service-name> \
<hypercore-key>
```
## Notes
- Only trusted peers can request seeding
- Use `blind-peering identity` to get your DHT public key
## License
Apache-2.0
---
**Module Type**: Tooling | **Ecosystem Role**: Blind Peer Client | **Dependencies**: blind-peering
+34
View File
@@ -0,0 +1,34 @@
# blind-relay-service - Blind Relay Service CLI
## Overview
blind-relay-service provides a CLI to run a blind-relay server. It exposes relay functionality for UDX streams over Protomux channels.
## Installation
```bash
npm install blind-relay-service
```
## Usage
```bash
blind-relay [-s, --storage <path>] [-p, --port <num>]
```
### Options
- `--storage, -s`: Storage path
- `--port, -p`: Port number
## Best Practices
- Use persistent storage for long-running relays
- Run behind firewall with appropriate port access
## License
Apache-2.0
---
**Module Type**: Tooling | **Ecosystem Role**: Relay Service | **Dependencies**: blind-relay
+45
View File
@@ -0,0 +1,45 @@
# bot-rpc - Bot RPC Server
## Overview
bot-rpc runs a bot as an RPC server and provides a simple client to send data to it.
## Installation
```bash
npm install bot-rpc
```
## Usage
### Server
```js
const rpc = new BotRpc()
rpc.addHandler((data, reply) => {
try {
console.info('Received data:', data)
} catch (err) {
console.error('Failed with:', err)
reply(`\nFailed with: ${err}`)
}
})
```
### Client
```js
send({ remote: '<rpc-public-key>', data: { hello: 'world' } })
```
## Notes
- Minimal API intended for bot control
- Use with hyperswarm/rpc or similar transport
## License
Apache-2.0
---
**Module Type**: RPC | **Ecosystem Role**: Bot Control | **Dependencies**: hyperswarm/rpc
+28
View File
@@ -0,0 +1,28 @@
# grpc-lnd - LND gRPC Wrapper
## Overview
grpc-lnd (package name `lnd-grpc`) is a placeholder module intended to expose LND gRPC functionality. The README is minimal and does not document APIs yet.
## Status
- **Minimal documentation**
- **Likely WIP**
## Installation
```bash
npm install grpc-lnd
```
## Notes
- Repository README is empty
- Check source for actual APIs
## License
Not specified in README
---
**Module Type**: Experimental | **Ecosystem Role**: LND gRPC | **Dependencies**: Unknown
+37
View File
@@ -0,0 +1,37 @@
# hp-rpc-cli - Hyperswarm RPC CLI
## Overview
hp-rpc-cli is a basic command-line client for hyperswarm/rpc. It sends RPC requests to a server peer key.
## Usage
```bash
hp-rpc-cli --help
hp-rpc-cli -s <server-peer-key> -m <method> -d <data>
```
## Options
- `-s` server peer key
- `-i` identity.json (keypair file)
- `-m` method name
- `-d` data as string
- `-f` data as file
- `-t` timeout in milliseconds
- `-bn` DHT bootstrap nodes (comma separated)
- `-dp` DHT port
- `-ds` DHT node keypair seed (32-byte hex)
## References
- Identity setup: https://github.com/prdn/hyper-cmd-docs/blob/main/identity.md
- Host resolution: https://github.com/prdn/hyper-cmd-docs/blob/main/resolve.md
## License
Apache-2.0
---
**Module Type**: Tooling | **Ecosystem Role**: RPC CLI | **Dependencies**: hyperswarm/rpc
+59
View File
@@ -0,0 +1,59 @@
# http-dht-proxy - HTTP to DHT Proxy
## Overview
http-dht-proxy relays HTTP requests to a DHT peer. It enables HTTP clients to communicate with services exposed over HyperDHT.
### Key Features
- **HTTP bridge**: Translate HTTP to DHT
- **CLI tool**: Run proxy from terminal
- **Example server/client**: Included sample apps
## Installation
```bash
npm install -g http-dht-proxy
```
## Usage
```bash
http-dht-proxy 8080
```
## Example
### Start Proxy
```bash
node bin.js
# HTTP-to-DHT proxy on 8080
```
### Start Sample Server
```bash
node example/server.js
# Local http server on 8081
# DHT public key <key>
```
### Send Requests
```bash
node example/client.js <dht-public-key>
node example/client-header.js <dht-public-key>
```
## Notes
- DHT public key can be passed in path or header
- Useful for exposing DHT services to HTTP clients
## License
Apache-2.0
---
**Module Type**: Networking | **Ecosystem Role**: HTTP Bridge | **Dependencies**: hyperdht
+218
View File
@@ -0,0 +1,218 @@
# hypercore-audit - Hypercore Storage Audit
## Overview
hypercore-audit validates and repairs a Hypercore's on-disk storage. It scans the tree, blocks, and bitfield layers to detect corruption and optionally clean up inconsistencies.
### Key Features
- **Storage audit**: Validate tree nodes, blocks, and bitfield
- **Dry run mode**: Report issues without modifying storage
- **Repair mode**: Remove invalid records and fix inconsistencies
- **Corruption detection**: Detect missing root nodes
- **Structured report**: Detailed audit statistics
### Use Cases
- **Recovery**: Diagnose corrupted cores
- **Maintenance**: Periodic storage verification
- **Debugging**: Identify storage layer issues
- **Migration checks**: Validate after upgrades
## Installation
```bash
npm install hypercore-audit
```
## Quick Start
```js
const Hypercore = require('hypercore')
const audit = require('hypercore-audit')
const core = new Hypercore('./data')
await core.ready()
// Dry run audit
const report = await audit(core.state.storage, { dryRun: true })
console.log(report)
```
## API Reference
### `audit(storage, options)`
Perform an audit on a Hypercore storage instance.
**Parameters:**
- `storage` - Hypercore storage instance (`core.state.storage`)
- `options` (object):
- `tree` (boolean): Audit tree nodes (default: true)
- `blocks` (boolean): Audit blocks (default: true)
- `bitfield` (boolean): Audit bitfield (default: true)
- `dryRun` (boolean): Do not modify storage (default: false)
**Returns:** Promise<Report>
### Report
```js
{
treeNodes: 247, // Valid tree nodes
blocks: 127, // Valid blocks
bits: 127, // Valid bitfield entries
droppedTreeNodes: 0, // Invalid tree nodes removed
droppedBlocks: 0, // Invalid blocks removed
droppedBits: 0, // Invalid bits removed
corrupt: false // True if root nodes missing
}
```
## Complete Examples
### Example 1: Safe Audit (Dry Run)
```js
const Hypercore = require('hypercore')
const audit = require('hypercore-audit')
async function auditCore(path) {
const core = new Hypercore(path)
await core.ready()
const report = await audit(core.state.storage, {
tree: true,
blocks: true,
bitfield: true,
dryRun: true
})
console.log('Audit report:', report)
if (report.corrupt) {
console.warn('Core is corrupt (missing root nodes)')
}
return report
}
auditCore('./data')
```
### Example 2: Repair Mode
```js
const Hypercore = require('hypercore')
const audit = require('hypercore-audit')
async function repairCore(path) {
const core = new Hypercore(path)
await core.ready()
// Warning: this will modify storage
const report = await audit(core.state.storage, {
dryRun: false
})
console.log('Repair report:', report)
if (report.droppedBlocks > 0 || report.droppedTreeNodes > 0) {
console.log('Repairs were applied')
}
return report
}
repairCore('./data')
```
### Example 3: Partial Audit
```js
const Hypercore = require('hypercore')
const audit = require('hypercore-audit')
async function auditBlocksOnly(core) {
const report = await audit(core.state.storage, {
tree: false,
blocks: true,
bitfield: false,
dryRun: true
})
console.log('Blocks audit:', report.blocks)
return report
}
```
### Example 4: Scheduled Maintenance
```js
const Hypercore = require('hypercore')
const audit = require('hypercore-audit')
class CoreMaintainer {
constructor(path) {
this.core = new Hypercore(path)
}
async init() {
await this.core.ready()
}
async scheduledAudit() {
const report = await audit(this.core.state.storage, {
dryRun: true
})
if (report.corrupt || report.droppedBlocks > 0) {
console.warn('Potential issues found:', report)
// Optionally trigger repair
}
return report
}
}
// Usage
const maintainer = new CoreMaintainer('./data')
await maintainer.init()
setInterval(() => {
maintainer.scheduledAudit().catch(console.error)
}, 24 * 60 * 60 * 1000) // Daily
```
## Best Practices
### Use Dry Run First
```js
// Always dry run before repair
const report = await audit(storage, { dryRun: true })
if (report.droppedBlocks > 0) {
// Decide if repair is needed
}
```
### Backup Before Repair
```js
// Make backup of storage before running with dryRun: false
```
### Monitor Corruption
```js
if (report.corrupt) {
console.error('Core is corrupt - missing root nodes')
}
```
## License
Apache-2.0
---
**Module Type**: Maintenance | **Ecosystem Role**: Storage Integrity | **Dependencies**: Hypercore
+198
View File
@@ -0,0 +1,198 @@
# hypercore-blob-server - Hypercore Blob HTTP Server
## Overview
hypercore-blob-server provides an HTTP server for streaming Hypercore blobs and Hyperdrive files. It generates secure links for blobs and supports range requests for partial downloads.
### Key Features
- **Blob streaming**: Serve Hypercore blobs over HTTP
- **Drive support**: Serve Hyperdrive files by filename
- **Range requests**: Partial downloads with `Range` headers
- **Token protection**: Optional server token
- **Suspend/resume**: Pause and resume server operations
### Use Cases
- **Media streaming**: Serve videos/audio from Hypercore
- **File hosting**: Serve Hyperdrive files
- **Content distribution**: HTTP links to P2P content
- **Resume downloads**: Range-based partial content
## Installation
```bash
npm install hypercore-blob-server
```
## Quick Start
```js
const BlobServer = require('hypercore-blob-server')
const Corestore = require('corestore')
const store = new Corestore('./storage')
const server = new BlobServer(store)
await server.listen()
// Generate link for a blob
const link = server.getLink(core.key, {
blob: blobId,
type: 'image/jpeg'
})
console.log('Blob URL:', link)
```
## API Reference
### `new BlobServer(store, options)`
Create a blob server instance.
**Parameters:**
- `store` - Corestore instance
- `options` (object):
- `port` (number): Port (default: 49833)
- `host` (string): Host (default: '127.0.0.1')
- `token` (string): Access token
- `protocol` (string): 'http' or 'https'
### `await server.listen()`
Start listening for requests.
### `server.getLink(key, options)`
Generate a URL for a blob or file.
**Parameters:**
- `key` - Hypercore or Hyperdrive key
- `options`:
- `host` (string): Override host
- `port` (number): Override port
- `protocol` (string): Override protocol
- `filename` (string): Drive filename
- `blob` (object): Blob ID
**Blob ID format:**
```js
{ blockOffset, blockLength, byteOffset, byteLength }
```
### `await server.suspend()`
Suspend server operations.
### `await server.resume()`
Resume server operations.
## Complete Examples
### Example 1: Serve Hyperdrive Files
```js
const BlobServer = require('hypercore-blob-server')
const Corestore = require('corestore')
const Hyperdrive = require('hyperdrive')
const store = new Corestore('./storage')
const drive = new Hyperdrive(store)
await drive.ready()
// Add a file
await drive.put('/hello.txt', Buffer.from('Hello from Hyperdrive'))
// Start server
const server = new BlobServer(store)
await server.listen()
// Generate file link
const link = server.getLink(drive.key, {
filename: '/hello.txt'
})
console.log('File URL:', link)
```
### Example 2: Range Requests
```js
const BlobServer = require('hypercore-blob-server')
// Client: Download part of a blob
const response = await fetch(blobUrl, {
headers: { 'Range': 'bytes=0-1023' }
})
const chunk = await response.arrayBuffer()
console.log('Downloaded', chunk.byteLength, 'bytes')
```
### Example 3: Token-Protected Server
```js
const BlobServer = require('hypercore-blob-server')
const Corestore = require('corestore')
const store = new Corestore('./storage')
const server = new BlobServer(store, {
token: 'my-secret-token'
})
await server.listen()
const link = server.getLink(core.key, {
blob: blobId
})
// Link includes token for access
console.log('Protected URL:', link)
```
### Example 4: Suspension Control
```js
const BlobServer = require('hypercore-blob-server')
const server = new BlobServer(store)
await server.listen()
// Suspend server (stop accepting requests)
await server.suspend()
console.log('Server suspended')
// Resume later
await server.resume()
console.log('Server resumed')
```
## Best Practices
### Use Range Requests for Large Files
```js
// Enable resume downloads
fetch(url, { headers: { Range: 'bytes=1000-' } })
```
### Secure Public Servers
```js
const server = new BlobServer(store, { token: process.env.BLOB_TOKEN })
```
### Bind to localhost for Local Use
```js
const server = new BlobServer(store, { host: '127.0.0.1' })
```
## License
Apache-2.0
---
**Module Type**: Networking | **Ecosystem Role**: HTTP Blob Serving | **Dependencies**: corestore
+170
View File
@@ -0,0 +1,170 @@
# hypercore-byte-stream - Byte Range Streams
## Overview
hypercore-byte-stream provides a Readable stream wrapper around a Hypercore, supporting byte-range reads within blobs. It is ideal for media streaming and partial data access.
### Key Features
- **Byte range reads**: Read specific byte ranges
- **Blob boundaries**: Respect blob boundaries via blob IDs
- **Prefetch control**: Tune prefetching behavior
- **Session ownership**: Closes core session when done
### Use Cases
- **Video streaming**: Serve byte-range video chunks
- **Audio streaming**: Partial reads for media players
- **Resume downloads**: Range-based file downloads
- **Large blob processing**: Avoid loading whole blob
## Installation
```bash
npm install hypercore-byte-stream
```
## Quick Start
```js
const ByteStream = require('hypercore-byte-stream')
// blob id defines outer bounds
const blobId = { blockOffset, blockLength, byteOffset, byteLength }
// Range options: { start, length }
const stream = new ByteStream(core, blobId, { start: 0, length: 1024 })
stream.on('data', (chunk) => {
console.log('Received', chunk.length, 'bytes')
})
```
### Convenience for Single-Blob Cores
```js
const stream = ByteStream.one(core, { start: 0, length: 1024 })
```
## API Reference
### `new ByteStream(core, id, options)`
Create a byte stream over a blob.
**Parameters:**
- `core` - Hypercore instance
- `id` - Blob ID `{ blockOffset, blockLength, byteOffset, byteLength }`
- `options`:
- `start` (number): Byte start offset
- `length` (number): Byte length to read
- `maxPrefetch` (number): Max prefetch size
### `ByteStream.one(core, options)`
Convenience for cores that contain a single blob.
## Complete Examples
### Example 1: Media Streaming
```js
const ByteStream = require('hypercore-byte-stream')
const fs = require('bare-fs')
async function streamToFile(core, blobId, outputPath) {
const stream = new ByteStream(core, blobId)
const file = fs.createWriteStream(outputPath)
return new Promise((resolve, reject) => {
stream.pipe(file)
file.on('finish', resolve)
file.on('error', reject)
})
}
```
### Example 2: Partial Download
```js
const ByteStream = require('hypercore-byte-stream')
async function downloadRange(core, blobId, start, length) {
const stream = new ByteStream(core, blobId, { start, length })
const chunks = []
for await (const chunk of stream) {
chunks.push(chunk)
}
return Buffer.concat(chunks)
}
```
### Example 3: HTTP Range Serving
```js
const http = require('bare-http1')
const ByteStream = require('hypercore-byte-stream')
const server = http.createServer((req, res) => {
const range = req.headers.range // e.g. "bytes=0-1023"
const [start, end] = parseRange(range)
const length = end - start + 1
const stream = new ByteStream(core, blobId, { start, length })
res.statusCode = 206
res.setHeader('Content-Range', `bytes ${start}-${end}/${blobId.byteLength}`)
res.setHeader('Content-Length', length)
stream.pipe(res)
})
```
### Example 4: Resume Download
```js
const ByteStream = require('hypercore-byte-stream')
async function resumeDownload(core, blobId, downloaded) {
const stream = new ByteStream(core, blobId, {
start: downloaded,
length: blobId.byteLength - downloaded
})
for await (const chunk of stream) {
// append to file
}
}
```
## Best Practices
### Respect Blob Boundaries
```js
// Ensure blobId covers the data you want
const stream = new ByteStream(core, blobId)
```
### Use maxPrefetch for Performance
```js
const stream = new ByteStream(core, blobId, { maxPrefetch: 1024 * 1024 })
```
### Handle Stream Completion
```js
stream.on('end', () => {
console.log('Stream complete')
})
```
## License
Apache-2.0
---
**Module Type**: Streaming | **Ecosystem Role**: Byte-Range Access | **Dependencies**: hypercore
+125
View File
@@ -0,0 +1,125 @@
# hypercore-detector - Detect Hypercore Type
## Overview
hypercore-detector determines whether a Hypercore is a basic core, Hyperbee, or Hyperdrive. It inspects the first block once available and returns a type hint.
### Key Features
- **Type detection**: core, bee, drive, or null
- **Wait option**: Await first block availability
- **Simple API**: One function call
### Use Cases
- **Dynamic handling**: Auto-detect core type
- **Tooling**: CLI tools that accept any core type
- **Routing**: Choose correct handlers per type
## Installation
```bash
npm install hypercore-detector
```
## Quick Start
```js
const detect = require('hypercore-detector')
let type = await detect(core)
if (!type) {
type = await detect(core, { wait: true })
}
if (type === 'bee') console.log('Hyperbee')
else if (type === 'drive') console.log('Hyperdrive')
else console.log('Hypercore')
```
## API Reference
### `detect(hypercore[, opts])`
Detect the type of a Hypercore.
**Parameters:**
- `hypercore` - Hypercore instance
- `opts`:
- `wait` (boolean): Wait for block 0 (default: false)
**Returns:** `'core' | 'bee' | 'drive' | null`
**Notes:**
- `null` if type cannot yet be determined
- `wait: true` waits for first block locally available
## Complete Examples
### Example 1: Auto Router
```js
const detect = require('hypercore-detector')
const Hyperbee = require('hyperbee')
const Hyperdrive = require('hyperdrive')
async function openAny(core) {
const type = await detect(core, { wait: true })
if (type === 'bee') return new Hyperbee(core)
if (type === 'drive') return new Hyperdrive(core)
return core
}
```
### Example 2: CLI Tool
```js
const detect = require('hypercore-detector')
async function inspect(core) {
const type = await detect(core, { wait: true })
console.log('Type:', type)
}
```
### Example 3: Timeout Detection
```js
const detect = require('hypercore-detector')
async function detectWithTimeout(core, timeout = 5000) {
const result = await Promise.race([
detect(core, { wait: true }),
new Promise(resolve => setTimeout(() => resolve(null), timeout))
])
return result
}
```
## Best Practices
### Ensure Replication
```js
// wait requires block 0 locally available
swarm.join(core.discoveryKey)
```
### Fallback Logic
```js
const type = await detect(core)
if (!type) {
// fallback or wait
}
```
## License
Apache-2.0
---
**Module Type**: Utility | **Ecosystem Role**: Type Detection | **Dependencies**: hypercore
+170
View File
@@ -0,0 +1,170 @@
# hypercore-e2e-tests - End-to-End Replication Tests
## Overview
hypercore-e2e-tests provides end-to-end replication tests for Hypercore. It includes CLI tools and Docker images to create, seed, and download a test core.
### Key Features
- **CLI tools**: Create/seed/download test flows
- **Docker images**: Easy deployment
- **Prometheus optional**: Connect to metrics scraper
- **Replication validation**: Full workflow testing
### Use Cases
- **Replication testing**: Validate Hypercore replication
- **Infrastructure checks**: Network and DHT validation
- **CI pipelines**: Automated E2E tests
## Installation
```bash
npm install -g hypercore-e2e-tests pino-pretty
```
## CLI Usage
### Create
```bash
hypercore-e2e-create | pino-pretty
```
### Seed
```bash
HYPERCORE_E2E_KEY=<public key> hypercore-e2e-seed | pino-pretty
```
### Download
```bash
HYPERCORE_E2E_KEY=<public key> hypercore-e2e-download | pino-pretty
```
## Docker Usage
### Create
```bash
--mount type=volume,source=hypercore-e2e-tests-create-volume,destination=/home/hypercore-e2e-tests/corestore \
--env HYPERCORE_E2E_PROMETHEUS_SECRET=... \
--env HYPERCORE_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY=... \
--env HYPERCORE_E2E_LENGTH=... \
ghcr.io/holepunchto/hypercore-e2e-tests-create
```
### Seed
```bash
--mount type=volume,source=hypercore-e2e-tests-seed-volume,destination=/home/hypercore-e2e-tests/corestore \
--env HYPERCORE_E2E_PROMETHEUS_SECRET=... \
--env HYPERCORE_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY=... \
--env HYPERCORE_E2E_KEY=... \
--env HYPERCORE_E2E_LENGTH=... \
ghcr.io/holepunchto/hypercore-e2e-tests-seed
```
### Download
```bash
--env HYPERCORE_E2E_PROMETHEUS_SECRET=... \
--env HYPERCORE_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY=... \
--env HYPERCORE_E2E_KEY=... \
--env HYPERCORE_E2E_LENGTH=... \
ghcr.io/holepunchto/hypercore-e2e-tests-download
```
## Environment Variables
- `HYPERCORE_E2E_KEY` - Core key to seed/download
- `HYPERCORE_E2E_LENGTH` - Core length to create/verify
- `HYPERCORE_E2E_PROMETHEUS_SECRET` - Optional metrics secret
- `HYPERCORE_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY` - Metrics scraper key
## Complete Examples
### Example 1: Local E2E Flow
```bash
# Terminal 1: Create core
hypercore-e2e-create | pino-pretty
# Copy printed key
# Terminal 2: Seed
HYPERCORE_E2E_KEY=<key> hypercore-e2e-seed | pino-pretty
# Terminal 3: Download
HYPERCORE_E2E_KEY=<key> hypercore-e2e-download | pino-pretty
```
### Example 2: CI Script
```bash
#!/bin/bash
set -e
KEY=$(hypercore-e2e-create | jq -r '.key')
HYPERCORE_E2E_KEY=$KEY hypercore-e2e-seed &
SEED_PID=$!
sleep 2
HYPERCORE_E2E_KEY=$KEY hypercore-e2e-download
kill $SEED_PID
```
### Example 3: Docker Compose
```yaml
version: '3'
services:
create:
image: ghcr.io/holepunchto/hypercore-e2e-tests-create
network_mode: host
volumes:
- create-volume:/home/hypercore-e2e-tests/corestore
seed:
image: ghcr.io/holepunchto/hypercore-e2e-tests-seed
network_mode: host
volumes:
- seed-volume:/home/hypercore-e2e-tests/corestore
download:
image: ghcr.io/holepunchto/hypercore-e2e-tests-download
network_mode: host
volumes:
create-volume:
seed-volume:
```
## Best Practices
### Use pino-pretty
```bash
hypercore-e2e-create | pino-pretty
```
### Isolate Storage
```bash
--mount type=volume,source=hypercore-e2e-tests-volume,destination=/home/hypercore-e2e-tests/corestore
```
### Validate Length
```bash
HYPERCORE_E2E_LENGTH=100000
```
## License
Apache-2.0
---
**Module Type**: Testing | **Ecosystem Role**: Replication E2E | **Dependencies**: hypercore, corestore
+164
View File
@@ -0,0 +1,164 @@
# hypercore-errors - Hypercore Error Types
## Overview
hypercore-errors defines standardized error constructors used across the Hypercore ecosystem. It provides typed error helpers for common failure modes.
### Key Features
- **Standardized errors**: Consistent error types
- **Simple constructors**: Error factories for each type
- **Interoperable**: Shared across Hypercore modules
### Use Cases
- **Error handling**: Match specific error types
- **Protocol logic**: Handle network and storage errors
- **Testing**: Assert specific failures
## Installation
```bash
npm install hypercore-errors
```
## Quick Start
```js
const { BLOCK_NOT_AVAILABLE } = require('hypercore-errors')
throw BLOCK_NOT_AVAILABLE()
```
## Error List
- `BAD_ARGUMENT`
- `STORAGE_EMPTY`
- `STORAGE_CONFLICT`
- `INVALID_SIGNATURE`
- `INVALID_CAPABILITY`
- `INVALID_CHECKSUM`
- `INVALID_OPERATION`
- `INVALID_PROOF`
- `BLOCK_NOT_AVAILABLE`
- `SNAPSHOT_NOT_AVAILABLE`
- `REQUEST_CANCELLED`
- `REQUEST_TIMEOUT`
- `SESSION_NOT_WRITABLE`
- `SESSION_CLOSED`
- `BATCH_UNFLUSHED`
- `BATCH_ALREADY_EXISTS`
- `BATCH_ALREADY_FLUSHED`
- `OPLOG_CORRUPT`
- `OPLOG_HEADER_OVERFLOW`
- `INVALID_OPLOG_VERSION`
- `WRITE_FAILED`
## API Reference
Each exported name is a factory returning an Error instance.
```js
const { INVALID_SIGNATURE } = require('hypercore-errors')
throw INVALID_SIGNATURE()
```
## Complete Examples
### Example 1: Handle Specific Errors
```js
const errors = require('hypercore-errors')
try {
await core.get(100)
} catch (err) {
if (err.code === errors.BLOCK_NOT_AVAILABLE().code) {
console.log('Block not available yet')
} else {
throw err
}
}
```
### Example 2: Wrapping Errors
```js
const { INVALID_OPERATION } = require('hypercore-errors')
function assertWritable(core) {
if (!core.writable) {
throw INVALID_OPERATION('Core is read-only')
}
}
```
### Example 3: Error Mapping
```js
const errors = require('hypercore-errors')
function mapError(err) {
switch (err.code) {
case 'BLOCK_NOT_AVAILABLE':
return 'Data not yet synced'
case 'REQUEST_TIMEOUT':
return 'Network timeout'
case 'INVALID_SIGNATURE':
return 'Data integrity check failed'
default:
return 'Unknown error'
}
}
```
### Example 4: Custom Error Handling
```js
const {
REQUEST_TIMEOUT,
REQUEST_CANCELLED,
SESSION_CLOSED
} = require('hypercore-errors')
async function safeRequest(fn) {
try {
return await fn()
} catch (err) {
if (err.code === REQUEST_TIMEOUT().code) {
console.log('Request timed out, retrying...')
} else if (err.code === REQUEST_CANCELLED().code) {
console.log('Request cancelled')
} else if (err.code === SESSION_CLOSED().code) {
console.log('Session closed')
} else {
throw err
}
}
}
```
## Best Practices
### Compare by Code
```js
if (err.code === BLOCK_NOT_AVAILABLE().code) {
// handle
}
```
### Use Typed Errors
```js
const { INVALID_PROOF } = require('hypercore-errors')
throw INVALID_PROOF('Proof failed verification')
```
## License
Apache-2.0
---
**Module Type**: Utility | **Ecosystem Role**: Error Types | **Dependencies**: None
+154
View File
@@ -0,0 +1,154 @@
# hypercore-id-encoding - Hypercore ID Encoding
## Overview
hypercore-id-encoding encodes Hypercore keys into z-base32 IDs and decodes both z-base32 and hex strings. It provides consistent ID formats for Hypercore keys.
### Key Features
- **z-base32 encoding**: User-friendly key IDs
- **Hex decoding**: Accepts hex and z-base32
- **Normalization**: Always returns z-base32
- **Validation**: Check ID validity
### Use Cases
- **Shareable IDs**: Short, user-friendly IDs
- **Input normalization**: Accept multiple formats
- **Key validation**: Verify keys before use
## Installation
```bash
npm install hypercore-id-encoding
```
## Quick Start
```js
const { encode, decode, normalize } = require('hypercore-id-encoding')
const id = encode(core.key) // z-base32 string
const hex = core.key.toString('hex')
const key1 = decode(id)
const key2 = decode(hex)
const normalized = normalize(hex) // z-base32
```
## API Reference
### `encode(hypercoreKey)`
Encode a 32-byte key into z-base32.
**Parameters:**
- `hypercoreKey` (Buffer | ArrayBuffer)
**Returns:** string (z-base32)
### `decode(hypercoreId)`
Decode z-base32 or hex into a key.
**Parameters:**
- `hypercoreId` (string | Buffer)
**Returns:** Buffer
**Rules:**
- 52-char string: z-base32
- 64-char string: hex
- Buffer: returned if valid
### `normalize(any)`
Decode and re-encode to z-base32.
**Returns:** string (z-base32)
### `isValid(any)`
Check if value is a valid key.
**Returns:** boolean
## Complete Examples
### Example 1: Accept Multiple Formats
```js
const { decode } = require('hypercore-id-encoding')
function openCore(store, id) {
const key = decode(id) // accepts z-base32 or hex
return store.get({ key })
}
```
### Example 2: Normalize User Input
```js
const { normalize } = require('hypercore-id-encoding')
function normalizeInput(input) {
try {
return normalize(input)
} catch (err) {
return null
}
}
const id = normalizeInput('0123abcd...')
if (id) console.log('Normalized ID:', id)
```
### Example 3: Validate IDs
```js
const { isValid } = require('hypercore-id-encoding')
function validate(id) {
if (!isValid(id)) {
throw new Error('Invalid Hypercore ID')
}
return true
}
```
### Example 4: Shareable Links
```js
const { encode } = require('hypercore-id-encoding')
function makeLink(key) {
const id = encode(key)
return `pear://${id}`
}
const link = makeLink(core.key)
console.log('Share link:', link)
```
## Best Practices
### Normalize Early
```js
const id = normalize(input)
// Use id consistently
```
### Validate Before Use
```js
if (!isValid(input)) throw new Error('Bad key')
```
## License
Apache-2.0
---
**Module Type**: Utility | **Ecosystem Role**: Key Encoding | **Dependencies**: None
+181
View File
@@ -0,0 +1,181 @@
# hypercore-logger - Distributed Logging for Hypercore
## Overview
hypercore-logger provides a distributed logger backed by Hypercore. It logs structured messages with system stats and supports live tailing (like `tail -f`) across peers.
### Key Features
- **Hypercore-backed**: Logs replicate via Hypercore
- **Structured entries**: Timestamp, stats, subsystem, message
- **Live tail**: Stream log entries as they arrive
- **CLI**: Built-in tailing CLI
- **Hyperswarm ready**: Works with standard replication
### Use Cases
- **Distributed debugging**: Collect logs from peers
- **Runtime diagnostics**: CPU/memory stats with logs
- **Operational visibility**: P2P app telemetry
## Installation
```bash
npm install hypercore-logger
```
## Quick Start
```js
const Logger = require('hypercore-logger')
const log = new Logger(core)
await log.ready()
await log.log({ hello: 'world' })
for await (const { timestamp, stats, message } of log.tail()) {
console.log(timestamp, stats, message)
}
```
## API Reference
### `new Logger(core)`
Create a new logger backed by a Hypercore.
### `log.key`
The Hypercore key.
### `log.discoveryKey`
The Hypercore discovery key.
### `await log.ready()`
Wait for logger to open.
### `await log.log(...msg)`
Log a message (stringified like `console.log`).
Each entry looks like:
```js
{
timestamp: Date.now(),
stats: {
cpus, cpu, cpuThread, cpuDelay,
rss, heapUsed, heapTotal, external
},
subsystem: log.subsystem,
message: '...'
}
```
### `log.tail(opts)`
Return a readable stream of log entries.
**Options:**
- `start` (number): Start index
- `end` (number): End index
### `await log.find(opts)`
Find range by timestamp.
**Options:**
- `gte` / `gt`: Start at >= or > timestamp
- `lte` / `lt`: End before > or >= timestamp
### `await log.close()`
Close logger and underlying core.
## CLI
```bash
npm install -g hypercore-logger
hypercore-logger --key <key>
```
**Flags:**
- `--key, -k` Log core key
- `--peer, -p` Noise key of peer
- `--storage, -s` Storage directory
## Complete Examples
### Example 1: Distributed Logging
```js
const Logger = require('hypercore-logger')
const Hyperswarm = require('hyperswarm')
const log = new Logger(core)
await log.ready()
const swarm = new Hyperswarm()
swarm.join(log.discoveryKey)
swarm.on('connection', (conn) => core.replicate(conn))
await log.log('Node started', { version: '1.0.0' })
```
### Example 2: Live Tail
```js
const log = new Logger(core)
await log.ready()
for await (const entry of log.tail()) {
console.log(new Date(entry.timestamp).toISOString(), entry.message)
}
```
### Example 3: Filter by Time
```js
const start = Date.now() - 60 * 60 * 1000 // last hour
const { start: s, end } = await log.find({ gte: start })
for await (const entry of log.tail({ start: s, end })) {
console.log(entry.message)
}
```
### Example 4: Subsystem Tagging
```js
const log = new Logger(core)
log.subsystem = 'network'
await log.log('Connected peer', peerId)
```
## Best Practices
### Replicate the Backing Core
```js
swarm.join(log.discoveryKey)
swarm.on('connection', (conn) => core.replicate(conn))
```
### Use Subsystems
```js
log.subsystem = 'db'
await log.log('Migration complete')
```
## License
Apache-2.0
---
**Module Type**: Observability | **Ecosystem Role**: Distributed Logging | **Dependencies**: hypercore, hyperschema
+158
View File
@@ -0,0 +1,158 @@
# hypercore-proof-queue - Proof Queue
## Overview
hypercore-proof-queue provides a simple, file-backed queue for Hypercore proofs. It allows one process to push proofs and another to consume them later or concurrently.
### Key Features
- **File-backed queue**: Persist proofs to disk
- **Simple API**: push + consume
- **Concurrent use**: Producer/consumer across processes
- **Hypercore proofs**: Designed for proof objects
### Use Cases
- **Replication pipelines**: Buffer proofs between processes
- **Background verification**: Queue proofs for later validation
- **Crash recovery**: Persist proofs across restarts
## Installation
```bash
npm install hypercore-proof-queue
```
## Quick Start
### Producer
```js
const HPQ = require('hypercore-proof-queue')
const q = new HPQ('/tmp/my-queue')
q.push({
discoveryKey,
fork: 0,
block: {
index: 10,
value: Buffer.from('hello'),
nodes: []
}
})
```
### Consumer
```js
const HPQ = require('hypercore-proof-queue')
const q = new HPQ('/tmp/my-queue', async function (proofs) {
console.log('incoming proofs', proofs)
})
```
## API Reference
### `new HPQ(path[, onproofs])`
Create a queue.
**Parameters:**
- `path` (string): Queue file path
- `onproofs` (function): Consumer callback
### `q.push(proof)`
Push a proof into the queue.
**Proof format:**
```js
{
discoveryKey,
fork,
block: {
index,
value,
nodes
}
}
```
## Complete Examples
### Example 1: Replication Pipeline
```js
const HPQ = require('hypercore-proof-queue')
// Producer: push proofs from replication
function onProof(discoveryKey, proof) {
q.push({ discoveryKey, ...proof })
}
// Consumer: verify proofs in background
const q = new HPQ('/tmp/proofs', async (proofs) => {
for (const proof of proofs) {
await verifyProof(proof)
}
})
```
### Example 2: Crash Recovery
```js
const HPQ = require('hypercore-proof-queue')
// On restart, pending proofs are still in file
const q = new HPQ('/tmp/proofs', async (proofs) => {
for (const proof of proofs) {
await processProof(proof)
}
})
```
### Example 3: Multi-Producer
```js
// Process A
const q = new HPQ('/tmp/shared-queue')
q.push(proofA)
// Process B
const q2 = new HPQ('/tmp/shared-queue')
q2.push(proofB)
```
### Example 4: Logging Proofs
```js
const q = new HPQ('/tmp/proofs', async (proofs) => {
console.log(`Received ${proofs.length} proofs`)
for (const p of proofs) {
console.log('Proof for', p.discoveryKey.toString('hex'))
}
})
```
## Best Practices
### Use Stable File Paths
```js
const q = new HPQ('/var/lib/myapp/proofs.queue')
```
### Validate Proof Objects
```js
if (!proof.discoveryKey || !proof.block) throw new Error('Invalid proof')
```
## License
Apache-2.0
---
**Module Type**: Utility | **Ecosystem Role**: Proof Buffering | **Dependencies**: None
+92
View File
@@ -0,0 +1,92 @@
# hypercore-scale-tests - Hypercore Scaling Experiments
## Overview
hypercore-scale-tests runs scaling experiments for Hypercore. It includes an experiment runner and a Prometheus metrics exporter, with results stored in a Hyperbee.
### Key Features
- **Experiment runner**: Continuous experiment execution
- **Metrics exporter**: `/metrics` Prometheus endpoint
- **Hyperbee storage**: Persist experiment results
- **Docker deployment**: Designed for containerized use
### Use Cases
- **Performance benchmarking**: Measure Hypercore scaling
- **Regression testing**: Detect performance regressions
- **Infrastructure monitoring**: Long-running experiment tracking
## Installation
```bash
npm install hypercore-scale-tests
```
## Quick Start
```bash
npm i
node run.js | pino-pretty
```
## Architecture
```mermaid
Runner[Experiment Runner] --> Bee[Hyperbee Results]
Bee --> Metrics[Prometheus Exporter]
Metrics --> Scraper[Prometheus Scraper]
```
## Configuration
### ENV Variables
Configured via `run.js` and env vars.
### Experiments File
Experiments defined in `config.json` (see `example-config.json`).
## Docker Deployment
```bash
sudo docker pull ghcr.io/holepunchto/hypercore-scale-tests:build-with-latest-deps
sudo docker stop hypercore-scale-tests
sudo docker rm hypercore-scale-tests
sudo docker run -d -p 127.0.0.1:52416:8080 \
--env HYPERCORE_SCALE_TEST_INTERVAL_MS=300000 \
--env HYPERCORE_SCALE_EXPERIMENTS_FILE_LOC=/home/runner/config/config.json \
--name hypercore-scale-tests \
--mount type=volume,source=hypercore-scale-tests-volume,destination=/home/runner/corestore \
--mount type=bind,source=/etc/hypercore-scale-experiments,destination=/home/runner/config,readonly \
--restart=on-failure \
--memory=1024M \
ghcr.io/holepunchto/hypercore-scale-tests:build-with-latest-deps
```
## Adding Experiments
1. Extend `Experiment` class
2. Implement `_runExperiment`
3. Add to `parseExperimentsConfig` in `run.js`
4. Update `example-config.json`
## Best Practices
### Keep Experiments Cancelable
```js
if (this.closing) return
```
### Separate Runner + Metrics if Needed
When experiments exceed single-process capacity, deploy runner and metrics separately and replicate the Hyperbee.
## License
Apache-2.0
---
**Module Type**: Testing | **Ecosystem Role**: Scaling Benchmarks | **Dependencies**: hyperbee, prom-client
+153
View File
@@ -0,0 +1,153 @@
# hypercore-signing-request - Shareable Signing Requests
## Overview
hypercore-signing-request generates shareable signing requests for Hypercore and Hyperdrive. These requests can be signed offline for multisig workflows.
### Key Features
- **Shareable requests**: Buffer format for distribution
- **Hypercore support**: Manifest-backed cores
- **Hyperdrive support**: Joint requests for metadata + blobs
- **Decoding**: Inspect request content
- **Signable buffer**: Validate signer authorization
### Use Cases
- **Manual multisig**: Offline signing workflows
- **Release approval**: Sign specific core lengths
- **Drive signing**: Sign both metadata and blobs
## Installation
```bash
npm install hypercore-signing-request
```
## Quick Start
```js
const { generate, decode, signable } = require('hypercore-signing-request')
const request = await generate(core, { length: core.length })
const decoded = decode(request)
console.log(decoded)
const bufferToSign = signable(publicKey, decoded)
```
## API Reference
### `generate(coreOrDrive, { length })`
Generate a signing request.
**Parameters:**
- `coreOrDrive` - Hypercore or Hyperdrive
- `length` (number): Core length to sign (default: core.length)
**Returns:** Buffer
**Notes:**
- Only manifest-backed cores
- Hyperdrive: only v1 manifest-backed
### `decode(requestBuffer)`
Decode a signing request.
**Returns:**
```js
{
version,
id,
key,
manifest,
treeHash,
length,
fork
}
```
### `signable(publicKey, req)`
Get the buffer that should be signed.
Validates signer eligibility.
## Complete Examples
### Example 1: Generate and Decode
```js
const { generate, decode } = require('hypercore-signing-request')
const req = await generate(core, { length: core.length })
const info = decode(req)
console.log('Core key:', info.key.toString('hex'))
console.log('Length:', info.length)
```
### Example 2: Offline Signing
```js
const { generate, decode, signable } = require('hypercore-signing-request')
const crypto = require('bare-crypto')
const req = await generate(core)
const decoded = decode(req)
// Sign with ed25519 key
const buffer = signable(publicKey, decoded)
const signature = crypto.sign('ed25519', buffer, privateKey)
```
### Example 3: Hyperdrive Request
```js
const { generate } = require('hypercore-signing-request')
const Hyperdrive = require('hyperdrive')
const drive = new Hyperdrive(store)
await drive.ready()
const request = await generate(drive)
// Request includes metadata + blob cores
```
### Example 4: Shareable Format
```js
const { generate } = require('hypercore-signing-request')
const z32 = require('z32')
const request = await generate(core)
const encoded = z32.encode(request)
console.log('Share this request:', encoded)
```
## Best Practices
### Validate Before Signing
```js
const decoded = decode(req)
const buffer = signable(publicKey, decoded)
```
### Use Length Pinning
```js
// Sign a specific length
const req = await generate(core, { length: 1000 })
```
## License
Apache-2.0
---
**Module Type**: Security | **Ecosystem Role**: Signing Workflow | **Dependencies**: hypercore
+136
View File
@@ -0,0 +1,136 @@
# hypercore-stats - Hypercore Metrics
## Overview
hypercore-stats collects metrics for Hypercore replication, with Prometheus support. It assumes cores replicate over UDX streams (standard in Hypercore/Hyperswarm).
### Key Features
- **Prometheus metrics**: Export to prom-client
- **Corestore integration**: Collect metrics from all cores
- **Human-readable output**: `toString()`
- **JSON output**: `toJson()`
### Use Cases
- **Monitoring**: Track replication performance
- **Observability**: Export metrics for dashboards
- **Debugging**: Inspect core activity
## Installation
```bash
npm install hypercore-stats
```
## Quick Start
```js
const Corestore = require('corestore')
const HypercoreStats = require('hypercore-stats')
const promClient = require('prom-client')
const store = new Corestore('storage')
const stats = await HypercoreStats.fromCorestore(store)
stats.registerPrometheusMetrics(promClient)
const metrics = await promClient.register.metrics()
console.log(metrics)
```
## Versions
- **V1**: Hypercore v10 + Corestore v6
- **V2**: Hypercore v11 + Corestore v7
## API Reference
### `HypercoreStats.fromCorestore(store)`
Create stats collector from corestore.
**Returns:** HypercoreStats
### `stats.registerPrometheusMetrics(promClient)`
Register Prometheus metrics.
### `stats.toString()`
Return human-readable stats.
### `stats.toJson()`
Return JSON stats.
## Complete Examples
### Example 1: Prometheus Export
```js
const Corestore = require('corestore')
const HypercoreStats = require('hypercore-stats')
const promClient = require('prom-client')
const http = require('bare-http1')
const store = new Corestore('storage')
const stats = await HypercoreStats.fromCorestore(store)
stats.registerPrometheusMetrics(promClient)
const server = http.createServer(async (req, res) => {
if (req.url === '/metrics') {
res.setHeader('Content-Type', promClient.register.contentType)
res.end(await promClient.register.metrics())
} else {
res.statusCode = 404
res.end('Not found')
}
})
server.listen(9100)
```
### Example 2: CLI Stats
```js
const Corestore = require('corestore')
const HypercoreStats = require('hypercore-stats')
const store = new Corestore('storage')
const stats = await HypercoreStats.fromCorestore(store)
setInterval(() => {
console.log(stats.toString())
}, 5000)
```
### Example 3: JSON Export
```js
const stats = await HypercoreStats.fromCorestore(store)
const json = stats.toJson()
console.log(JSON.stringify(json, null, 2))
```
## Best Practices
### Match Version
```js
// Ensure stats version matches hypercore/corestore version
```
### Export Metrics Endpoint
```js
// Use /metrics endpoint for Prometheus scraping
```
## License
Apache-2.0
---
**Module Type**: Observability | **Ecosystem Role**: Metrics | **Dependencies**: corestore, prom-client
+97
View File
@@ -0,0 +1,97 @@
# hyperdht-stats - HyperDHT Metrics
## Overview
hyperdht-stats collects metrics for HyperDHT and exposes them via Prometheus or JSON/text output.
### Key Features
- **Prometheus integration**: Register metrics with prom-client
- **Text output**: `toString()` overview
- **JSON output**: `toJson()` overview
### Use Cases
- **Monitoring**: DHT health metrics
- **Observability**: Export metrics for dashboards
- **Diagnostics**: Inspect DHT activity
## Installation
```bash
npm install hyperdht-stats
```
## Quick Start
```js
const HyperDHT = require('hyperdht')
const DhtStats = require('hyperdht-stats')
const promClient = require('prom-client')
const dht = new HyperDHT()
const stats = new DhtStats(dht)
stats.registerPrometheusMetrics(promClient)
const metrics = await promClient.register.metrics()
console.log(metrics)
```
## API Reference
### `new DhtStats(dht)`
Create stats collector for a DHT instance.
### `stats.registerPrometheusMetrics(promClient)`
Register metrics with prom-client.
### `stats.toString()`
Text summary of metrics.
### `stats.toJson()`
JSON summary of metrics.
## Complete Examples
### Example 1: Metrics Endpoint
```js
const HyperDHT = require('hyperdht')
const DhtStats = require('hyperdht-stats')
const promClient = require('prom-client')
const http = require('bare-http1')
const dht = new HyperDHT()
const stats = new DhtStats(dht)
stats.registerPrometheusMetrics(promClient)
http.createServer(async (req, res) => {
if (req.url === '/metrics') {
res.setHeader('Content-Type', promClient.register.contentType)
res.end(await promClient.register.metrics())
} else {
res.statusCode = 404
res.end('Not found')
}
}).listen(9101)
```
### Example 2: Logging Stats
```js
setInterval(() => {
console.log(stats.toString())
}, 5000)
```
## License
Apache-2.0
---
**Module Type**: Observability | **Ecosystem Role**: DHT Metrics | **Dependencies**: hyperdht, prom-client
+117
View File
@@ -0,0 +1,117 @@
# hyperdrive-swarm-test - Hyperdrive Swarm Performance Test
## Overview
hyperdrive-swarm-test is a simple performance test for Hyperdrive replication over Hyperswarm. It includes a server that seeds a large file and a client that downloads it while measuring throughput.
### Key Features
- **Server/Client scripts**: Seed and download
- **Throughput tracking**: Speedometer + tiny-byte-size
- **Swarm replication**: Uses Hyperswarm
- **Configurable storage**: Custom storage paths
### Use Cases
- **Replication benchmarking**: Measure download speed
- **Network diagnostics**: Test swarm connectivity
- **Performance regression**: Compare changes over time
## Usage
### Server
```bash
node server.js --storage /tmp/server
```
The server prints a drive key to share with clients.
### Client
```bash
node client.js --key <drive-key> --storage /tmp/client
```
## How It Works
- Server creates a Hyperdrive and writes a large file (if needed)
- Server announces via Hyperswarm
- Client connects and downloads the file
- Both measure upload/download speed
## Complete Examples
### Example 1: Run Server
```bash
node server.js --storage /tmp/server
# Output:
# storing corestore in /tmp/server
# run: node client.js --key=<drive-id>
```
### Example 2: Run Client
```bash
node client.js --key <drive-id> --storage /tmp/client
# Output:
# total 64.0MB speed 4.2MB/s peers 1
```
## Implementation Notes
Server (simplified):
```js
const store = new Corestore('/tmp/server')
const drive = new Hyperdrive(store)
await drive.ready()
// Write large file if needed
if (drive.version < 2) {
const ws = drive.createWriteStream('/file')
for (let i = 0; i < 32768; i++) ws.write(Buffer.alloc(65536))
ws.end()
}
const swarm = new Hyperswarm({ keyPair: await store.createKeyPair('swarming') })
swarm.on('connection', (c) => drive.replicate(c))
swarm.join(drive.discoveryKey)
```
Client (simplified):
```js
const drive = new Hyperdrive(store, key)
await drive.ready()
const swarm = new Hyperswarm({ keyPair: await store.createKeyPair('swarming') })
swarm.on('connection', (c) => drive.replicate(c))
swarm.join(drive.discoveryKey)
await drive.getBlobs()
drive.download('/file')
```
## Best Practices
### Use Separate Storage
```bash
--storage /tmp/server
--storage /tmp/client
```
### Measure Consistently
Run the test multiple times under similar network conditions.
## License
ISC
---
**Module Type**: Testing | **Ecosystem Role**: Hyperdrive Performance | **Dependencies**: hyperdrive, hyperswarm
+68
View File
@@ -0,0 +1,68 @@
# hyperswarm-capability - Stream-Coupled Capabilities
## Overview
hyperswarm-capability produces a stream-coupled capability for a key. It enables two peers to prove they share a secret key over a specific stream.
### Key Features
- **Stream-bound**: Capability tied to stream
- **Shared secret**: Requires shared key
- **Simple API**: generate + verify
### Use Cases
- **Authentication**: Prove key knowledge
- **Capability gating**: Allow/deny stream actions
- **Secure handshakes**: Lightweight proof
## Installation
```bash
npm install hyperswarm-capability
```
## Quick Start
```js
const HyperswarmCapability = require('hyperswarm-capability')
const c = new HyperswarmCapability()
const key = Buffer.from('shared-secret')
const cap = c.generate(stream, key)
// send cap to peer
if (c.verify(stream, key, cap)) {
console.log('Capability verified')
}
```
## API Reference
### `new HyperswarmCapability()`
Create capability helper.
### `cap = c.generate(stream, key)`
Generate capability for stream/key.
### `c.verify(stream, key, cap)`
Verify capability.
**Returns:** boolean
## Best Practices
- Use strong random keys
- Bind to the actual stream you authenticate
## License
Apache-2.0
---
**Module Type**: Security | **Ecosystem Role**: Stream Auth | **Dependencies**: hyperswarm
+51
View File
@@ -0,0 +1,51 @@
# @hyperswarm/doctor - Hyperswarm Debug Tool
## Overview
@hyperswarm/doctor is a CLI debugging tool for Hyperswarm. It can print environment diagnostics and run a server/client transfer test.
### Key Features
- **Diagnostics**: Print swarm environment info
- **Test server**: Spin up test server
- **Test client**: Connect and test transfer
## Installation
```bash
npm install -g @hyperswarm/doctor
```
## Usage
```bash
hyperswarm-doctor
hyperswarm-doctor --server
hyperswarm-doctor --client=pubkey
```
## Examples
### Server
```bash
hyperswarm-doctor --server
```
### Client
```bash
hyperswarm-doctor --client=<server-public-key>
```
## Best Practices
- Run server and client on different hosts for full test
- Use when debugging connectivity issues
## License
Apache-2.0
---
**Module Type**: Tooling | **Ecosystem Role**: Networking Diagnostics | **Dependencies**: hyperswarm
+75
View File
@@ -0,0 +1,75 @@
# hyperswarm-e2e-tests - Hyperswarm End-to-End Tests
## Overview
hyperswarm-e2e-tests provides Docker-based end-to-end tests for Hyperswarm transfer performance. It includes server and client images for seeding and downloading a test file over Hyperswarm.
### Key Features
- **Docker images**: Server and client containers
- **Prometheus integration**: Optional metrics scraper
- **Large file testing**: Seed specific file sizes
- **Host networking**: Realistic network conditions
### Use Cases
- **Swarm validation**: Ensure Hyperswarm works end-to-end
- **Performance tests**: Measure transfer speeds
- **Infrastructure monitoring**: Track network health
## Docker Usage
### Prepare File
```bash
fallocate -l 1G file-to-seed
```
### Server
```bash
--mount type=bind,source=/path/of/dir/with/file-to-seed,destination=/home/hyperswarm-e2e-tests/serve/ \
--env HYPERSWARM_E2E_PROMETHEUS_ALIAS=unique-prom-dht-alias \
--env HYPERSWARM_E2E_PROMETHEUS_SECRET=... \
--env HYPERSWARM_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY=... \
--env HYPERSWARM_E2E_DISCOVERY_KEY=unique-disc-key-to-serve-under \
ghcr.io/holepunchto/hyperswarm-e2e-tests-server
```
### Client
```bash
--env HYPERSWARM_E2E_PROMETHEUS_ALIAS=unique-prom-dht-alias \
--env HYPERSWARM_E2E_PROMETHEUS_SECRET=... \
--env HYPERSWARM_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY=... \
--env HYPERSWARM_E2E_DISCOVERY_KEY=disc-key-of-the-server \
ghcr.io/holepunchto/hyperswarm-e2e-tests-client
```
## Environment Variables
- `HYPERSWARM_E2E_DISCOVERY_KEY` - Discovery key for swarm
- `HYPERSWARM_E2E_PROMETHEUS_ALIAS` - Metrics alias
- `HYPERSWARM_E2E_PROMETHEUS_SECRET` - Metrics auth secret
- `HYPERSWARM_E2E_PROMETHEUS_SCRAPER_PUBLIC_KEY` - Scraper key
## Best Practices
### Use Host Networking
```bash
--network=host
```
### Seed Large Files
```bash
fallocate -l 1G file-to-seed
```
## License
Apache-2.0
---
**Module Type**: Testing | **Ecosystem Role**: Swarm E2E | **Dependencies**: hyperswarm
+85
View File
@@ -0,0 +1,85 @@
# @hyperswarm/seeders - Seeders-Only Swarm
## Overview
@hyperswarm/seeders provides a swarm that only connects to seeders, verified by a mutable record in the DHT. It enables a trusted list of seeds for availability.
### Key Features
- **Seeders-only**: Connect only to registered seeds
- **Verifiable record**: Stored in DHT, tamper-resistant
- **Publicly readable**: Seeds list is not private
- **Optional core info**: Publish core length/fork
### Use Cases
- **Controlled replication**: Only trusted seeders
- **Public availability**: Share seeder list
- **Resilience**: Maintain seed set
## Installation
```bash
npm install @hyperswarm/seeders
```
## Quick Start
```js
const Seeders = require('@hyperswarm/seeders')
const swarm = new Seeders(firstSeedPublicKey, {
dht,
keyPair,
maxClientConnections: 2
})
swarm.on('connection', (conn) => {
console.log('got connection...')
})
if (swarm.owner) {
await swarm.join({
seeds: [publicKey1, publicKey2],
core: { length: 42, fork: 0 }
})
} else {
await swarm.join()
}
```
## API Reference
### `new Seeders(firstSeedPublicKey, options)`
**Options:**
- `dht` - DHT instance
- `keyPair` - Keypair for identity
- `maxClientConnections` - Max connections (default: 2)
### `swarm.owner`
True if this instance owns the seed record.
### `await swarm.join([record])`
Join swarm. If owner, supply record:
```js
{
seeds: [publicKey1, publicKey2],
core: { length, fork }
}
```
## Best Practices
- Use a stable DHT keypair for seed ownership
- Publish updated seed list when adding seeds
## License
Apache-2.0
---
**Module Type**: Networking | **Ecosystem Role**: Seed Control | **Dependencies**: hyperdht, hyperswarm
+90
View File
@@ -0,0 +1,90 @@
# hyperswarm-stats - Hyperswarm Metrics
## Overview
hyperswarm-stats collects metrics for Hyperswarm and the underlying DHT, with Prometheus support.
### Key Features
- **Prometheus integration**: Register metrics
- **Text output**: `toString()` overview
- **JSON output**: `toJson()` overview
### Use Cases
- **Monitoring**: Swarm health metrics
- **Observability**: Dashboard metrics
- **Diagnostics**: Inspect swarm behavior
## Installation
```bash
npm install hyperswarm-stats
```
## Quick Start
```js
const Hyperswarm = require('hyperswarm')
const HyperswarmStats = require('hyperswarm-stats')
const promClient = require('prom-client')
const swarm = new Hyperswarm()
const stats = new HyperswarmStats(swarm)
stats.registerPrometheusMetrics(promClient)
const metrics = await promClient.register.metrics()
console.log(metrics)
```
## API Reference
### `new HyperswarmStats(swarm)`
Create stats collector.
### `stats.registerPrometheusMetrics(promClient)`
Register metrics.
### `stats.toString()`
Text summary.
### `stats.toJson()`
JSON summary.
## Complete Examples
### Example 1: Metrics Endpoint
```js
const http = require('bare-http1')
http.createServer(async (req, res) => {
if (req.url === '/metrics') {
res.setHeader('Content-Type', promClient.register.contentType)
res.end(await promClient.register.metrics())
} else {
res.statusCode = 404
res.end('Not found')
}
}).listen(9102)
```
### Example 2: Logging
```js
setInterval(() => {
console.log(stats.toString())
}, 5000)
```
## License
Apache-2.0
---
**Module Type**: Observability | **Ecosystem Role**: Swarm Metrics | **Dependencies**: hyperswarm, prom-client
+112
View File
@@ -0,0 +1,112 @@
# @hyperswarm/testnet - Local DHT Testnet
## Overview
@hyperswarm/testnet helps you spin up a local Hyperswarm testnet. It creates multiple HyperDHT nodes and returns bootstrap addresses for testing.
### Key Features
- **Local testnet**: Spin up N DHT nodes
- **Bootstrap addresses**: Easy config for tests
- **Extra nodes**: Create additional ephemeral nodes
- **Iterable**: Iterate over nodes
### Use Cases
- **Unit tests**: Isolated swarm environment
- **Integration tests**: Multi-node DHT setups
- **Local development**: Debug DHT behavior
## Installation
```bash
npm install @hyperswarm/testnet
```
## Quick Start
```js
const createTestnet = require('@hyperswarm/testnet')
const testnet = await createTestnet(10) // 10 nodes
console.log(testnet.bootstrap)
```
## API Reference
### `createTestnet(size = 10, options)`
Create a new testnet.
**Options:**
- `port` (number): Preferred port
- `host` (string): Preferred host (default: 127.0.0.1)
- `teardown` (function): Optional teardown helper
### `testnet.nodes`
Array of DHT nodes.
### `testnet.bootstrap`
Bootstrap addresses array.
### `testnet.createNode(options)`
Create additional ephemeral node.
### `for (const node of testnet)`
Iterate over nodes.
## Complete Examples
### Example 1: Use in Tests
```js
const test = require('brittle')
const createTestnet = require('@hyperswarm/testnet')
const Hyperswarm = require('hyperswarm')
test('swarm connects', async (t) => {
const testnet = await createTestnet(3, { teardown: t.teardown })
const swarm = new Hyperswarm({ bootstrap: testnet.bootstrap })
const topic = Buffer.alloc(32).fill('test')
swarm.join(topic)
t.teardown(() => swarm.destroy())
})
```
### Example 2: Additional Nodes
```js
const testnet = await createTestnet(5)
const extra = testnet.createNode()
console.log('Total nodes:', testnet.nodes.length)
```
## Best Practices
### Use Teardown Hook
```js
const testnet = await createTestnet(5, { teardown: t.teardown })
```
### Reuse Bootstrap
```js
const swarm = new Hyperswarm({ bootstrap: testnet.bootstrap })
```
## License
Apache-2.0
---
**Module Type**: Testing | **Ecosystem Role**: Local DHT | **Dependencies**: hyperdht
+133
View File
@@ -0,0 +1,133 @@
# test-suspend - Process Suspension Testing
## Overview
test-suspend provides utilities for testing process suspension and resumption. It simulates suspend/idle/resume cycles for application testing.
### Key Features
- **Suspend control**: Pause process execution
- **Idle detection**: Wait until process idle
- **Resume**: Continue execution
- **Test-friendly**: Promise-based API
### Use Cases
- **Suspend/resume testing**: Mobile or background workflows
- **Power management**: Simulate OS suspensions
- **Reliability tests**: Ensure correct handling of idle
## Installation
```bash
npm install test-suspend
```
## Quick Start
```js
const suspend = require('test-suspend')
const s = await suspend()
console.log('suspended')
await s.idle()
console.log('became idle')
await s.resume()
console.log('resumed')
```
## API Reference
### `await suspend()`
Suspend the process.
**Returns:** Suspender object
### `s.idle()`
Wait until process becomes idle.
### `s.resume()`
Resume the process.
## Complete Examples
### Example 1: Background Task Test
```js
const suspend = require('test-suspend')
async function testBackgroundHandling() {
// Start background task
startTask()
// Suspend process
const s = await suspend()
console.log('Process suspended')
await s.idle()
console.log('Process idle')
// Resume
await s.resume()
console.log('Process resumed')
// Verify task handling
checkTaskState()
}
```
### Example 2: Test Suite Integration
```js
const test = require('brittle')
const suspend = require('test-suspend')
test('app survives suspend', async (t) => {
const app = startApp()
const s = await suspend()
await s.idle()
// Simulate app state change
app.onSuspend()
await s.resume()
app.onResume()
t.ok(app.isHealthy())
})
```
## Best Practices
### Use in Controlled Tests
```js
// Avoid in production
if (process.env.NODE_ENV === 'test') {
await suspend()
}
```
### Handle Timeouts
```js
const s = await suspend()
await Promise.race([
s.idle(),
new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 5000))
])
```
## License
Apache-2.0
---
**Module Type**: Testing | **Ecosystem Role**: Suspend Simulation | **Dependencies**: None