research/interconnections

Interconnections Guide

How Holepunch components compose to build complete P2P applications.

Overview

The Holepunch ecosystem is designed as a set of composable modules. Understanding how these components interconnect is key to building effective P2P applications.

flowchart TB
    subgraph "Application Layer"
        APP["Your Application"]
    end
    
    subgraph "Composition Patterns"
        P1["Hypercore + Swarm"]
        P2["Drive + Autobase"]
        P3["Bee + Multi-writer"]
        P4["Full Stack"]
    end
    
    subgraph "Core Components"
        CORE["Hypercore"]
        DRIVE["Hyperdrive"]
        BEE["Hyperbee"]
        AUTO["Autobase"]
        SWARM["Hyperswarm"]
        DHT["HyperDHT"]
        PMUX["Protomux"]
    end
    
    APP --> P1
    APP --> P2
    APP --> P3
    APP --> P4
    P1 --> CORE
    P1 --> SWARM
    P2 --> DRIVE
    P2 --> AUTO
    P3 --> BEE
    P3 --> AUTO
    P4 --> CORE
    P4 --> DRIVE
    P4 --> SWARM
    P4 --> DHT
    SWARM --> DHT
    CORE --> PMUX
    DRIVE --> CORE
    BEE --> CORE
    AUTO --> CORE

---

1. Hypercore + Hyperswarm

1.1 Pattern Overview

The simplest and most common pattern: replicate a single Hypercore over the network.

Use Cases:

• Chat messages
• Event logs
• Audit trails
• Simple databases

1.2 Architecture

sequenceDiagram
    participant App as Application
    participant Core as Hypercore
    participant Swarm as Hyperswarm
    participant DHT as HyperDHT
    participant Peer as Remote Peer
    
    App->>Core: Create feed
    Core-->>App: discoveryKey
    
    App->>Swarm: join(discoveryKey)
    Swarm->>DHT: announce(discoveryKey)
    
    Peer->>DHT: lookup(discoveryKey)
    DHT-->>Peer: App's address
    
    Peer->>Swarm: connect
    Swarm->>App: connection event
    
    App->>Core: replicate(conn)
    Core->>Peer: sync data

1.3 Implementation

const Hypercore = require('hypercore')
const Hyperswarm = require('hyperswarm')

// Create feed
const core = new Hypercore('./my-feed')
await core.ready()

// Join swarm
const swarm = new Hyperswarm()
swarm.join(core.discoveryKey, { server: true, client: true })

// Handle connections
swarm.on('connection', (conn) => {
  // Pipe replication stream to connection
  core.replicate(conn)
})

// Append data
await core.append(Buffer.from('Hello World'))

1.4 Data Flow

┌──────────────────────────────────────────────────────────┐
│                    Data Flow                             │
├──────────────────────────────────────────────────────────┤
│                                                          │
│  Application                                             │
│       │                                                  │
│       ▼                                                  │
│  ┌──────────────┐    append()    ┌──────────────┐       │
│  │  Hypercore   │ ◄──────────────│   Your Data  │       │
│  │  (storage)   │                │              │       │
│  └──────┬───────┘                └──────────────┘       │
│         │                                                │
│         │ replicate()                                    │
│         ▼                                                │
│  ┌──────────────┐    join()     ┌──────────────┐       │
│  │ Hyperswarm   │ ◄──────────────│  discoveryKey│       │
│  │  (network)   │                │              │       │
│  └──────┬───────┘                └──────────────┘       │
│         │                                                │
│         │ Noise Stream                                   │
│         ▼                                                │
│  ┌──────────────┐                                       │
│  │  Remote Peer │                                       │
│  └──────────────┘                                       │
│                                                          │
└──────────────────────────────────────────────────────────┘

---

2. Hyperdrive + Hyperswarm

2.1 Pattern Overview

Distribute files using Hyperdrive over P2P network.

Use Cases:

• File sharing
• Static site hosting
• Package distribution
• Media libraries

2.2 Architecture

flowchart TB
    subgraph "Local Node"
        APP["Application"]
        DRIVE["Hyperdrive"]
        BEE["Hyperbee<br/>Metadata"]
        BLOB["Hyperblobs<br/>Content"]
        CORE1["Hypercore 1"]
        CORE2["Hypercore 2"]
        SWARM["Hyperswarm"]
    end
    
    subgraph "Remote Peer"
        DRIVE2["Hyperdrive"]
        SWARM2["Hyperswarm"]
    end
    
    APP --> DRIVE
    DRIVE --> BEE
    DRIVE --> BLOB
    BEE --> CORE1
    BLOB --> CORE2
    DRIVE --> SWARM
    SWARM --> SWARM2
    SWARM2 --> DRIVE2

2.3 Implementation

const Hyperdrive = require('hyperdrive')
const Hyperswarm = require('hyperswarm')

// Create drive
const drive = new Hyperdrive('./my-drive')
await drive.ready()

// Write files
await drive.put('/hello.txt', Buffer.from('Hello World'))
await drive.put('/data.json', Buffer.from(JSON.stringify({ key: 'value' })))

// Join swarm
const swarm = new Hyperswarm()
swarm.join(drive.discoveryKey, { server: true, client: true })

// Replicate on connection
swarm.on('connection', (conn) => {
  drive.replicate(conn)
})

// Read files
const content = await drive.get('/hello.txt')
console.log(content.toString()) // 'Hello World'

2.4 Drive Structure

Hyperdrive Structure:

┌─────────────────────────────────────────────────┐
│              Hyperdrive                         │
├─────────────────────────────────────────────────┤
│                                                 │
│  ┌──────────────┐      ┌──────────────┐        │
│  │  Hyperbee    │      │ Hyperblobs   │        │
│  │  (Metadata)  │      │  (Content)   │        │
│  │              │      │              │        │
│  │ /hello.txt   │─────>│ "Hello..."   │        │
│  │   hash: abc  │      │              │        │
│  │   size: 11   │      │              │        │
│  │              │      │              │        │
│  │ /data.json   │─────>│ "{key:..."   │        │
│  │   hash: def  │      │              │        │
│  │   size: 16   │      │              │        │
│  └──────────────┘      └──────────────┘        │
│                                                 │
└─────────────────────────────────────────────────┘

---

3. Hyperbee + Hyperswarm

3.1 Pattern Overview

Distributed key-value database with P2P replication.

Use Cases:

• User profiles
• Configuration storage
• Indexed data
• Sorted collections

3.2 Architecture

flowchart TB
    subgraph "Hyperbee Structure"
        BEE["Hyperbee"]
        
        subgraph "B-Tree Index"
            ROOT["Root Node"]
            L1["Level 1"]
            L2["Level 2"]
            LEAF["Leaf Nodes"]
        end
        
        CORE["Hypercore<br/>(Underlying)"]
    end
    
    BEE --> ROOT
    ROOT --> L1
    L1 --> L2
    L2 --> LEAF
    LEAF --> CORE

3.3 Implementation

const Hyperbee = require('hyperbee')
const Hypercore = require('hypercore')
const Hyperswarm = require('hyperswarm')

// Create bee
const core = new Hypercore('./my-bee')
const bee = new Hyperbee(core, { keyEncoding: 'utf-8', valueEncoding: 'json' })
await bee.ready()

// Put data
await bee.put('user:1', { name: 'Alice', age: 30 })
await bee.put('user:2', { name: 'Bob', age: 25 })

// Join swarm
const swarm = new Hyperswarm()
swarm.join(core.discoveryKey, { server: true, client: true })

swarm.on('connection', (conn) => {
  core.replicate(conn)
})

// Query
const user = await bee.get('user:1')
console.log(user.value) // { name: 'Alice', age: 30 }

// Range query
const stream = bee.createReadStream({ gte: 'user:', lt: 'user;' })
for await (const entry of stream) {
  console.log(entry.key, entry.value)
}

---

4. Autobase + Multi-writer

4.1 Pattern Overview

Enable multiple writers to collaborate on a single dataset.

Use Cases:

• Collaborative editing
• Multi-user databases
• Shared workspaces
• Distributed governance

4.2 Architecture

flowchart TB
    subgraph "Multi-Writer Setup"
        subgraph "Writer A"
            COREA["Hypercore A"]
            APPA["App A"]
        end
        
        subgraph "Writer B"
            COREB["Hypercore B"]
            APPB["App B"]
        end
        
        subgraph "Writer C"
            COREC["Hypercore C"]
            APPC["App C"]
        end
        
        subgraph "Autobase"
            MERGE["Causal Merge"]
            VIEW["Linearized View"]
        end
    end
    
    COREA --> MERGE
    COREB --> MERGE
    COREC --> MERGE
    MERGE --> VIEW
    APPA --> COREA
    APPB --> COREB
    APPC --> COREC

4.3 Implementation

const Autobase = require('autobase')
const Hypercore = require('hypercore')
const Hyperswarm = require('hyperswarm')

// Create input feeds for each writer
const feedA = new Hypercore('./feed-a')
const feedB = new Hypercore('./feed-b')
const feedC = new Hypercore('./feed-c')

// Create autobase
const base = new Autobase({
  inputs: [feedA, feedB, feedC],
  localInput: feedA // This writer's feed
})

// Append (causally)
await base.append({ message: 'Hello from A' })

// Read linearized view
for await (const node of base.createReadStream()) {
  console.log(node.value)
}

// Replicate all inputs
const swarm = new Hyperswarm()
swarm.join(base.discoveryKey)

swarm.on('connection', (conn) => {
  feedA.replicate(conn)
  feedB.replicate(conn)
  feedC.replicate(conn)
})

4.4 Causal Ordering

Timeline:

Writer A: ───[A1]──────[A2]───────────────
                ↓        ↓
Writer B: ─────────[B1]────────[B2]───────
                       ↓         ↓
Writer C: ────────────────────[C1]──[C2]──

Linearized Order (by Autobase):
A1 → B1 → A2 → C1 → B2 → C2

(Causal order preserved: A1 before A2, B1 before B2, etc.)

---

5. Corestore + Multi-core

5.1 Pattern Overview

Efficiently manage multiple Hypercores with shared storage.

Use Cases:

• Multi-feed applications
• Namespaced data
• Efficient storage
• Batch operations

5.2 Architecture

flowchart TB
    subgraph "Corestore"
        CS["Corestore"]
        
        subgraph "Namespace A"
            CORE1["Hypercore 1"]
            CORE2["Hypercore 2"]
        end
        
        subgraph "Namespace B"
            CORE3["Hypercore 3"]
            CORE4["Hypercore 4"]
        end
        
        STORAGE["Shared Storage<br/>RocksDB"]
    end
    
    CS --> CORE1
    CS --> CORE2
    CS --> CORE3
    CS --> CORE4
    CORE1 --> STORAGE
    CORE2 --> STORAGE
    CORE3 --> STORAGE
    CORE4 --> STORAGE

5.3 Implementation

const Corestore = require('corestore')
const Hyperswarm = require('hyperswarm')

// Create store
const store = new Corestore('./my-store')

// Get cores by name
const core1 = store.get({ name: 'messages' })
const core2 = store.get({ name: 'profiles' })
const core3 = store.get({ name: 'files' })

await Promise.all([core1.ready(), core2.ready(), core3.ready()])

// Use cores
await core1.append(Buffer.from('Message 1'))
await core2.append(Buffer.from('Profile data'))

// Replicate all cores
const swarm = new Hyperswarm()

swarm.on('connection', (conn) => {
  store.replicate(conn)
})

// Join all discovery keys
swarm.join(core1.discoveryKey)
swarm.join(core2.discoveryKey)
swarm.join(core3.discoveryKey)

---

6. Protomux + Protocol Layer

6.1 Pattern Overview

Multiplex multiple protocols over a single encrypted connection.

Use Cases:

• Multi-protocol apps
• RPC services
• Channel separation
• Protocol versioning

6.2 Architecture

flowchart LR
    CONN["Encrypted<br/>Connection"]
    MUX["Protomux"]
    
    subgraph "Channels"
        CH1["Channel A<br/>Hypercore"]
        CH2["Channel B<br/>RPC"]
        CH3["Channel C<br/>Custom"]
    end
    
    CONN --> MUX
    MUX --> CH1
    MUX --> CH2
    MUX --> CH3

6.3 Implementation

const Protomux = require('protomux')
const Hypercore = require('hypercore')
const RPC = require('protomux-rpc')

// Create mux on connection
const mux = new Protomux(socket)

// Add Hypercore channel
const core = new Hypercore('./my-feed')
const coreChannel = mux.createChannel({
  protocol: 'hypercore'
})

// Add RPC channel
const rpc = new RPC(mux, {
  protocol: 'my-rpc'
})

rpc.respond('ping', (req) => {
  return 'pong'
})

// Use channels independently
await coreChannel.send({ type: 'sync', bitfield: [...] })
const response = await rpc.request('ping')

---

7. Full Stack Composition

7.1 Complete Application Stack

flowchart TB
    subgraph "Application"
        UI["UI Layer"]
        STATE["State Management"]
        API["App Logic"]
    end
    
    subgraph "Data Layer"
        AUTO["Autobase<br/>Multi-writer"]
        DRIVE["Hyperdrive<br/>Files"]
        BEE["Hyperbee<br/>Index"]
        CORE["Hypercore<br/>Logs"]
    end
    
    subgraph "Network Layer"
        SWARM["Hyperswarm"]
        DHT["HyperDHT"]
        SEC["Secret Streams"]
        PMUX["Protomux"]
    end
    
    subgraph "Storage"
        STORE["Corestore"]
        ROCKS["RocksDB"]
    end
    
    UI --> STATE
    STATE --> API
    API --> AUTO
    API --> DRIVE
    API --> BEE
    AUTO --> CORE
    DRIVE --> CORE
    BEE --> CORE
    CORE --> STORE
    STORE --> ROCKS
    API --> SWARM
    SWARM --> DHT
    SWARM --> SEC
    SEC --> PMUX
    PMUX --> CORE

7.2 Implementation Example

const Corestore = require('corestore')
const Hyperdrive = require('hyperdrive')
const Hyperbee = require('hyperbee')
const Autobase = require('autobase')
const Hyperswarm = require('hyperswarm')
const Protomux = require('protomux')

class P2PApp {
  constructor(storagePath) {
    this.store = new Corestore(storagePath)
    this.swarm = new Hyperswarm()
  }
  
  async init() {
    // Create data structures
    this.drive = new Hyperdrive(this.store)
    this.index = new Hyperbee(this.store.get({ name: 'index' }))
    
    // Create autobase for multi-writer
    this.base = new Autobase({
      inputs: [this.store.get({ name: 'local' })],
      localInput: this.store.get({ name: 'local' })
    })
    
    await Promise.all([
      this.drive.ready(),
      this.index.ready(),
      this.base.ready()
    ])
    
    // Setup networking
    this.setupNetworking()
  }
  
  setupNetworking() {
    // Join topics
    this.swarm.join(this.drive.discoveryKey)
    this.swarm.join(this.base.discoveryKey)
    
    // Handle connections
    this.swarm.on('connection', (conn) => {
      // Setup protomux
      const mux = new Protomux(conn)
      
      // Replicate all data
      this.store.replicate(conn)
      
      // Handle custom protocols
      this.setupProtocols(mux)
    })
  }
  
  setupProtocols(mux) {
    // Add custom RPC
    // Add app-specific channels
  }
  
  async writeFile(path, data) {
    await this.drive.put(path, data)
    await this.index.put(`file:${path}`, { 
      updated: Date.now(),
      size: data.length 
    })
  }
  
  async appendLog(data) {
    await this.base.append(data)
  }
}

// Usage
const app = new P2PApp('./my-app')
await app.init()
await app.writeFile('/doc.txt', Buffer.from('Hello'))
await app.appendLog({ action: 'file-created', path: '/doc.txt' })

---

8. Common Patterns

8.1 Pattern Summary Table

Pattern	Components	Use Case	Complexity
Basic Feed	Hypercore + Swarm	Logs, events	Low
File Share	Drive + Swarm	File transfer	Low
Database	Bee + Swarm	Key-value store	Medium
Multi-writer	Autobase + Swarm	Collaboration	Medium
Multi-core	Corestore + Swarm	Complex apps	Medium
Full Stack	All components	Production apps	High

8.2 Trade-offs

Approach	Pros	Cons
Single Core	Simple, fast	Single writer
Autobase	Multi-writer	More complex
Hyperdrive	File semantics	Larger overhead
Hyperbee	Fast queries	Read-only views
Corestore	Efficient storage	Single namespace

---

9. Connection Flow Deep Dive

9.1 Complete Connection Establishment

sequenceDiagram
    participant A as Peer A
    participant DHT as HyperDHT
    participant B as Peer B
    
    Note over A,B: Discovery Phase
    
    A->>DHT: announce(topic, address)
    B->>DHT: lookup(topic)
    DHT-->>B: [Peer A address]
    
    Note over A,B: Connection Phase
    
    A->>B: UDP probe (hole punching)
    B->>A: UDP probe (hole punching)
    
    alt Direct connection succeeds
        A->>B: TCP/UDX connect
        B->>A: Accept
    else Hole punching fails
        A->>Relay: Connect via relay
        B->>Relay: Connect via relay
        Relay->>Relay: Bridge connection
    end
    
    Note over A,B: Encryption Phase
    
    A->>B: Noise_XX handshake
    B->>A: Ephemeral + static keys
    A->>B: Static key + auth
    
    Note over A,B: Protocol Phase
    
    A->>B: Protomux channel open
    B->>A: Channel accept
    
    A->>B: Hypercore replicate
    B->>A: Bitfield exchange
    
    loop Data Sync
        A->>B: Request blocks
        B->>A: Send blocks + proofs
    end

9.2 Protocol Negotiation

// Protomux allows multiple protocols
const mux = new Protomux(socket)

// Hypercore protocol
const hypercoreChannel = mux.createChannel({
  protocol: 'hypercore/1.0.0',
  id: core.discoveryKey
})

// Custom app protocol
const appChannel = mux.createChannel({
  protocol: 'my-app/2.0.0',
  id: appId
})

// Both run over same encrypted connection

---

10. Best Practices

10.1 Composition Guidelines

1. Start Simple: Begin with Hypercore + Swarm
2. Add Complexity: Introduce Autobase only when needed
3. Use Corestore: For multi-core apps
4. Separate Concerns: Different data types → different cores
5. Version Protocols: Use semantic versioning

10.2 Performance Tips

Tip	Implementation
Batch operations	Use corestore.batch()
Sparse sync	Request only needed blocks
Connection pooling	Reuse Hyperswarm connections
Lazy loading	Load data on demand
Indexing	Use Hyperbee for queries

10.3 Security Checklist

• Verify all signatures
• Use encrypted connections
• Validate input data
• Implement capability checks
• Audit dependencies

---

See Also

• Architecture - System architecture
• Core Concepts - Key concepts
• Building Tools - Development tools
• Existing Projects - Real-world examples