872 lines
51 KiB
Markdown
872 lines
51 KiB
Markdown
<!-- lead -->
|
|
The Problem That Wouldn't Leave Me Alone
|
|
|
|
For years, I'd come back to the same frustrating question: why can't we just run peer-to-peer networking in a browser? Not WebRTC with its signaling server requirements, not WebSockets that are really just TCP wrapped in HTTP, but real honest-to-goodness P2P where two browsers connect directly to each other without anyone in the middle.
|
|
|
|
The browser sandbox exists for good reasons. If any website could open raw sockets, bind to ports, participate in UDP protocols, the security implications would be enormous. Malicious sites could run scanning tools, bypass firewalls, create botnets. The sandbox protects users from themselves and from attackers. This isn't a bug in browser design, it's a feature that's kept the web usable for decades.
|
|
|
|
But the restriction creates an enormous gap. Want to build a chat application where messages go directly between users? You can't. Want to create a collaborative editing tool without a central server? You can't. Want to make a file sharing app that doesn't require uploading to some cloud service first? You can't. Every real-time web application that's ever been built follows the same client-server pattern: all data flows through a server that you have to deploy, maintain, scale, and pay for.
|
|
|
|
I'd watched Hyperswarm emerge and mature. It was solving the hard P2P problems: distributed hash tables for discovery, UDP hole-punching for NAT traversal, the Noise protocol for encryption. People were building incredible decentralized applications with it. But they were all running in Node.js or Bun or some other server-side runtime. The browser remained locked out.
|
|
|
|
<center>This bothered me more than it probably should have.<BR><h1>Repo: <a href="https://git.ssh.surf/snxraven/BridgeSwarm">https://git.ssh.surf/snxraven/BridgeSwarm</a></h1><BR></center>
|
|
|
|
|
|
**BridgeSwarm** is a revolutionary system that brings **peer-to-peer (P2P) networking** to regular web browsers without requiring any browser modifications.
|
|
|
|
It acts as a bridge between the browser's sandboxed environment and the powerful P2P capabilities of the **Hyperswarm** networking stack.
|
|
|
|
In simpler terms: BridgeSwarm allows your web application to connect directly to other instances of your web application running in different browsers, on different computers, anywhere in the world - without needing a central server to relay messages.
|
|
|
|
|
|
<center><img src="https://ssh.surf/images/bs.webp"></center>
|
|
|
|
<BR>
|
|
|
|
### The Problem with Traditional Web Apps
|
|
|
|
Traditional web applications follow a **client-server model**:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
[User's Browser] <------------------> [Central Server] <------------------> [Other Users]
|
|
HTTP/WebSocket HTTP/WebSocket
|
|
</pre>
|
|
</div>
|
|
|
|
* When User A wants to send a message to User B, the message goes to the server
|
|
* The server stores the message (optional) and forwards it to User B
|
|
* This creates a **single point of failure** - if the server goes down, no one can communicate
|
|
* This also creates **latency** - messages have to travel potentially thousands of miles twice
|
|
* And it creates **bandwidth costs** - the server has to handle ALL traffic
|
|
|
|
### The P2P Revolution
|
|
|
|
P2P networking flips this model:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
[User's Browser] <------------------> [Other User's Browser]
|
|
Direct Connection
|
|
(P2P/Hyperswarm)
|
|
</pre>
|
|
</div>
|
|
|
|
* When User A wants to communicate with User B, they connect **directly**
|
|
* No central server is required for communication
|
|
* The server can still exist for initial discovery or as a fallback
|
|
* Lower latency because data doesn't have to make extra hops
|
|
* More resilient - no single point of failure for communication
|
|
|
|
### The Challenge: Browsers Can't Do P2P
|
|
|
|
Here's the problem: **browsers are sandboxed**.
|
|
|
|
They can't create raw network connections, can't access UDP ports, and can't participate in P2P protocols directly.
|
|
|
|
This is a security feature - it protects users from malicious websites.
|
|
|
|
**BridgeSwarm solves this** by providing:
|
|
* A **browser extension** that runs in the browser
|
|
* A **native host** (a small program) that runs outside the browser
|
|
* A **bridge** between them using Chrome's native messaging protocol
|
|
|
|
The native host does the actual P2P networking (using Hyperswarm), and the browser extension provides an API that web pages can use.
|
|
|
|
## Understanding P2P Networking
|
|
|
|
### What is Peer-to-Peer?
|
|
|
|
In a P2P network, every participant (called a "peer") can be both a client and a server.
|
|
|
|
Peers communicate directly with each other, sharing resources like bandwidth, storage, and computation.
|
|
|
|
### Key P2P Concepts
|
|
|
|
#### Discovery
|
|
In a P2P network, you need to find other peers.
|
|
|
|
This is called "discovery." Hyperswarm uses **topics** for discovery:
|
|
- A **topic** is like a chat room name
|
|
- Anyone joining the same topic can discover each other
|
|
- Topics are 32 bytes (often represented as 64 hex characters)
|
|
|
|
#### Hole Punching
|
|
The internet isn't directly accessible - most computers are behind NAT (Network Address Translation) devices and firewalls.
|
|
|
|
P2P networks use techniques like **UDP hole punching** to establish direct connections even behind NAT.
|
|
|
|
Hyperswarm handles all of this automatically.
|
|
|
|
#### Encryption
|
|
P2P connections are encrypted using **Noise Protocol Framework**. This ensures that even if someone intercepts the connection, they can't read the data.
|
|
|
|
#### Public Key Identity
|
|
Every peer has a **cryptographic key pair**:
|
|
- A **public key** (can be shared freely, identifies you)
|
|
- A **private key** (kept secret, used for encryption/authentication)
|
|
|
|
Your public key is your identity in the P2P network.
|
|
|
|
## OSI Model vs P2P: Understanding the Differences
|
|
|
|
To truly understand P2P networking, it helps to compare it to the traditional networking model that powers the internet: the **OSI Model**.
|
|
|
|
### The OSI Model: Traditional Networking
|
|
|
|
The **OSI (Open Systems Interconnection) Model** is the conceptual framework that describes how data moves through a network. It has **7 layers**:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Layer 7: Application │ HTTP, WebSocket, DNS │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ Layer 6: Presentation │ TLS/SSL, JPEG, GIF, UTF-8 │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ Layer 5: Session │ Session management, API │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ Layer 4: Transport │ TCP, UDP │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ Layer 3: Network │ IP, Routing │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ Layer 2: Data Link │ Ethernet, WiFi, MAC addresses │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ Layer 1: Physical │ Cables, fiber, radio waves │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
</pre>
|
|
</div>
|
|
|
|
In traditional client-server networking:
|
|
|
|
1. **Your browser** (Layer 7: Application) makes an HTTP request
|
|
2. **TLS** (Layer 6: Presentation) encrypts the data
|
|
3. **TCP** (Layer 4: Transport) ensures reliable delivery
|
|
4. **IP** (Layer 3: Network) routes the data across the internet
|
|
5. The **server** receives it and responds in reverse
|
|
|
|
### P2P Networking: A Different Paradigm
|
|
|
|
P2P networking doesn't replace the OSI model - it's built **on top of it**.
|
|
|
|
The difference is **who** initiates connections and **how** data flows:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<table style="border-collapse:collapse;font-family:'Courier New',monospace;font-size:15px;margin:0;">
|
|
<tr>
|
|
<th style="border:1px solid #ccc;padding:10px 15px;text-align:left;vertical-align:top;">Aspect</th>
|
|
<th style="border:1px solid #ccc;padding:10px 15px;text-align:left;vertical-align:top;">Client-Server (OSI-based)</th>
|
|
<th style="border:1px solid #ccc;padding:10px 15px;text-align:left;vertical-align:top;">Peer-to-Peer</th>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Connection Initiation</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Client connects to central server</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Peers connect directly to each other</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Roles</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Fixed: client vs server</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Dynamic: every peer can be both</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Data Flow</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Always through server</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Direct peer-to-peer</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Single Point of Failure</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">The server</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">None (for communication)</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Scalability</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Server must handle all traffic</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Bandwidth shared across peers</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Discovery</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Server knows all clients</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">DHT/peer exchange</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Identity</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Username/password, sessions</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Cryptographic key pairs</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Trust Model</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Trust the server</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Trust the network/peer keys</td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
|
|
### How P2P Uses the OSI Layers
|
|
|
|
P2P applications still use all 7 OSI layers - they just use them differently:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
Traditional Web (Client-Server):
|
|
┌─────────┐ ┌─────────┐ ┌─────────┐
|
|
│ Browser │ ──────► │ Server │ ◄───────│ Browser │
|
|
└─────────┘ HTTP └─────────┘ HTTP └─────────┘
|
|
(Central)
|
|
|
|
P2P Networking:
|
|
┌─────────┐ ┌─────────┐
|
|
│ Browser │ ◄───────│ Browser │
|
|
└─────────┘ Noise └─────────┘ (Direct)
|
|
│ │
|
|
└──────────────┘
|
|
DHT
|
|
(for discovery only)
|
|
</pre>
|
|
</div>
|
|
|
|
### The Role of Each Layer in P2P
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<table style="border-collapse:collapse;font-family:'Courier New',monospace;font-size:15px;margin:0;">
|
|
<tr>
|
|
<th style="border:1px solid #ccc;padding:10px 15px;text-align:left;vertical-align:top;">Layer</th>
|
|
<th style="border:1px solid #ccc;padding:10px 15px;text-align:left;vertical-align:top;">P2P Usage</th>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 7 (Application)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Your app protocol (chat, file sharing)</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 6 (Presentation)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Protocol Buffers, MessagePack, custom encodings</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 5 (Session)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">P2P connections are long-lived sessions</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 4 (Transport)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">TCP/UDP for actual data transfer</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 3 (Network)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">IP routing, NAT traversal</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 2 (Data Link)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Ethernet/WiFi physical transport</td>
|
|
</tr>
|
|
<tr>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;font-weight:bold;">Layer 1 (Physical)</td>
|
|
<td style="border:1px solid #ccc;padding:10px 15px;vertical-align:top;">Cables, wireless signals</td>
|
|
</tr>
|
|
</table>
|
|
</div>
|
|
|
|
### Key Networking Differences
|
|
|
|
#### Discovery vs Connection
|
|
|
|
In traditional networking, a **DNS server** tells your browser the IP address of a server. You connect directly to that IP.
|
|
|
|
In P2P, **discovery** and **connection** are separate:
|
|
- **Discovery**: DHT (Distributed Hash Table) tells you which peers are interested in the same topic
|
|
- **Connection**: You establish direct encrypted connections to those peers
|
|
|
|
```javascript
|
|
// P2P Discovery (via DHT)
|
|
await swarm.join('my-topic'); // Register interest in topic
|
|
// DHT tells you: "These peers are also interested in 'my-topic'"
|
|
|
|
// P2P Connection (direct)
|
|
swarm.on('connection', (conn, peerInfo) => {
|
|
// Direct encrypted connection to peer
|
|
});
|
|
```
|
|
|
|
#### No Port Forwarding Required
|
|
|
|
Traditional servers need **port forwarding** - you configure your router to forward incoming connections to your server.
|
|
|
|
P2P uses **NAT traversal** (hole punching):
|
|
- Your peer initiates an outgoing connection to a relay
|
|
- The relay helps both peers discover their public IP:port
|
|
- Both peers send packets to each other, "punching holes" in the NAT
|
|
- After hole punching, direct P2P communication is possible
|
|
|
|
#### Always-On vs On-Demand
|
|
|
|
**Client-Server**: Server runs 24/7, clients connect as needed
|
|
**P2P**: Peers can come and go; the network adapts
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
Traditional: [Client] ──────► [Server] ──────► [Client]
|
|
Always-on Always-on
|
|
|
|
P2P: [Peer A] ──────► [Peer B]
|
|
(may join/leave anytime)
|
|
│
|
|
▼
|
|
[Peer C] ──────► (discovers via DHT)
|
|
</pre>
|
|
</div>
|
|
|
|
#### Trust Model
|
|
|
|
**Client-Server**: You trust the server to:
|
|
- Not read your data (depends on server honesty)
|
|
- Not go offline
|
|
- Not sell your data
|
|
|
|
**P2P**: You trust:
|
|
- **Encryption**: Noise protocol ensures even if intercepted, data is unreadable
|
|
- **Peer identities**: Public keys identify peers cryptographically
|
|
- **No central authority**: No single point of trust/control
|
|
|
|
### Why This Matters for BridgeSwarm
|
|
|
|
BridgeSwarm brings P2P capabilities to browsers, but browsers are inherently client-side.
|
|
|
|
Here's how it works:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
┌──────────────────────────────────────────────────────────────────┐
|
|
│ Your Web Page │
|
|
│ │
|
|
│ JavaScript calls: swarm.join('topic'), conn.write(data) │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌─────────────────────────────────────────────────────────┐ │
|
|
│ │ Browser Extension (api.js) │ │
|
|
│ │ Converts JavaScript calls to native messaging │ │
|
|
│ └─────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌─────────────────────────────────────────────────────────┐ │
|
|
│ │ Native Host (Hyperswarm) │ │
|
|
│ │ Actually does P2P: DHT discovery, NAT traversal │ │
|
|
│ └─────────────────────────────────────────────────────────┘ │
|
|
└──────────────────────────────────────────────────────────────────┘
|
|
</pre>
|
|
</div>
|
|
|
|
The key insight: **your web page doesn't do P2P directly**. The native host does.
|
|
|
|
The browser extension just provides the API.
|
|
|
|
This is important because:
|
|
* The native host can handle the complex P2P networking
|
|
* The browser stays sandboxed and secure
|
|
* You get P2P superpowers with familiar JavaScript APIs
|
|
|
|
## Hyperswarm: The Networking Engine
|
|
|
|
**Hyperswarm** is the underlying P2P networking library that BridgeSwarm wraps.
|
|
|
|
It's a powerful networking stack that handles:
|
|
|
|
### What Hyperswarm Does
|
|
|
|
* **Topic-based Discovery**
|
|
- Join a topic (32-byte identifier)
|
|
- Get notified when other peers join the same topic
|
|
- Automatically attempt to connect to discovered peers
|
|
|
|
* **NAT Traversal**
|
|
- UDP hole punching to bypass NAT
|
|
- Fall back to relay servers if direct connection fails
|
|
- Works behind most home routers and firewalls
|
|
|
|
* **Connection Management**
|
|
- Establish encrypted connections
|
|
- Handle multiple simultaneous connections
|
|
- Automatic reconnection
|
|
|
|
* **Peer Information**
|
|
- Each peer has a public key (their identity)
|
|
- Can track which topics a peer is interested in
|
|
|
|
BridgeSwarm exposes this functionality to browsers through its extension.
|
|
|
|
## The Architecture: How Everything Fits Together
|
|
|
|
BridgeSwarm consists of three main parts working together:
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Your Web Page │
|
|
│ │
|
|
│ window.BridgeSwarm │
|
|
│ ├── new BridgeSwarm({ appName: 'chat' }) │
|
|
│ ├── swarm.join('topic') │
|
|
│ ├── swarm.on('connection', (conn, peerInfo) => {}) │
|
|
│ ├── conn.write(data) │
|
|
│ └── BridgeSwarm.request('beeGet', { key: 'foo' }) │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
│ window.postMessage
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Chrome Extension (Content Script) │
|
|
│ │
|
|
│ content.js │
|
|
│ ├── Receives messages from page via postMessage │
|
|
│ ├── Forwards to background via chrome.runtime.sendMessage │
|
|
│ ├── Receives events from background │
|
|
│ ├── Dispatches as CustomEvents to page │
|
|
│ └── Injects api.js, framed-stream.js, protomux-bundle.js │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
│ chrome.runtime.sendMessage
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Chrome Extension (Background Service Worker) │
|
|
│ │
|
|
│ background.js │
|
|
│ ├── Maintains native messaging port to host │
|
|
│ ├── Tracks pending requests (promises) │
|
|
│ ├── Routes events to correct tabs │
|
|
│ ├── Manages swarm/tab associations │
|
|
│ ├── Handles reconnection on disconnect │
|
|
│ └── Stores extension settings │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
│ Native Messaging (stdin/stdout)
|
|
│ 4-byte length prefix + JSON
|
|
▼
|
|
┌──────────────────────────────────────────────────────────────────────────────┐
|
|
│ Native Host (Node.js/Bare) │
|
|
│ │
|
|
│ host.js │
|
|
│ ├── Runs Hyperswarm for P2P networking │
|
|
│ ├── Manages swarms (join/leave topics) │
|
|
│ ├── Handles connections (data, end, error events) │
|
|
│ ├── Runs Corestore/Hypercore/Hyperbee/Hyperdrive/Autobase/Hyperdb │
|
|
│ ├── Can attach Protomux and HRPC to connections │
|
|
│ └── Sends events and responses back to extension │
|
|
│ │
|
|
└──────────────────────────────────────────────────────────────────────────────┘
|
|
</pre>
|
|
</div>
|
|
|
|
## Finding the Way In
|
|
|
|
Chrome extensions have this feature called native messaging. It's been around, but the official documentation focuses on mundane use cases: integrating with password managers, connecting to desktop notification systems, that sort of thing. The technical capability underneath is much more powerful.
|
|
|
|
Native messaging lets an extension spawn a process outside the browser and communicate with it through standard input and standard output. The browser handles spawning the process, manages its lifecycle, and provides a clean message-passing interface. Nothing fancy, but effective.
|
|
|
|
The crucial realization hit me like a freight train: the native process can run whatever code it wants. JavaScript in the browser can't do P2P networking, but JavaScript running in a native process outside the browser can do absolutely anything. The extension becomes a bridge between the privileged world inside the browser and the powerful world outside.
|
|
|
|
<div style="display:flex;justify-content:center;align-items:center;padding:15px 20px;margin:0;">
|
|
<pre style="margin:0;padding:12px;font-family:'Courier New',monospace;font-size:15px;line-height:1.15;white-space:pre;overflow-x:auto;">
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Chrome Browser │
|
|
│ │
|
|
│ ┌─────────────┐ ┌─────────────────────────────────┐ │
|
|
│ │ Web Page │ │ Chrome Extension │ │
|
|
│ │ │ │ │ │
|
|
│ │window.Bridge│◄──────► │ content.js ──► background.js │ │
|
|
│ │ Swarm │ │ (service worker) │ │
|
|
│ └─────────────┘ └──────────────┬──────────────────┘ │
|
|
│ │ │
|
|
└──────────────────────────────────────────│──────────────────────┘
|
|
│ native messaging
|
|
│ (stdin/stdout)
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Native Host (Bare/Node.js) │
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
|
|
│ │ Hyperswarm │ │ Hyperbee │ │ Hyperdrive │ │
|
|
│ │ (P2P DHT) │ │ (key/value) │ │ (file system) │ │
|
|
│ └──────────────┘ └──────────────┘ └────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
|
|
│ │ Hypercore │ │ Autobase │ │ Hyperdb │ │
|
|
│ │ (append-log) │ │(multi-writer)│ │ (database) │ │
|
|
│ └──────────────┘ └──────────────┘ └────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
</pre>
|
|
</div>
|
|
|
|
The native host runs Hyperswarm, which handles all the peer discovery through the distributed hash table. It manages the NAT traversal magic that lets connections work behind home routers and corporate firewalls. It performs the Noise protocol handshake to establish encrypted sessions. It runs all the data storage systems that Hyperswarm supports. Nothing P2P happens without the native host being involved.
|
|
|
|
The extension acts as the intermediary. The service worker maintains the persistent connection to the native host, handles routing messages between different tabs, tracks which swarm belongs to which tab, manages request-response pairs so asynchronous operations return to their correct callers, and deals with disconnections and reconnections. The content script that gets injected into web pages serves as the bridge between the page's JavaScript context and the extension's privileged context.
|
|
|
|
The injected API that developers use is the final piece. It creates the BridgeSwarm class, handles all the JavaScript-side event emission, and manages communication with the extension through postMessage. The API is intentionally clean and simple, something that feels familiar to anyone who's used a networking library before.
|
|
|
|
## How Messages Actually Flow Through the System
|
|
|
|
The communication between extension and host uses Chrome's native messaging protocol, which itself is beautifully simple. Each message gets serialized as JSON, prefixed with a four-byte little-endian integer indicating the message length, then written to standard output. The receiving side reads the first four bytes to figure out how many more bytes to read, parses the JSON, and processes it.
|
|
|
|
```javascript
|
|
// What the wire format looks like:
|
|
// [4 bytes: length][N bytes: JSON]
|
|
|
|
// Sending a message from host to extension:
|
|
const message = JSON.stringify({ type: 'event', event: 'connection', payload: {...} });
|
|
const length = Buffer.alloc(4);
|
|
length.writeUInt32LE(message.length);
|
|
process.stdout.write(Buffer.concat([length, Buffer.from(message)]));
|
|
```
|
|
|
|
This is remarkably similar to how many other protocols work under the hood, but it gets the job done reliably and is easy to debug when things go wrong. Binary data gets base64-encoded in the JSON payload, which adds some overhead but keeps the implementation simple.
|
|
|
|
Here's what actually happens when your web page calls `swarm.join("my-topic")`:
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Page as Web Page (api.js)
|
|
participant Content as Content Script
|
|
participant Background as Service Worker
|
|
participant Host as Native Host
|
|
|
|
Page->>Content: postMessage({ type: 'bridge-swarm-bridge', payload })
|
|
Content->>Background: chrome.runtime.sendMessage()
|
|
Background->>Host: port.postMessage({ id: 'req_123', type: 'join', payload })
|
|
|
|
Note over Host: Hyperswarm.join(topic)<br/>Announces to DHT<br/>Begins peer discovery
|
|
|
|
Host->>Background: messenger.send({ id: 'req_123', type: 'response', payload: { ok: true } })
|
|
Background->>Content: sendResponse(payload)
|
|
Content->>Page: dispatchEvent('bridge-swarm-bridge-response')
|
|
Page->>Page: resolve Promise, user gets control back
|
|
```
|
|
|
|
The request ID tracking is crucial. When the web page makes a request, it doesn't know how long it will take. The background service worker assigns a unique ID, stores the resolve and reject functions in a Map, sends the message to the host, and then returns control to the page immediately. When the response comes back with that same ID, the background looks up the stored functions and resolves or rejects the promise. This makes the API feel synchronous to the developer even though there's a tremendous amount of asynchronous machinery happening underneath.
|
|
|
|
Now let's trace what happens when a peer connects:
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Host as Native Host
|
|
participant Background as Service Worker
|
|
participant Content as Content Script
|
|
participant Page as Web Page
|
|
|
|
Note over Host: Hyperswarm discovers peer<br/>Noise handshake<br/>Connection established
|
|
|
|
Host->>Background: messenger.send({ type: 'event', event: 'connection', payload: { connId, swarmId, peerInfo } })
|
|
|
|
Note over Background: Look up which tab owns swarmId<br/>Send only to that tab
|
|
|
|
Background->>Content: tabs.sendMessage(tabId, { type: 'bridge-swarm-event', payload })
|
|
Content->>Page: window.dispatchEvent('bridge-swarm-event')
|
|
|
|
Note over Page: api.js receives event<br/>Creates BridgeSwarmConnection<br/>Emits 'connection' event
|
|
|
|
Page->>Page: user callback fires with (conn, peerInfo)
|
|
```
|
|
|
|
The routing logic here is critical. Multiple tabs might each have their own BridgeSwarm instances with different swarm IDs. When the native host emits a connection event, the background needs to figure out which tab should receive it. Every time a BridgeSwarm instance initializes, it registers its swarm ID with the background. The background maintains a mapping of swarm IDs to tab IDs, so when events arrive, it can look up the correct destination.
|
|
|
|
## The Code Behind It All
|
|
|
|
Let's look at what this actually looks like in practice. Here's how the native host handles incoming requests:
|
|
|
|
```javascript
|
|
// native-host/host.js - simplified
|
|
|
|
const commands = {
|
|
init: async ({ swarmId, options }) => {
|
|
const swarm = new Hyperswarm(options);
|
|
swarm.listen(); // Accept incoming connections
|
|
swarms.set(swarmId, swarm);
|
|
|
|
// Track connections for this swarm
|
|
swarm.on('connection', (socket, peerInfo) => {
|
|
const connId = `conn_${Date.now()}_${Math.random().toString(36).slice(2,8)}`;
|
|
|
|
connections.set(connId, { socket, swarmId, peerInfo });
|
|
|
|
// Emit event back to extension
|
|
emit('connection', { connId, swarmId, peerInfo: serializePeerInfo(peerInfo) });
|
|
|
|
// Handle incoming data on this connection
|
|
socket.on('data', (data) => {
|
|
emit('data', { connId, swarmId, data: data.toString('base64') });
|
|
});
|
|
|
|
socket.on('end', () => {
|
|
connections.delete(connId);
|
|
emit('end', { connId, swarmId });
|
|
});
|
|
});
|
|
|
|
return { ok: true };
|
|
},
|
|
|
|
join: async ({ swarmId, topic }) => {
|
|
const swarm = swarms.get(swarmId);
|
|
const topicBuffer = Buffer.from(topic.padEnd(32, '\0')).slice(0, 32);
|
|
swarm.join(topicBuffer);
|
|
return { ok: true };
|
|
},
|
|
|
|
write: async ({ connId, data }) => {
|
|
const conn = connections.get(connId);
|
|
if (!conn) return { error: 'Connection not found' };
|
|
|
|
const buffer = Buffer.from(data, 'base64');
|
|
conn.socket.write(buffer);
|
|
return { ok: true };
|
|
},
|
|
|
|
destroy: async ({ swarmId }) => {
|
|
const swarm = swarms.get(swarmId);
|
|
if (swarm) {
|
|
swarm.destroy();
|
|
swarms.delete(swarmId);
|
|
}
|
|
return { ok: true };
|
|
}
|
|
};
|
|
```
|
|
|
|
The host maintains several key mappings. The `swarms` Map tracks all active Hyperswarm instances by their swarm ID. The `connections` Map tracks all active P2P connections, keyed by a unique connection ID. There's also a mapping from swarm IDs to connection IDs so the host knows which connections belong to which swarm.
|
|
|
|
Now here's what the web page API looks like from the developer's perspective:
|
|
|
|
```javascript
|
|
// This is all you need to write as a developer
|
|
|
|
// Wait for the API to be ready
|
|
await BridgeSwarm.ready();
|
|
|
|
// Create a swarm instance
|
|
const swarm = new BridgeSwarm({ appName: 'my-chat-app' });
|
|
|
|
// Join a topic - this is how peers find each other
|
|
await swarm.join('some-topic-name');
|
|
|
|
// Handle incoming peer connections
|
|
swarm.on('connection', (conn, peerInfo) => {
|
|
console.log('New peer connected:', peerInfo.publicKey.slice(0, 8) + '...');
|
|
|
|
// Handle incoming data
|
|
conn.on('data', (data) => {
|
|
const message = new TextDecoder().decode(data);
|
|
console.log('Received:', message);
|
|
});
|
|
|
|
// Send data to this peer
|
|
conn.write(new TextEncoder().encode('Hello, peer!'));
|
|
});
|
|
|
|
// Broadcast to all connected peers
|
|
for (const conn of swarm.connections()) {
|
|
conn.write(new TextEncoder().encode('Hello, everyone!'));
|
|
}
|
|
|
|
// Clean up when done
|
|
await swarm.leave('some-topic-name');
|
|
swarm.destroy();
|
|
```
|
|
|
|
That ten lines of code does an enormous amount of work under the hood. It initializes a Hyperswarm instance, generates a cryptographic key pair, joins a distributed hash table topic, discovers other peers interested in the same topic, establishes encrypted connections to each peer, handles NAT traversal, manages connection lifecycle, and emits events when things happen.
|
|
|
|
## The Data Storage Layer
|
|
|
|
Beyond just peer-to-peer networking, BridgeSwarm exposes the full Hyperswarm data stack to web pages. This means you can build applications that share not just messages but actual data structures.
|
|
|
|
```javascript
|
|
// Hyperbee - key/value store like a P2P Redis
|
|
await BridgeSwarm.request('beePut', { key: 'username', value: 'alice' });
|
|
const result = await BridgeSwarm.request('beeGet', { key: 'username' });
|
|
console.log(result.value); // 'alice'
|
|
|
|
// Hyperdrive - P2P file system
|
|
const fileContent = 'Hello, world!';
|
|
const base64 = btoa(fileContent);
|
|
await BridgeSwarm.request('drivePut', { path: '/readme.txt', base64 });
|
|
const file = await BridgeSwarm.request('driveGet', { path: '/readme.txt' });
|
|
|
|
// Hyperdb - schema-based database
|
|
await BridgeSwarm.request('hyperdbInsert', {
|
|
collection: 'users',
|
|
doc: { id: 'user1', name: 'Alice', email: '[email protected]' }
|
|
});
|
|
const user = await BridgeSwarm.request('hyperdbGet', {
|
|
collection: 'users',
|
|
query: { id: 'user1' }
|
|
});
|
|
```
|
|
|
|
These data operations flow through the same message-passing infrastructure as everything else. The native host manages the storage, handles replication between peers if desired, and returns results to the web page.
|
|
|
|
## Tab Swarms considering local and remote
|
|
|
|
One subtle issue was how each tab got its identity. Hyperswarm generates a cryptographic key pair when you create the swarm, and you can't change it later. Initially, all tabs using the same application would end up with identical public keys, making it impossible to tell them apart. The solution was generating a unique swarm ID for each BridgeSwarm instance, which creates a separate Hyperswarm with its own independent key pair. Each tab now has its own distinct identity while still being able to discover and communicate with other tabs on the same topic.
|
|
|
|
## Example Application: P2P Chat
|
|
|
|
Here's a complete working chat application that demonstrates the system in action. This is essentially what's in the chat-advanced example in the repository.
|
|
|
|
```javascript
|
|
// Complete P2P chat in about 60 lines
|
|
|
|
const state = {
|
|
swarm: null,
|
|
connections: new Map(),
|
|
nickname: 'Anonymous',
|
|
publicKey: ''
|
|
};
|
|
|
|
// Initialize when user clicks Join
|
|
async function joinChat(topic, nickname) {
|
|
state.nickname = nickname;
|
|
|
|
// Create the swarm
|
|
state.swarm = new window.BridgeSwarm({ appName: 'chat-app' });
|
|
|
|
// Listen for connections
|
|
state.swarm.on('connection', handleConnection);
|
|
|
|
// Join the topic to discover peers
|
|
await state.swarm.join(topic);
|
|
|
|
// Get our public key
|
|
state.publicKey = await state.swarm.getPublicKey();
|
|
|
|
// Broadcast our presence
|
|
broadcastPresence();
|
|
}
|
|
|
|
function handleConnection(conn, peerInfo) {
|
|
const connId = Date.now() + '-' + Math.random().toString(36).slice(2, 8);
|
|
|
|
// Track this connection
|
|
state.connections.set(connId, { conn, peerInfo, nickname: 'Unknown' });
|
|
|
|
// Send handshake
|
|
conn.write(JSON.stringify({
|
|
type: 'handshake',
|
|
nickname: state.nickname
|
|
}));
|
|
|
|
// Handle incoming data
|
|
conn.on('data', (data) => {
|
|
const msg = JSON.parse(new TextDecoder().decode(data));
|
|
|
|
switch (msg.type) {
|
|
case 'handshake':
|
|
// Update peer's nickname
|
|
const connObj = state.connections.get(connId);
|
|
if (connObj) connObj.nickname = msg.nickname;
|
|
broadcastPresence();
|
|
break;
|
|
|
|
case 'chat':
|
|
displayMessage(msg.author, msg.content, msg.timestamp);
|
|
break;
|
|
}
|
|
});
|
|
|
|
// Handle disconnect
|
|
conn.on('end', () => {
|
|
state.connections.delete(connId);
|
|
broadcastPresence();
|
|
});
|
|
}
|
|
|
|
function sendMessage(text) {
|
|
const msg = {
|
|
type: 'chat',
|
|
author: state.nickname,
|
|
content: text,
|
|
timestamp: Date.now()
|
|
};
|
|
|
|
// Send to all connected peers
|
|
for (const [, conn] of state.connections) {
|
|
conn.conn.write(JSON.stringify(msg));
|
|
}
|
|
|
|
// Display our own message
|
|
displayMessage(state.nickname, text, msg.timestamp);
|
|
}
|
|
|
|
function broadcastPresence() {
|
|
const peers = Array.from(state.connections.values()).map(c => c.nickname);
|
|
updateUserList([state.nickname, ...peers]);
|
|
}
|
|
```
|
|
|
|
This example shows the full flow. Users join a topic, connections establish automatically, messages send directly between peers, and presence information syncs across the network. There's no server required.
|
|
|
|
## Repository and Installation
|
|
|
|
The full implementation lives at [https://git.ssh.surf/snxraven/BridgeSwarm](https://git.ssh.surf/snxraven/BridgeSwarm).
|
|
|
|
For the quickest start, you can run the installer directly without cloning:
|
|
|
|
```bash
|
|
curl -fsSL https://ssh.surf/bridgeswarm/install.sh -o install.sh && bash install.sh
|
|
```
|
|
|
|
This downloads BridgeSwarm, builds the native host, packages the extension, and opens your browser to load it. The installer handles all the compilation and configuration automatically.
|
|
|
|
If you prefer manual installation, clone the repository and run the setup script:
|
|
|
|
```bash
|
|
git clone https://git.ssh.surf/snxraven/BridgeSwarm.git
|
|
cd BridgeSwarm
|
|
./scripts/install.sh
|
|
```
|
|
|
|
Then load the extension by opening chrome://extensions, enabling developer mode, clicking "Load unpacked," and selecting the extension/ folder.
|
|
|
|
## What's Possible Now
|
|
|
|
With this system running in a browser, the types of applications you can build fundamentally change. A real-time chat application requires no server deployment, no database setup, no scaling configuration. Users connect directly, messages flow between them, and there's no infrastructure for you to maintain.
|
|
|
|
Collaborative document editing becomes achievable without a central coordination server. Multiple users can connect to the same topic, exchange CRDT-based updates through Hypercore replication, and see each other's changes in real-time. The data lives in the users' browsers, synchronized directly between them.
|
|
|
|
File sharing applications work without uploading files to some cloud service first. One user sends a file directly to another over the encrypted P2P connection. No intermediate server, no storage costs, no privacy concerns about uploading files to third-party services.
|
|
|
|
Gaming applications benefit from the low latency that direct connections provide. For games where latency matters less like turn-based strategy or card games the peer-to-peer model works excellently. Even real-time games can leverage client-side prediction to compensate for network delays.
|
|
|
|
The applications are bounded only by what you can imagine. Any software that traditionally requires a central server to coordinate users can potentially be rebuilt as a direct peer-to-peer application.
|
|
|
|
## Any Page, Anywhere
|
|
|
|
One of the most powerful features of BridgeSwarm is its universal compatibility. It works on any web page whether served over HTTPS or opened from your local filesystem. The extension's content script injects into every page, turning virtually any website into a P2P application without requiring special infrastructure from the operator.
|
|
|
|
This enables scenarios that were previously impractical. For example, a blog comment section no longer needs a central server or database. Comments flow directly between readers, eliminating single points of failure.
|
|
|
|
Implementation is trivial. Simply include the BridgeSwarm API script like any other library. The extension supplies the API automatically. Comments broadcast peer to peer in real time among current viewers.
|
|
|
|
For persistence, keep comments in memory for ephemeral or live discussions, or use Hyperdb (a schema based P2P database) for durable storage. Hyperdb replicates across online peers so late arrivals still sync. Define schemas, insert documents, and query everything through the same simple API.
|
|
|
|
The hybrid approach combines both: real time P2P for active users plus Hyperdb persistence. The same model powers live collaboration features such as viewer lists and inline annotations, interactive widgets like stock tickers, polls and Q&A, all without any backend.
|
|
|
|
Chrome's standard content scripts handle injection (configurable for file:// URLs). Pages auto join topic based swarms, connecting viewers seamlessly.
|
|
|
|
Developers simply add a script tag. There is no need for WebSocket servers, signaling services, or third party costs. The system even improves offline resilience by letting users share data peer to peer when connectivity is poor.
|
|
|
|
Users stay in full control via the settings panel (enable/disable, skip local files, debug output). Best of all, it works perfectly with just two readers and has no infrastructure cost or scaling limitations.
|
|
|
|
|
|
## Configuration and Defaults
|
|
|
|
The extension includes a settings panel accessible by right-clicking the extension icon. These settings control how BridgeSwarm behaves by default when a page requests a swarm, allowing site operators and users to customize the behavior without requiring code changes.
|
|
|
|
The default application name gets used when a page creates a swarm without explicitly specifying one. The maximum peers setting controls how many simultaneous connections a swarm will maintain, with zero meaning unlimited. Request timeout determines how long the API waits for a response from the native host before rejecting a promise. Ready timeout controls how long BridgeSwarm.ready() waits for the API to become available.
|
|
|
|
There's also a setting that defaults to enabled which skips injection on file:// URLs. This is useful for developers who want to test their pages locally without triggering the P2P behavior. When this setting is disabled, the API gets injected into local files just like any other page, allowing full testing of P2P functionality without deploying to a server.
|
|
|
|
The notify on disconnect option shows a browser notification when the native host disconnects unexpectedly, which is helpful for debugging but can be annoying in production. Debug mode enables verbose logging to the console, which makes it much easier to understand why connections aren't forming or why messages aren't being delivered. These settings persist across browser sessions, so once you configure them the way you like, they stay that way.
|
|
|
|
## The Core Idea
|
|
|
|
The browser was never designed for peer-to-peer networking. The sandbox, the security model, the entire architecture assumes a client-server world. But through native messaging, we've found a way to bring P2P capability to the browser without compromising the security model.
|
|
|
|
The architecture isn't elegant. There's a native process, an extension, multiple layers of message passing, routing logic, fallback behavior for edge cases. It's complex in ways that pure server-based solutions aren't. But it works, and what it enables makes the complexity worthwhile.
|
|
|
|
For years, building real-time web applications meant accepting the server as a necessary intermediary. Now there's another option. The code is there, the examples are there, the documentation is there. If you've ever wanted to build applications where users communicate directly without a server in the middle, the tools are finally available.
|