snxraven/ravenscott-blog
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Raven Scott 2024-10-28 16:47:03 -04:00
parent 02678b29a6
commit 9ee315d8b0
1 changed files with 99 additions and 54 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

151
markdown/Decentralized, Firewall-Resistant Peer-to-Peer DNS System.md
View File

@ -88,16 +88,28 @@ By establishing a framework where domain registries are optional rather than man
With this P2P DNS system, the ability to create and maintain digital identities is democratized. Users around the world can reclaim control over their namespaces, creating a more open, resilient, and censorship-resistant Internet. This redefines what it means to have access to the digital world, with a new level of autonomy and security, all powered by decentralized technology.
## Code Walkthrough
To create a comprehensive code walkthrough for the above setup, here’s an expanded breakdown:
This implementation consists of multiple components, from peer discovery to DNS handling and HTTP proxying. Let’s explore each section in detail.
## Code Walkthrough: Peer Discovery, DNS, and HTTP Proxying in a P2P Network
### Environment Variables and Dependencies
This implementation involves components for peer discovery, DNS resolution, virtual networking, and HTTP proxying, integrating `Hyperswarm` and `Corestore` to achieve a decentralized DNS system. Here's an in-depth analysis:
The code starts by loading environment variables and initializing necessary dependencies. The environment variable `masterNetworkDiscoveryKey` allows for creating isolated DNS networks, where only peers with the same key can discover each other.
### Loading Environment Variables and Dependencies
The code initializes environment variables with `dotenv`, allowing flexibility in configuring settings like `masterNetworkDiscoveryKey`. This key enables peers to create isolated networks, ensuring only peers with the matching key can discover each other.
Dependencies:
- **`exec`** for executing shell commands.
- **`dgram`** for UDP socket communication, essential for DNS operations.
- **`dns-packet`** to handle DNS packet creation and decoding.
- **`HolesailClient`** to manage P2P tunneling, essential for connecting domains over a decentralized network.
- **`Corestore`** for distributed storage, managing and syncing domain records among peers.
- **`Hyperswarm`** for peer discovery and creating P2P connections.
- **`http`** to handle HTTP requests.
- **`crypto`** to manage hashing for secure key generation and identification.
```javascript
require('dotenv').config(); // Load environment variables
require('dotenv').config();
const { exec } = require('child_process');
const dgram = require('dgram');
const dnsPacket = require('dns-packet');
@ -105,17 +117,16 @@ const HolesailClient = require('holesail-client');
const Corestore = require('corestore');
const Hyperswarm = require('hyperswarm');
const http = require('http');
const b4a = require('b4a');
const { createHash } = require('crypto');
const net = require('net');
```
### Corestore and Hyperswarm Setup for P2P Synchronization
### Corestore and Hyperswarm Configuration
The P2P DNS system uses **Corestore** for decentralized data storage and **Hyperswarm** for peer discovery and connectivity. These allow peers to sync domain records in a distributed network.
Here, `Corestore` and `Hyperswarm` are set up to manage decentralized DNS entries and establish peer-to-peer connections, respectively.
- **Corestore** is used to manage DNS data in a decentralized way. Each domain and its hash are stored in `dnsCore`, a key-value store that synchronizes records across peers.
- **Hyperswarm** creates a P2P network where peers with the same discovery key (generated from `masterNetworkDiscoveryKey` or `dnsCore.discoveryKey`) can connect and sync their data.
1. **Corestore** serves as a distributed key-value store that allows DNS records to be synchronized across peers.
2. **Hyperswarm** is initialized for P2P discovery, using a unique topic derived from `masterNetworkDiscoveryKey` or `dnsCore.discoveryKey`. Only peers with matching topics can join the same swarm, allowing for secure data replication.
```javascript
const store = new Corestore('./my-storage');
@ -124,18 +135,19 @@ const masterNetworkDiscoveryKey = process.env.masterNetworkDiscoveryKey
? Buffer.from(process.env.masterNetworkDiscoveryKey, 'hex')
: null;
const dnsCore = store.get({ name: 'dns-core' });
console.log(`Loaded masterNetworkDiscoveryKey: ${masterNetworkDiscoveryKey ? masterNetworkDiscoveryKey.toString('hex') : 'None'}`);
```
### Virtual Interface Creation and IP Mapping
### Virtual Network Interfaces for Domain Isolation
To handle P2P domains locally, the system creates virtual network interfaces (on macOS) and assigns IPs in the range `192.168.100.x`. Each domain is mapped to a unique IP, allowing it to be isolated within the local subnet.
Domains are mapped to unique local IP addresses within the `192.168.100.x` subnet. This setup isolates each domain by assigning a virtual network interface for each, thus preventing conflict across domains.
The function `createVirtualInterface` uses `ifconfig` to create a network interface alias for each domain.
- **`removeExistingInterface`**: Removes any existing virtual interface to avoid conflicts.
- **`createVirtualInterface`**: Uses `ifconfig` to create a local network alias, ensuring each domain has a dedicated IP.
- **`createInterfaceForDomain`**: Manages the IP assignment, incrementing the `currentIP` for each new domain.
```javascript
async function createVirtualInterface(subnetName, subnetCIDR) {
await removeExistingInterface(subnetName);
return new Promise((resolve, reject) => {
exec(`sudo ifconfig ${subnetName} alias ${subnetCIDR}`, (err, stdout, stderr) => {
if (err) {
@ -150,24 +162,13 @@ async function createVirtualInterface(subnetName, subnetCIDR) {
}
```
The `createInterfaceForDomain` function manages the automatic IP assignment by creating virtual interfaces for each domain and incrementing the IP range.
```javascript
async function createInterfaceForDomain(domain) {
const subnetID = currentIP;
const subnetName = `lo0`;
const subnetCIDR = `192.168.100.${subnetID}/24`;
await createVirtualInterface(subnetName, subnetCIDR);
domainToIPMap[domain] = `192.168.100.${subnetID}`;
currentIP++;
return domainToIPMap[domain];
}
```
### DNS Server and Query Handling
The DNS server uses `dgram` to listen on UDP port 53, processing incoming DNS requests. It checks whether the requested domain exists in the P2P DNS core or falls back on a public DNS lookup.
The DNS server listens on UDP port 53, decoding and responding to DNS requests. It checks if a domain exists in the P2P DNS core, using fallback mechanisms if needed.
- **`fetchP2PRecord`**: Retrieves the domain’s hash from the P2P DNS core.
- **`checkPublicDNS`**: Resolves domains outside the P2P network by querying Cloudflare’s 1.1.1.1 DNS server.
- **DNS Response Logic**: If the domain exists in P2P DNS, it responds with the locally mapped IP; otherwise, it uses the public DNS result.
```javascript
dnsServer.on('message', async (msg, rinfo) => {
@ -186,29 +187,77 @@ dnsServer.on('message', async (msg, rinfo) => {
});
```
The function `checkPublicDNS` queries Cloudflare's 1.1.1.1 DNS server to resolve domains not available in the P2P DNS.
### Holesail Client for Decentralized Tunneling and Connection Management
### Holesail Client for P2P Tunneling and Connection Management
In this implementation, the **Holesail client** is central to achieving decentralized, peer-to-peer (P2P) tunneling, which allows domains to be accessible across peers without relying on a traditional DNS infrastructure. Holesail is a critical component, acting as a secure bridge for domain-specific connections.
Holesail provides the essential P2P tunneling required to maintain decentralized DNS connections across peers. Each domain is associated with a unique `hash`, which Holesail uses to establish a tunnel between peers. Connections are automatically restarted if they become unresponsive.
Here's a breakdown of how Holesail functions within this system:
#### Purpose of Holesail in P2P Networking
The Holesail client facilitates direct communication between peers, bypassing the need for centralized servers by creating tunnels that connect domains across the network. Each domain entry has a unique hash, which Holesail uses to establish a tunnel. This unique hash acts as an identifier for each domain in the P2P network, ensuring traffic is routed accurately.
#### Key Functions and Features of Holesail
1. **Starting Tunnels for P2P Domains**:
Holesail uses the domain’s `hash` (generated or retrieved from the P2P DNS core) as an anchor point for the tunnel connection. By associating the domain hash with a unique local IP and port, the Holesail client can reroute incoming requests to the correct peer over the network.
2. **Automatic Connection Monitoring and Restarting**:
Connections in a P2P network can be less stable than in traditional networking. Holesail monitors each tunnel’s status and automatically restarts connections if they become unresponsive. This feature is implemented through a responsive check on each domain’s port. If Holesail detects an issue, it recreates the interface for the domain and starts a new tunnel connection, ensuring continuity of service.
3. **Connection Reusability**:
To optimize resources, Holesail reuses existing connections when possible. Each active connection is stored in the `activeConnections` object, which allows the client to check if a tunnel for a given domain is already active. If a tunnel is found, Holesail reuses it instead of initiating a new one, improving efficiency and reducing resource usage.
4. **Connection Lifecycle Management**:
Each Holesail connection has a lifecycle. To prevent stale or unresponsive connections from lingering, the client uses a timeout mechanism to automatically destroy and remove the tunnel from `activeConnections` after five minutes (300,000 ms). This cleanup process helps conserve resources and ensures only necessary connections remain active.
5. **Integration with Virtual Interface Management**:
When a new connection is needed for a domain, Holesail works alongside the `createInterfaceForDomain` function, which assigns a unique local IP to each domain. This allows domains to be isolated within the local network and ensures that each has a dedicated path through which Holesail can route traffic. By maintaining the virtual interface alongside the tunnel, Holesail manages traffic seamlessly across peers.
#### Code Example of Holesail Connection Management
Here's how Holesail is used to start and manage a P2P connection:
```javascript
async function restartHolesailClient(domain, hash, ip, port) {
if (!await checkPortResponsive(ip, port)) {
logDebug(`Port ${port} on ${ip} is unresponsive; restarting Holesail client`);
async function startHolesailClient(domain, hash, ip, port) {
logDebug(`Attempting to start/reuse Holesail client for domain: ${domain}`);
if (activeConnections[domain]) {
activeConnections[domain].destroy();
logDebug(`Reusing existing Holesail client for domain: ${domain} on ${ip}:${port}`);
return activeConnections[domain];
}
logDebug(`Starting new Holesail client for domain: ${domain}, hash: ${hash}, IP: ${ip}, Port: ${port}`);
const connector = setupConnector(hash);
const holesailClient = new HolesailClient(connector);
holesailClient.connect({ port: port, address: ip, reuseAddr: true }, () => {
logDebug(`Holesail client for ${domain} connected on ${ip}:${port}`);
});
activeConnections[domain] = holesailClient;
setTimeout(() => {
logDebug(`Destroying Holesail client for domain ${domain}`);
holesailClient.destroy();
delete activeConnections[domain];
}
await createInterfaceForDomain(domain);
startHolesailClient(domain, hash, ip, port);
}
}, 300000);
return holesailClient;
}
```
### HTTP Proxy for Accessing P2P Domains
In this function:
- **`setupConnector(hash)`** sets up the connector with the domain’s hash, allowing Holesail to identify and route traffic correctly.
- **`holesailClient.connect()`** initiates the connection to the specified IP and port, handling requests sent to the domain.
- **Timeout for Connection Lifecycle** ensures the tunnel is automatically destroyed if unused, freeing up resources.
The HTTP proxy listens on port 80 and routes traffic to domains in the P2P network. It uses the Holesail tunnel to establish connections and proxy HTTP requests to the correct IP.
Holesail is essential for bridging the gap between DNS resolution and accessible peer connections. By using Holesail, each domain can securely connect across peers within the P2P network, overcoming traditional DNS constraints and enabling a scalable, decentralized DNS solution.
### HTTP Proxy for Routing P2P Traffic
The HTTP proxy listens on port 80 and reroutes traffic to the appropriate IP for domains within the P2P network. This enables HTTP-based access for P2P-resolved domains.
```javascript
http.createServer(async (req, res) => {
@ -227,9 +276,9 @@ http.createServer(async (req, res) => {
}).listen(80, '127.0.0.1');
```
### Syncing TLDs and Hashes Across Peers
### Synchronizing DNS Records Across Peers
The `addDomain` function is used to append new domains and their associated hashes to the `dnsCore`. This ensures the domain records are accessible and synchronized across all peers in the P2P network.
The `addDomain` function enables peers to add new domains to the P2P DNS system, appending records to `dnsCore`, making them accessible and synchronized across the network.
```javascript
async function addDomain(domain, hash) {
@ -240,17 +289,15 @@ async function addDomain(domain, hash) {
}
```
### Hyperswarm and DNS Core Synchronization
### Hyperswarm Integration for Peer Synchronization
Upon startup, the system joins the Hyperswarm network with a unique topic derived from either the `masterNetworkDiscoveryKey` or `dnsCore.discoveryKey`. This allows only peers with the same discovery key to join the same network, effectively isolating private networks.
The DNS core joins the Hyperswarm network using the configured `masterNetworkDiscoveryKey`, ensuring only authorized peers connect and sync DNS data.
```javascript
(async () => {
await dnsCore.ready();
const topic = masterNetworkDiscoveryKey || dnsCore.discoveryKey;
logDebug(`DNS Core ready, joining Hyperswarm with topic:
${topic.toString('hex')}`);
logDebug(`DNS Core ready, joining Hyperswarm with topic: ${topic.toString('hex')}`);
swarm.join(topic, { server: true, client: true });
swarm.on('connection', (conn) => {
@ -260,8 +307,6 @@ Upon startup, the system joins the Hyperswarm network with a unique topic derive
})();
```
This configuration ensures that only peers with matching keys or topics can access and replicate DNS data, allowing for both public and private P2P DNS networks.
## Security and Privacy Implications
This P2P DNS system’s architecture offers significant privacy advantages. By decentralizing DNS queries and encrypting traffic over Holesail tunnels, it: