280 lines
12 KiB
Markdown
280 lines
12 KiB
Markdown
<!-- lead -->
|
||
Streamlining Node.js Tunnels: Isolation, port management, and resource efficiency for peak performance!
|
||
|
||
In this post, I dive deeply into recent optimizations made to a Node.js clustered application managing tunneled requests with isolated connections. I explore the issues with the initial setup, outline each enhancement in detail, and contrast the old and new methods. These changes aim to improve tunnel isolation, streamline resource management, and prevent critical errors that could disrupt the application’s operation.
|
||
|
||
# Source
|
||
## https://git.ssh.surf/hypermc/hyperMC-Web-Relay
|
||
|
||
# Git Commit
|
||
## https://s.nodejs.lol/ff51iaPLY
|
||
|
||
|
||
## Initial Setup and Issues
|
||
|
||
Our application originally served HTTP requests over a peer-to-peer network using **clustered Node.js workers**. Each incoming request established a **tunnel** to relay data through the `hyperdht` protocol using public keys derived from subdomain headers. The tunnels enabled communication to unique remote peers, allowing each HTTP request to reach its intended destination.
|
||
|
||
### Key Components in the Original Code
|
||
|
||
1. **Clustered Node.js Workers**: Using `cluster`, the application spawns multiple workers, leveraging all CPU cores for better concurrency and faster request handling.
|
||
2. **HyperDHT Tunnels**: For each request, the application creates a tunnel to relay data between the client and the destination using the `hyperdht` protocol.
|
||
3. **Port Management for Tunnels**: To assign ports for tunnel servers, I used randomly generated port numbers within a specified range.
|
||
|
||
### The Problem
|
||
|
||
As the application scaled, several issues began to emerge:
|
||
|
||
1. **Tunnel Confusion Across Requests**:
|
||
- Since tunnels weren’t strictly isolated to individual requests, responses sometimes bled into unrelated requests, causing data mix-ups.
|
||
- Persistent tunnels without proper cleanup led to stale or unintended connections, increasing the risk of incorrect data delivery.
|
||
|
||
2. **Port Conflicts**:
|
||
- The application frequently encountered `EADDRINUSE` errors, meaning some generated ports were already in use by other tunnels.
|
||
- Port conflicts led to worker crashes, causing downtime and reduced concurrency.
|
||
|
||
3. **Inefficient Resource Management**:
|
||
- Tunnels remained open even after requests completed, resulting in unnecessary resource consumption.
|
||
- Workers were busy managing unused connections instead of handling new requests, leading to performance bottlenecks.
|
||
|
||
Given these challenges, I set out to improve tunnel isolation, ensure reliable port availability, and enhance resource efficiency.
|
||
|
||
|
||
|
||
## New Approach: Enhanced Isolation, Dynamic Port Allocation, and Resource Management
|
||
|
||
To tackle these issues, I implemented several key improvements:
|
||
|
||
### 1. Strict Tunnel Isolation Per Request
|
||
|
||
Previously, tunnels were reused across requests, leading to data mix-ups and unintended connections. In the new approach:
|
||
|
||
- **Unique Tunnel Instances**: Each HTTP request now creates a dedicated `tunnelServer` instance, serving only that specific request. This ensures strict one-to-one mapping between the request and the tunnel, eliminating any chance of cross-request interference.
|
||
- **No Shared Tunnel State**: By eliminating shared tunnel tracking objects, each request operates with complete isolation, reducing complexity and risk of data leakage.
|
||
|
||
**Code Difference**:
|
||
|
||
**Old Method**:
|
||
```javascript
|
||
if (!tunnels[publicKey]) {
|
||
tunnels[publicKey] = port; // Assign port to a tunnel that may get reused
|
||
}
|
||
```
|
||
|
||
**New Method**:
|
||
```javascript
|
||
const tunnelServer = net.createServer((servsock) => {
|
||
// Dedicated tunnel for each request
|
||
});
|
||
```
|
||
|
||
With this change, each tunnel becomes ephemeral, existing solely to complete a single request before it’s removed, reducing unintended interactions between requests.
|
||
|
||
### 2. Robust Port Availability Check with `getAvailablePort`
|
||
|
||
In the initial implementation, the application generated random ports without checking their availability, leading to frequent `EADDRINUSE` errors. To address this:
|
||
|
||
- **Port Checking with `net.createServer`**: I enhanced `getAvailablePort` by creating a temporary server to verify port availability. If the port is free, the function closes the test server and assigns that port to the new tunnel. If the port is already in use, it retries until it finds a free port.
|
||
- **Automatic Retry Mechanism**: This approach ensures no `EADDRINUSE` errors by dynamically testing ports until an available one is found.
|
||
|
||
**Code Difference**:
|
||
|
||
**Old Method**:
|
||
```javascript
|
||
const port = 1337 + Math.floor(Math.random() * 1000); // No check for availability
|
||
```
|
||
|
||
**New Method**:
|
||
```javascript
|
||
async function getAvailablePort() {
|
||
return new Promise((resolve, reject) => {
|
||
const tryPort = () => {
|
||
const port = 1337 + Math.floor(Math.random() * 1000);
|
||
const tester = net.createServer()
|
||
.once('error', (err) => {
|
||
if (err.code === 'EADDRINUSE') {
|
||
tryPort(); // Retry if port is in use
|
||
} else {
|
||
reject(err);
|
||
}
|
||
})
|
||
.once('listening', () => {
|
||
tester.close(() => resolve(port)); // Port is available
|
||
})
|
||
.listen(port, '127.0.0.1');
|
||
};
|
||
tryPort();
|
||
});
|
||
}
|
||
```
|
||
|
||
This method guarantees that ports are only assigned if they are actually available, ensuring reliable tunnel creation and eliminating port-related crashes.
|
||
|
||
### 3. Automatic Tunnel Closure for Efficient Resource Management
|
||
|
||
Previously, tunnels remained open even after completing requests, wasting system resources and risking data leaks. Now, each tunnel is closed as soon as the associated response finishes.
|
||
|
||
- **Tunnel Lifecycle Bound to Request Lifecycle**: Using the `res.on('finish')` event, the tunnel server closes immediately after the response is sent, freeing resources for other requests.
|
||
- **Reduced Memory and CPU Overhead**: By closing tunnels promptly, workers are freed to handle new requests, reducing CPU and memory consumption.
|
||
|
||
**Code Difference**:
|
||
|
||
**Old Method**:
|
||
```javascript
|
||
// Tunnels were left open, requiring manual cleanup and causing resource issues
|
||
```
|
||
|
||
**New Method**:
|
||
```javascript
|
||
res.on('finish', () => {
|
||
tunnelServer.close(() => {
|
||
if (DEBUG === 1 || CONINFO === 1) console.log("Tunnel closed after request completion.");
|
||
});
|
||
});
|
||
```
|
||
|
||
With this approach, the system efficiently reclaims resources after each request, making the application more scalable and responsive under load.
|
||
|
||
|
||
|
||
## Detailed Code Walkthrough
|
||
|
||
Here’s the fully optimized code with all the changes:
|
||
|
||
```javascript
|
||
const cluster = require('cluster');
|
||
const numCPUs = require('os').cpus().length;
|
||
const net = require('net');
|
||
|
||
if (cluster.isMaster) {
|
||
console.log(`Master ${process.pid} is running`);
|
||
console.log(`Total Workers ${numCPUs * 6}`);
|
||
for (let i = 0; i < numCPUs * 4; i++) {
|
||
cluster.fork();
|
||
}
|
||
|
||
cluster.on('exit', (worker, code, signal) => {
|
||
console.log(`Worker ${worker.process.pid} died`);
|
||
});
|
||
} else {
|
||
const fs = require('fs');
|
||
const http = require('http');
|
||
const httpProxy = require('http-proxy');
|
||
const HyperDHTServer = require('hyperdht');
|
||
const b32 = require("hi-base32");
|
||
const agent = new http.Agent({ maxSockets: Number.MAX_VALUE });
|
||
const content = fs.readFileSync('404.txt', 'utf8');
|
||
const DEBUG = 0;
|
||
const CONINFO = 0;
|
||
const dhtServer = new HyperDHTServer();
|
||
|
||
const startServer = async () => {
|
||
console.log(`Worker ${process.pid} started`);
|
||
await dhtServer.ready();
|
||
|
||
const proxy = httpProxy.createProxyServer({
|
||
ws: true,
|
||
agent: agent,
|
||
timeout: 360000
|
||
});
|
||
|
||
const server = http.createServer(async function (req, res) {
|
||
try {
|
||
const split = req.headers.host.split('.');
|
||
const publicKey = Buffer.from(b32.decode.asBytes(split[0].toUpperCase()));
|
||
|
||
if (publicKey.length < 32) {
|
||
console.log("Invalid Connection!");
|
||
res.writeHead(418, { 'Content-Type': 'text/html' });
|
||
res.end(content);
|
||
return;
|
||
}
|
||
|
||
const port = await getAvailablePort();
|
||
const tunnelServer = net.createServer(function (servsock) {
|
||
const socket = dhtServer.connect(publicKey);
|
||
let open = { local: true, remote: true };
|
||
|
||
servsock.on('data', (d) => socket.write(d));
|
||
socket.on('data', (d) => servsock.write(d));
|
||
|
||
const remoteend = () => {
|
||
if (open.remote) socket.end();
|
||
open.remote = false;
|
||
};
|
||
const localend = () => {
|
||
if (open.local) servsock.end();
|
||
open.local = false;
|
||
};
|
||
|
||
servsock.on('error', remoteend);
|
||
servsock.on('finish', remoteend);
|
||
servsock.on('end', remoteend);
|
||
socket.on('finish', localend);
|
||
socket.on('error', localend);
|
||
socket.on('end', localend);
|
||
});
|
||
|
||
tunnelServer.listen(port, "127.0.0.1", () => {
|
||
if (DEBUG === 1 || CONINFO === 1) console.log(`Tunnel server listening on port ${port}`);
|
||
|
||
proxy.web(req, res, {
|
||
target: 'http://127.0.0.1:' + port
|
||
}, function (e) {
|
||
console.log("Proxy Web Error: ", e);
|
||
res.writeHead(404, { 'Content-Type': 'text/html' });
|
||
res.end(content);
|
||
});
|
||
|
||
res.on('finish', () => {
|
||
tunnelServer.close(() => {
|
||
if (DEBUG === 1 || CONINFO === 1) console.log("Tunnel closed after request completion.");
|
||
});
|
||
});
|
||
});
|
||
} catch (e) {
|
||
console.error("Error Occurred: ", e);
|
||
}
|
||
|
||
|
||
});
|
||
|
||
server.listen(8081, () => {
|
||
console.log(`Worker ${process.pid} listening on port 8081`);
|
||
});
|
||
};
|
||
|
||
startServer().catch(console.error);
|
||
|
||
async function getAvailablePort() {
|
||
return new Promise((resolve, reject) => {
|
||
const tryPort = () => {
|
||
const port = 1337 + Math.floor(Math.random() * 1000);
|
||
const tester = net.createServer()
|
||
.once('error', (err) => {
|
||
if (err.code === 'EADDRINUSE') {
|
||
tryPort();
|
||
} else {
|
||
reject(err);
|
||
}
|
||
})
|
||
.once('listening', () => {
|
||
tester.close(() => resolve(port));
|
||
})
|
||
.listen(port, '127.0.0.1');
|
||
};
|
||
tryPort();
|
||
});
|
||
}
|
||
}
|
||
```
|
||
|
||
|
||
|
||
## Final Thoughts
|
||
|
||
The new design introduces strict isolation for tunnels, efficient port management, and automatic resource cleanup. By implementing these changes, I:
|
||
- Solved the `EADDRINUSE` errors by dynamically checking port availability.
|
||
- Isolated tunnels to prevent cross-request data confusion.
|
||
- Enhanced performance and scalability by closing tunnels immediately after requests finish.
|
||
|
||
These updates not only improve reliability but also ensure that the application scales effectively, even under heavy load. This level of optimization is essential for any high-traffic Node.js service, as it directly impacts the user experience, system stability, and overall application performance. |