p2ns/docs/ARCHITECTURE.md

P2NS Architecture

This document describes the internal architecture of P2NS, including module organization, data flow, and key implementation details.

Module Organization

P2NS is organized into several module categories under the includes/ directory:

includes/
├── admin/              # Admin interface and API
├── core/               # Core P2NS functionality
├── infrastructure/     # System utilities and patterns
├── maintenance/        # Cleanup, backup, and resource management
├── networking/         # DNS, proxying, and tunneling
├── plugins/            # Plugin system and SDK
└── security/           # Certificate management

Core Modules

core/core.js

Autopass write facade and auto-voting. Re-exports getConsensusState and getConsensusMetrics from the consensus sidecar read path. Handles:

• Domain claim creation (claim:domain:claimant) via dns-pass-queue dual-write
• Vote management (vote:domain:claimant:voter)
• Auto-voting logic
• Autopass entries cache for admin/auto-vote (getAllEntries)

core/consensus-*.js

Autobase consensus sidecar (RFC 0001):

• consensus-autobase.js — Sidecar lifecycle, bootstrap, hydration, background init, getConsensusStatus
• consensus-events.js — Event encode/decode, KV→event mapping
• consensus-apply.js — Apply handler; in-memory per-domain view
• consensus-view.js — Sole getConsensusState implementation
• consensus-resolver.js — Pure quorum/tie-break resolution logic

core/domains.js

Domain management including:

• Adding/removing domains
• Loading domains from domains.json
• File watching for hot reloading
• Internal domain detection

core/domain_cleanup.js

Comprehensive domain removal including:

• P2P network claim removal (full removal for resolved claimants, partial removal for competing claims)
• Holesail client cleanup
• Virtual interface removal
• DNS preference cleanup

Supports two removal modes:

• Full removal: When user is the resolved claimant, removes all claims, votes, and infrastructure
• Partial removal: When user has a competing claim, removes only their claim and votes they cast

Infrastructure Modules

infrastructure/logger.js + infrastructure/log-files.js

Leveled logging with component prefixes, routed to split files under logs/ via log-files.js:

• logDebug / logInfo / logWarn / logError
• core → console + core.log; Internal Proxy, HTTP Proxy, DNS, Plugins, Holesail → dedicated files (no console)

Admin log tail uses log-websocket.js (subscribe-log, file-log, log-snapshot). Plugin logs use separate plugin-log WebSocket messages and app.log per plugin.

infrastructure/state.js

Global state management including:

• Domain-to-IP mappings
• Holesail connections
• TLS/HTTP servers
• Swarm peer channels (peerChannels), plugin RPC state (pluginChannels), and metrics
• Version preferences
• DNS pass instance

infrastructure/config.js

Configuration validation on startup:

• Environment variable parsing
• Default value handling
• Type validation

infrastructure/validation.js

Input validation utilities:

• Domain name validation
• Holesail hash validation
• IP address validation
• Port validation

infrastructure/circuit_breaker.js

Circuit breaker pattern implementation for preventing cascading failures:

const { getCircuitBreaker } = require('./circuit_breaker');

const breaker = getCircuitBreaker('dns-service', {
  failureThreshold: 5,    // Failures before opening
  resetTimeout: 60000,    // Time before half-open
  monitoringWindow: 60000 // Window for counting failures
});

// States: CLOSED -> OPEN -> HALF_OPEN -> CLOSED
await breaker.execute(async () => {
  // Protected operation
}, 'dns-query');

infrastructure/rate_limit.js

In-memory rate limiting for API endpoints:

• Configurable requests per window
• Per-IP tracking
• Automatic cleanup
• Local IP exemption
• Endpoint exemption (GET requests, health checks)

const { checkRateLimit } = require('./rate_limit');

const result = checkRateLimit(req);
if (result) {
  // Rate limited - return 429 response
}

infrastructure/error_handler.js

User-friendly error handling:

• Error code translation (EADDRINUSE, ENOENT, etc.)
• Production-safe error messages
• Error response formatting

infrastructure/async_errors.js

Async error handling utilities:

const { wrapAsync, safePromise, retryWithBackoff } = require('./async_errors');

// Wrap async function with error logging
const safeFn = wrapAsync(asyncFn, 'context-name');

// Execute promise without throwing
const { success, result, error } = await safePromise(promise, 'context');

// Retry with exponential backoff
const result = await retryWithBackoff(asyncFn, {
  maxRetries: 3,
  initialDelay: 1000,
  maxDelay: 10000
});

infrastructure/utils.js

Common utility functions:

• Time conversion helpers
• String manipulation
• Object utilities

Networking Modules

networking/dns.js

UDP DNS server implementation:

• P2P domain resolution
• Local DNS record lookup
• Public DNS fallback
• DNS conflict handling

networking/dns_pool.js

DNS resolver connection pooling:

const { dnsPool } = require('./dns_pool');

// Query with automatic failover
const response = await dnsPool.query({
  type: 'query',
  questions: [{ name: 'example.com', type: 'A' }]
});

// Supports multiple DNS servers via PUBLIC_DNS_SERVER env var
// Example: PUBLIC_DNS_SERVER=1.1.1.1,8.8.8.8,9.9.9.9

Features:

• Connection pooling (default 5 connections)
• Round-robin query distribution
• Automatic failover between DNS servers
• Query timeout handling (5 seconds)
• Proper listener cleanup

networking/holesail.js

Holesail server/client management:

• Server creation and lifecycle
• Client creation with lazy initialization
• Connection persistence
• Configuration file management

networking/holesail_child.js

Child process for Holesail instances:

• IPC communication with parent
• Console log redirection
• Port availability checking
• Error handling

networking/internal_domains_proxy.js

HTTPS proxy for internal domains:

• Plugin request routing
• Static file serving
• WebSocket upgrade handling
• TLS termination

networking/p2p_domains_proxy.js

HTTPS proxy for P2P domains:

• Holesail client management
• Version preference handling (P2P vs public)
• SSL/TLS tunneling
• Connection timeout management

networking/virtual_interfaces.js

Virtual network interface management:

• IP alias creation on loopback
• Multi-subnet support
• Cross-platform (macOS, Linux, Windows)
• Interface cleanup

Maintenance Modules

maintenance/backup.js

Backup and restore system:

• Automatic scheduled backups
• Manual backup creation
• Backup rotation
• Restore functionality
• Metadata tracking

maintenance/cleanup.js

Resource cleanup on shutdown:

• Server closure
• Connection termination
• Interface removal
• Port freeing

maintenance/metrics.js

System metrics collection:

• Request statistics
• Response time tracking
• Error rate monitoring
• Historical data aggregation

maintenance/resource_tracker.js

Resource lifecycle management:

const { resourceTracker } = require('./resource_tracker');

// Register a resource
const id = resourceTracker.register('socket', socket, () => socket.close());

// Cleanup specific resource
await resourceTracker.cleanup(id);

// Cleanup all resources
await resourceTracker.cleanupAll();

// Get resource counts
const counts = resourceTracker.getCounts();
// { socket: 5, timer: 3, server: 2 }

maintenance/resource_validation.js

Periodic resource validation:

• Stale connection detection
• Orphaned server cleanup
• State map consistency checks
• Configurable validation interval

Plugin Modules

plugins/plugin-handler.js

Plugin lifecycle management:

• Plugin discovery from plugin-sites/
• Loading/unloading
• Request routing
• WebSocket registration

plugins/sdk.js

Plugin SDK providing access to:

• State (sdk.state)
• DNS operations (sdk.dns)
• Domain management (sdk.domains)
• Holesail operations (sdk.holesail)
• Peer management (sdk.peers)
• Certificate management (sdk.certificates)
• Local DNS (sdk.localDns)
• Interfaces (sdk.interfaces)
• Metrics (sdk.metrics)
• WebSocket (sdk.websocket)
• Configuration (sdk.config)
• Backup (sdk.backup)
• Subscriptions (sdk.subscriptions)
• HTTP client (sdk.http)
• File system (sdk.fs)
• Events (sdk.events)
• Utilities (sdk.utils)
• Security (sdk.security)
• Logging (sdk.log)
• Router (sdk.router)
• Admin panel (sdk.admin)
• Database (sdk.db)
• Channels (sdk.channels)
• Drives (sdk.drives)
• Authentication (sdk.auth)
• Profiles (sdk.profiles)

plugins/db-manager.js

HyperDB database management for plugins:

• Schema building
• Database initialization
• Replication management

plugins/drive-manager.js

Hyperdrive file system management:

• Drive creation and caching
• File operations
• Replication

plugins/channel-manager.js

Plugin protomux-rpc management:

• registerPluginProtocol — per-plugin RPC muxes and method handlers
• Request/event routing via channel-rpc.js and plugin-rpc-contract.js
• Peer RPC session tracking in state.pluginChannels

plugins/replication-manager.js

Database replication over Hyperswarm:

• Global topic replication
• Peer connection handling
• Core synchronization

plugins/hyperdb-builder.js

HyperDB schema builder:

• Schema generation from config.json
• Helper function loading
• Spec file generation

plugins/auth-utils.js

Authentication utilities for plugins:

• Ed25519 token generation
• Token verification
• Request authentication

Security Modules

security/certificate_authority.js

TLS certificate management:

• Root CA generation
• Domain certificate creation
• Certificate installation (macOS, Linux, Windows)
• Expiration monitoring

Data Flow

DNS Resolution Flow

Client Request
      │
      ▼
DNS Server (port 53)
      │
      ├─► Local DNS Records (cache/local_dns.json)
      │         │
      │         └─► Return if found
      │
      ├─► P2P Domain Check
      │         │
      │         ├─► Consensus Resolution
      │         │         │
      │         │         └─► Return internal IP
      │         │
      │         └─► Start Holesail Client (if needed)
      │
      └─► Public DNS Fallback (dns_pool.js)
                │
                └─► Return public IP

Proxy Request Flow

HTTPS Request (port 443)
      │
      ▼
SNI Extraction
      │
      ├─► Internal Domain
      │         │
      │         └─► Plugin Handler
      │                   │
      │                   ├─► API Endpoint
      │                   │
      │                   └─► Static Files
      │
      └─► P2P Domain
                │
                ├─► Version Preference Check
                │         │
                │         ├─► P2P Mode
                │         │         │
                │         │         └─► Holesail Tunnel
                │         │
                │         └─► Public Mode
                │                   │
                │                   └─► Direct Connection
                │
                └─► Proxy to Target

Plugin Request Flow

Plugin Request
      │
      ▼
Plugin Handler
      │
      ├─► Static File Check (www/)
      │         │
      │         └─► Serve if exists
      │
      └─► handler() Function
                │
                ├─► API Routes
                │
                ├─► WebSocket Upgrade
                │
                └─► Return false (404)

State Management

Global state is managed in infrastructure/state.js:

module.exports = {
  // Domain mappings
  domainToIP: new Map(),        // domain -> IP
  holesails: new Map(),         // domain -> Holesail instance
  
  // Server instances
  tlsServers: new Map(),        // domain -> TLS server
  httpServers: new Map(),       // domain -> HTTP server
  
  // Peer management
  peerChannels: new Map(),      // peerId -> { conn, mux } (swarm); plugin RPC state is in pluginChannels
  pluginChannels: new Map(),    // pluginDomain -> protocol -> { peerChannels: Map(peerId -> { rpc, ... }) }
  peerMetrics: new Map(),       // peerId -> metrics
  peerHistory: new Map(),       // peerId -> history
  blockedPeers: new Set(),      // blocked peer IDs
  
  // DNS
  dnsPass: null,                // Autopass instance
  versionPreferences: {},       // domain -> 'p2p' | 'public'
  publicIpForDomain: {},        // domain -> public IP
  domainsWithBoth: new Set(),   // domains with P2P + public
  
  // Timeouts
  holesailClientTimeouts: new Map(), // domain -> timeout ID
  
  // Configuration
  subnets: [],                  // Subnet configurations
  currentSubnetIndex: 0,        // Current subnet for allocation
  currentIPIndex: 2,            // Current IP index in subnet
};

Error Handling Strategy

1. Async Errors: Use wrapAsync() or safePromise() for async operations
2. Circuit Breaker: Protect external service calls
3. Rate Limiting: Prevent abuse of API endpoints
4. Graceful Degradation: Fall back to alternative services
5. Retry Logic: Use retryWithBackoff() for transient failures
6. Resource Cleanup: Track and cleanup resources on shutdown

Configuration

Environment variables are loaded from .env and validated on startup. Key categories:

• Storage: STORAGE_DIR, DOMAINS_FILE, LOCAL_DNS_FILE
• Networking: DNS_PORT, HTTPS_PORT, HTTP_PORT, PUBLIC_DNS_SERVER
• Holesail: INTERNAL_PORT, HOLESAIL_TIMEOUT
• Consensus: CONSENSUS_QUORUM_THRESHOLD, CONSENSUS_MIN_VOTES, CONSENSUS_TIE_BREAKER, CONSENSUS_VOTE_VALIDATION, CONSENSUS_INIT_TIMEOUT_MS
• Backup: BACKUP_DIR, BACKUP_RETENTION, BACKUP_INTERVAL
• Metrics: METRICS_RETENTION_MS, METRICS_SAMPLING_RATE
• Rate Limiting: RATE_LIMIT_MAX_REQUESTS, RATE_LIMIT_WINDOW_MS

See the main README.md for complete configuration reference.

Related Documentation

• Main README - Overview and usage
• Plugin System - Plugin development guide
• Plugin SDK - Plugin SDK API reference
• REST API - API endpoint documentation
• HyperDB - Database integration
• Hyperdrive - Distributed file system
• Plugin Channels - P2P communication