# P2NS Architecture

This document describes the internal architecture of P2NS, including module organization, data flow, and key implementation details.

## Module Organization

P2NS is organized into several module categories under the `includes/` directory:

```
includes/
├── admin/              # Admin interface and API
├── core/               # Core P2NS functionality
├── infrastructure/     # System utilities and patterns
├── maintenance/        # Cleanup, backup, and resource management
├── networking/         # DNS, proxying, and tunneling
├── plugins/            # Plugin system and SDK
└── security/           # Certificate management
```

## Core Modules

### `core/core.js`
Autopass write facade and auto-voting. Re-exports `getConsensusState` and `getConsensusMetrics` from the consensus sidecar read path. Handles:
- Domain claim creation (`claim:domain:claimant`) via dns-pass-queue dual-write
- Vote management (`vote:domain:claimant:voter`)
- Auto-voting logic
- Autopass entries cache for admin/auto-vote (`getAllEntries`)

### `core/consensus-*.js`
Autobase consensus sidecar ([RFC 0001](rfcs/0001-autobase-consensus.md)):
- **`consensus-autobase.js`** — Sidecar lifecycle, bootstrap, hydration, background init, `getConsensusStatus`
- **`consensus-events.js`** — Event encode/decode, KV→event mapping
- **`consensus-apply.js`** — Apply handler; in-memory per-domain view
- **`consensus-view.js`** — Sole `getConsensusState` implementation
- **`consensus-resolver.js`** — Pure quorum/tie-break resolution logic

### `core/domains.js`
Domain management including:
- Adding/removing domains
- Loading domains from `domains.json`
- File watching for hot reloading
- Internal domain detection

### `core/domain_cleanup.js`
Comprehensive domain removal including:
- P2P network claim removal (full removal for resolved claimants, partial removal for competing claims)
- Holesail client cleanup
- Virtual interface removal
- DNS preference cleanup

Supports two removal modes:
- **Full removal**: When user is the resolved claimant, removes all claims, votes, and infrastructure
- **Partial removal**: When user has a competing claim, removes only their claim and votes they cast

## Infrastructure Modules

### `infrastructure/logger.js` + `infrastructure/log-files.js`
Leveled logging with component prefixes, routed to split files under `logs/` via `log-files.js`:
- `logDebug` / `logInfo` / `logWarn` / `logError`
- **core** → console + `core.log`; **Internal Proxy**, **HTTP Proxy**, **DNS**, **Plugins**, **Holesail** → dedicated files (no console)

Admin log tail uses `log-websocket.js` (`subscribe-log`, `file-log`, `log-snapshot`). Plugin logs use separate `plugin-log` WebSocket messages and `app.log` per plugin.

### `infrastructure/state.js`
Global state management including:
- Domain-to-IP mappings
- Holesail connections
- TLS/HTTP servers
- Swarm peer channels (`peerChannels`), plugin RPC state (`pluginChannels`), and metrics
- Version preferences
- DNS pass instance

### `infrastructure/config.js`
Configuration validation on startup:
- Environment variable parsing
- Default value handling
- Type validation

### `infrastructure/validation.js`
Input validation utilities:
- Domain name validation
- Holesail hash validation
- IP address validation
- Port validation

### `infrastructure/circuit_breaker.js`
Circuit breaker pattern implementation for preventing cascading failures:

```javascript
const { getCircuitBreaker } = require('./circuit_breaker');

const breaker = getCircuitBreaker('dns-service', {
  failureThreshold: 5,    // Failures before opening
  resetTimeout: 60000,    // Time before half-open
  monitoringWindow: 60000 // Window for counting failures
});

// States: CLOSED -> OPEN -> HALF_OPEN -> CLOSED
await breaker.execute(async () => {
  // Protected operation
}, 'dns-query');
```

### `infrastructure/rate_limit.js`
In-memory rate limiting for API endpoints:
- Configurable requests per window
- Per-IP tracking
- Automatic cleanup
- Local IP exemption
- Endpoint exemption (GET requests, health checks)

```javascript
const { checkRateLimit } = require('./rate_limit');

const result = checkRateLimit(req);
if (result) {
  // Rate limited - return 429 response
}
```

### `infrastructure/error_handler.js`
User-friendly error handling:
- Error code translation (EADDRINUSE, ENOENT, etc.)
- Production-safe error messages
- Error response formatting

### `infrastructure/async_errors.js`
Async error handling utilities:

```javascript
const { wrapAsync, safePromise, retryWithBackoff } = require('./async_errors');

// Wrap async function with error logging
const safeFn = wrapAsync(asyncFn, 'context-name');

// Execute promise without throwing
const { success, result, error } = await safePromise(promise, 'context');

// Retry with exponential backoff
const result = await retryWithBackoff(asyncFn, {
  maxRetries: 3,
  initialDelay: 1000,
  maxDelay: 10000
});
```

### `infrastructure/utils.js`
Common utility functions:
- Time conversion helpers
- String manipulation
- Object utilities

## Networking Modules

### `networking/dns.js`
UDP DNS server implementation:
- P2P domain resolution
- Local DNS record lookup
- Public DNS fallback
- DNS conflict handling

### `networking/dns_pool.js`
DNS resolver connection pooling:

```javascript
const { dnsPool } = require('./dns_pool');

// Query with automatic failover
const response = await dnsPool.query({
  type: 'query',
  questions: [{ name: 'example.com', type: 'A' }]
});

// Supports multiple DNS servers via PUBLIC_DNS_SERVER env var
// Example: PUBLIC_DNS_SERVER=1.1.1.1,8.8.8.8,9.9.9.9
```

Features:
- Connection pooling (default 5 connections)
- Round-robin query distribution
- Automatic failover between DNS servers
- Query timeout handling (5 seconds)
- Proper listener cleanup

### `networking/holesail.js`
Holesail server/client management:
- Server creation and lifecycle
- Client creation with lazy initialization
- Connection persistence
- Configuration file management

### `networking/holesail_child.js`
Child process for Holesail instances:
- IPC communication with parent
- Console log redirection
- Port availability checking
- Error handling

### `networking/internal_domains_proxy.js`
HTTPS proxy for internal domains:
- Plugin request routing
- Static file serving
- WebSocket upgrade handling
- TLS termination

### `networking/p2p_domains_proxy.js`
HTTPS proxy for P2P domains:
- Holesail client management
- Version preference handling (P2P vs public)
- SSL/TLS tunneling
- Connection timeout management

### `networking/virtual_interfaces.js`
Virtual network interface management:
- IP alias creation on loopback
- Multi-subnet support
- Cross-platform (macOS, Linux, Windows)
- Interface cleanup

## Maintenance Modules

### `maintenance/backup.js`
Backup and restore system:
- Automatic scheduled backups
- Manual backup creation
- Backup rotation
- Restore functionality
- Metadata tracking

### `maintenance/cleanup.js`
Resource cleanup on shutdown:
- Server closure
- Connection termination
- Interface removal
- Port freeing

### `maintenance/metrics.js`
System metrics collection:
- Request statistics
- Response time tracking
- Error rate monitoring
- Historical data aggregation

### `maintenance/resource_tracker.js`
Resource lifecycle management:

```javascript
const { resourceTracker } = require('./resource_tracker');

// Register a resource
const id = resourceTracker.register('socket', socket, () => socket.close());

// Cleanup specific resource
await resourceTracker.cleanup(id);

// Cleanup all resources
await resourceTracker.cleanupAll();

// Get resource counts
const counts = resourceTracker.getCounts();
// { socket: 5, timer: 3, server: 2 }
```

### `maintenance/resource_validation.js`
Periodic resource validation:
- Stale connection detection
- Orphaned server cleanup
- State map consistency checks
- Configurable validation interval

## Plugin Modules

### `plugins/plugin-handler.js`
Plugin lifecycle management:
- Plugin discovery from `plugin-sites/`
- Loading/unloading
- Request routing
- WebSocket registration

### `plugins/sdk.js`
Plugin SDK providing access to:
- State (`sdk.state`)
- DNS operations (`sdk.dns`)
- Domain management (`sdk.domains`)
- Holesail operations (`sdk.holesail`)
- Peer management (`sdk.peers`)
- Certificate management (`sdk.certificates`)
- Local DNS (`sdk.localDns`)
- Interfaces (`sdk.interfaces`)
- Metrics (`sdk.metrics`)
- WebSocket (`sdk.websocket`)
- Configuration (`sdk.config`)
- Backup (`sdk.backup`)
- Subscriptions (`sdk.subscriptions`)
- HTTP client (`sdk.http`)
- File system (`sdk.fs`)
- Events (`sdk.events`)
- Utilities (`sdk.utils`)
- Security (`sdk.security`)
- Logging (`sdk.log`)
- Router (`sdk.router`)
- Admin panel (`sdk.admin`)
- Database (`sdk.db`)
- Channels (`sdk.channels`)
- Drives (`sdk.drives`)
- Authentication (`sdk.auth`)
- Profiles (`sdk.profiles`)

### `plugins/db-manager.js`
HyperDB database management for plugins:
- Schema building
- Database initialization
- Replication management

### `plugins/drive-manager.js`
Hyperdrive file system management:
- Drive creation and caching
- File operations
- Replication

### `plugins/channel-manager.js`
Plugin protomux-rpc management:
- `registerPluginProtocol` — per-plugin RPC muxes and method handlers
- Request/event routing via `channel-rpc.js` and `plugin-rpc-contract.js`
- Peer RPC session tracking in `state.pluginChannels`

### `plugins/replication-manager.js`
Database replication over Hyperswarm:
- Global topic replication
- Peer connection handling
- Core synchronization

### `plugins/hyperdb-builder.js`
HyperDB schema builder:
- Schema generation from config.json
- Helper function loading
- Spec file generation

### `plugins/auth-utils.js`
Authentication utilities for plugins:
- Ed25519 token generation
- Token verification
- Request authentication

## Security Modules

### `security/certificate_authority.js`
TLS certificate management:
- Root CA generation
- Domain certificate creation
- Certificate installation (macOS, Linux, Windows)
- Expiration monitoring

## Data Flow

### DNS Resolution Flow

```
Client Request
      │
      ▼
DNS Server (port 53)
      │
      ├─► Local DNS Records (cache/local_dns.json)
      │         │
      │         └─► Return if found
      │
      ├─► P2P Domain Check
      │         │
      │         ├─► Consensus Resolution
      │         │         │
      │         │         └─► Return internal IP
      │         │
      │         └─► Start Holesail Client (if needed)
      │
      └─► Public DNS Fallback (dns_pool.js)
                │
                └─► Return public IP
```

### Proxy Request Flow

```
HTTPS Request (port 443)
      │
      ▼
SNI Extraction
      │
      ├─► Internal Domain
      │         │
      │         └─► Plugin Handler
      │                   │
      │                   ├─► API Endpoint
      │                   │
      │                   └─► Static Files
      │
      └─► P2P Domain
                │
                ├─► Version Preference Check
                │         │
                │         ├─► P2P Mode
                │         │         │
                │         │         └─► Holesail Tunnel
                │         │
                │         └─► Public Mode
                │                   │
                │                   └─► Direct Connection
                │
                └─► Proxy to Target
```

### Plugin Request Flow

```
Plugin Request
      │
      ▼
Plugin Handler
      │
      ├─► Static File Check (www/)
      │         │
      │         └─► Serve if exists
      │
      └─► handler() Function
                │
                ├─► API Routes
                │
                ├─► WebSocket Upgrade
                │
                └─► Return false (404)
```

## State Management

Global state is managed in `infrastructure/state.js`:

```javascript
module.exports = {
  // Domain mappings
  domainToIP: new Map(),        // domain -> IP
  holesails: new Map(),         // domain -> Holesail instance
  
  // Server instances
  tlsServers: new Map(),        // domain -> TLS server
  httpServers: new Map(),       // domain -> HTTP server
  
  // Peer management
  peerChannels: new Map(),      // peerId -> { conn, mux } (swarm); plugin RPC state is in pluginChannels
  pluginChannels: new Map(),    // pluginDomain -> protocol -> { peerChannels: Map(peerId -> { rpc, ... }) }
  peerMetrics: new Map(),       // peerId -> metrics
  peerHistory: new Map(),       // peerId -> history
  blockedPeers: new Set(),      // blocked peer IDs
  
  // DNS
  dnsPass: null,                // Autopass instance
  versionPreferences: {},       // domain -> 'p2p' | 'public'
  publicIpForDomain: {},        // domain -> public IP
  domainsWithBoth: new Set(),   // domains with P2P + public
  
  // Timeouts
  holesailClientTimeouts: new Map(), // domain -> timeout ID
  
  // Configuration
  subnets: [],                  // Subnet configurations
  currentSubnetIndex: 0,        // Current subnet for allocation
  currentIPIndex: 2,            // Current IP index in subnet
};
```

## Error Handling Strategy

1. **Async Errors**: Use `wrapAsync()` or `safePromise()` for async operations
2. **Circuit Breaker**: Protect external service calls
3. **Rate Limiting**: Prevent abuse of API endpoints
4. **Graceful Degradation**: Fall back to alternative services
5. **Retry Logic**: Use `retryWithBackoff()` for transient failures
6. **Resource Cleanup**: Track and cleanup resources on shutdown

## Configuration

Environment variables are loaded from `.env` and validated on startup. Key categories:

- **Storage**: `STORAGE_DIR`, `DOMAINS_FILE`, `LOCAL_DNS_FILE`
- **Networking**: `DNS_PORT`, `HTTPS_PORT`, `HTTP_PORT`, `PUBLIC_DNS_SERVER`
- **Holesail**: `INTERNAL_PORT`, `HOLESAIL_TIMEOUT`
- **Consensus**: `CONSENSUS_QUORUM_THRESHOLD`, `CONSENSUS_MIN_VOTES`, `CONSENSUS_TIE_BREAKER`, `CONSENSUS_VOTE_VALIDATION`, `CONSENSUS_INIT_TIMEOUT_MS`
- **Backup**: `BACKUP_DIR`, `BACKUP_RETENTION`, `BACKUP_INTERVAL`
- **Metrics**: `METRICS_RETENTION_MS`, `METRICS_SAMPLING_RATE`
- **Rate Limiting**: `RATE_LIMIT_MAX_REQUESTS`, `RATE_LIMIT_WINDOW_MS`

See the main [README.md](../README.md#environment-variables) for complete configuration reference.

## Related Documentation

- [Main README](../README.md) - Overview and usage
- [Plugin System](plugins/README.md) - Plugin development guide
- [Plugin SDK](plugins/PLUGIN_SDK.md) - Plugin SDK API reference
- [REST API](RESTAPI.md) - API endpoint documentation
- [HyperDB](plugins/HYPERDB.md) - Database integration
- [Hyperdrive](plugins/HYPERDRIVE.md) - Distributed file system
- [Plugin Channels](plugins/PLUGIN_CHANNELS.md) - P2P communication