snxraven/p2ns
2
1
Fork 1
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
461c1aa4c82257cca9b6267f375bf6883b434f21
p2ns/docs/README_LONGFORM.md
T

1121 lines
72 KiB
Markdown
Raw Blame History

# P2NS: Peer-to-Peer Decentralized DNS System
P2NS (Peer-to-Peer Name System) is a robust, firewall-resistant peer-to-peer (P2P) DNS resolution system designed to operate independently of centralized DNS infrastructure. Utilizing UDP hole-punching via the Holesail library, P2NS enables seamless connectivity across challenging network environments, including those behind firewalls, NAT, CGNAT, and restricted connections like 4G/5G or satellite internet (e.g., Starlink).
Built in Node.js, P2NS integrates decentralized data storage (Corestore), peer discovery (Hyperswarm), secure invitations (Autopass), and dynamic tunneling (Holesail). It supports hybrid DNS resolution, automatic virtual network interface creation, HTTP/HTTPS proxying with WebSocket support, TLS certificate management, a peer directory, and a comprehensive web-based admin interface. Ideal for decentralized applications and privacy-focused networking, P2NS is currently optimized for macOS and Linux, with Windows support in progress.
[![screenshot](https://git.ssh.surf/snxraven/p2ns/raw/branch/main/images/domains-tab.png)](https://git.ssh.surf/snxraven/p2ns/raw/branch/main/images/domains-tab.png)
Example Peer-to-Peer domain: https://cert.decode (globally avalible to all peers)
[![screenshot](https://git.ssh.surf/snxraven/p2ns/raw/branch/main/images/p2p-domain-cert-dot-decode.png)](https://git.ssh.surf/snxraven/p2ns/raw/branch/main/images/p2p-domain-cert-dot-decode.png)
Local Example Plugin site with Peer-to-Peer access via the P2NS SDK.
[![screenshot](https://git.ssh.surf/snxraven/p2ns/raw/branch/main/images/internal-domain-example-dot-plugin.png)](https://git.ssh.surf/snxraven/p2ns/raw/branch/main/images/internal-domain-example-dot-plugin.png)
## Table of Contents
- [Features](#features)
- [Architecture Overview](#architecture-overview)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Running the System](#running-the-system)
- [Adding Domains](#adding-domains)
- [Consensus and Voting](#consensus-and-voting)
- [Using the Peer Directory](#using-the-peer-directory)
- [Using the Admin Interface](#using-the-admin-interface)
- [DNS Conflict Selector](#dns-conflict-selector)
- [P2P Domain Conflicts](#p2p-domain-conflicts)
- [Hybrid DNS Resolution](#hybrid-dns-resolution)
- [Proxying and Tunneling](#proxying-and-tunneling)
- [Service Subscriptions](#service-subscriptions)
- [Certificate Authority and TLS](#certificate-authority-and-tls)
- [Logging and Debugging](#logging-and-debugging)
- [Configuration Options](#configuration-options)
- [Environment Variables](#environment-variables)
- [Backup and Recovery](#backup-and-recovery)
- [Health Checks and Diagnostics](#health-checks-and-diagnostics)
- [System Metrics and Monitoring](#system-metrics-and-monitoring)
- [Subnet Configuration](#subnet-configuration)
- [Peer Management](#peer-management)
- [Resource Validation and Cleanup](#resource-validation-and-cleanup)
- [Troubleshooting](#troubleshooting)
- [Security Considerations](#security-considerations)
- [Recent Enhancements](#recent-enhancements)
- [Future Enhancements](#future-enhancements)
- [Contributing](#contributing)
- [Additional Documentation](#additional-documentation)
## Features
P2NS provides a comprehensive set of features for decentralized networking:
- **Decentralized DNS Resolution**: Resolves domains to hashes via a P2P core, independent of ICANN or central DNS servers.
- **Firewall and NAT Traversal**: Uses Holesail for UDP hole-punching to connect across restrictive networks.
- **Dynamic Local IP Assignment**: Assigns unique local IPs (e.g., `192.168.3.x`) to domains via virtual network interfaces.
- **Hybrid DNS Resolution**: Falls back to public DNS (e.g., `1.1.1.1`) for non-P2P domains, with custom local DNS record support.
- **DNS Conflict Selector**: Allows users to choose between P2P and public DNS records for domains with both, managed via the admin interface and persisted in `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`).
- **P2P Domain Conflicts**: Allows users to choose between their local claim hash and the consensus-resolved hash for domains where they have a local claim but another claimant won consensus, stored in `selector_cache.json`.
- **Integrated Proxy Servers**: HTTP (port 80) and HTTPS (port 443) proxies with automatic HTTP-to-HTTPS redirection and WebSocket support. Supports SSL/TLS connections for domains that require secure tunneling.
- **Tunnel Auto-Recovery**: On local tunnel `ECONNREFUSED`, proxy paths restart Holesail clients and perform a one-shot retry for idempotent HTTP methods before returning `503`.
- **Holesail Server and Client Management**: Persistent Holesail servers and clients for tunneling, manageable via the admin interface.
- **Invitation-Based Peer Joining**: Master nodes issue invites to joiners, with optional support for joiners to issue invites (`ALLOW_ANY_WRITER_INVITES`). Master nodes proactively send invites to new peers and automatically attempt to reconnect when connections drop.
- **Voting and Consensus**: Simple voting system for domain claims to resolve conflicts and ensure consensus.
- **Automatic Certificate Management**: Generates and installs a root CA and per-domain certificates with SANs for secure TLS connections.
- **Peer Directory Interface**: Web-based directory at `https://peer.directory` for browsing and searching domains.
- **Web-Based Admin Interface**: Comprehensive management panel at `https://p2ns.admin` for domains, Holesail servers/clients, local DNS, certificates, interfaces, logs, settings, and plugins.
- **Plugin Management System**: Full plugin lifecycle management with start/stop/restart capabilities, action registration, settings management, and real-time log viewing via the admin interface.
- **Stateful Persistence**: Stores data in Corestore, domains in `cache/domains.json`, local DNS in `cache/local_dns.json`, Holesail configurations in `cache/holesail_servers.json` and `cache/holesail_clients.json`, DNS version preferences in `cache/selector_cache.json`, service subscriptions in `cache/subscriptions.json`, peer tracking in `cache/peer_history.json` and `cache/peer_metrics.json`, and plugin settings in `cache/plugin-settings/{domain}.json`.
- **Backup and Recovery**: Automatic and manual backup system with rotation, restore functionality, and metadata tracking.
- **Health Checks and Diagnostics**: Comprehensive health monitoring with liveness/readiness probes, DNS lookup, ping, traceroute, connection testing, and bandwidth monitoring.
- **System Metrics and Monitoring**: Real-time and historical metrics collection with configurable sampling, aggregation, and retention periods.
- **Subnet Configuration**: Multi-subnet support with CIDR notation, allowing flexible IP allocation across multiple network ranges.
- **Resource Validation and Tracking**: Automatic validation of system resources, tracking of peer connections, and resource cleanup.
- **Circuit Breaker Pattern**: Automatic failure detection and recovery with exponential backoff retry logic.
- **Rate Limiting**: Built-in rate limiting for API endpoints to prevent abuse and ensure system stability.
- **Enhanced Error Handling**: Comprehensive error handling with retry logic, graceful degradation, and detailed error reporting.
- **Logging**: Split log files under `logs/` (core, proxy, HTTP proxy, DNS, plugins, Holesail) with live tail in the admin **Logs** tab.
- **Graceful Shutdown**: Cleans up connections, channels, virtual interfaces, and child processes on exit.
## Architecture Overview
P2NS is a modular Node.js application with the following components:
- **Core Components**:
- **Corestore & Autopass** (`core.js`): Manages decentralized storage and secure writer additions. Stores claims (`claim:domain:claimant`) and votes (`vote:domain:claimant:voter`).
- **Hyperswarm** (`swarm.js`): Handles peer discovery using a fixed topic (`sha256('p2ns-dns')`).
- **Protomux / protomux-rpc** (`p2ns.js`, `includes/core/core-rpc.js`): Core invite and consensus on `p2ns.core-request-rpc`; plugin traffic via per-protocol RPC muxes (see [plugins/PLUGIN_CHANNELS.md](plugins/PLUGIN_CHANNELS.md)).
- **DNS Handling** (`dns.js`):
- UDP server on port 53 for P2P and public DNS resolution.
- Supports local DNS records, conflict resolution, and fallback to public DNS.
- **Proxying** (`internal_domains_proxy.js`, `p2p_domains_proxy.js`):
- HTTPS server (port 443) and HTTP redirect server (port 80) for internal and P2P domains.
- Handles WebSocket upgrades for real-time applications.
- Supports SSL/TLS connections for domains that require secure tunneling (uses HTTPS/WSS when SSL flag is enabled).
- Manages version preferences (P2P vs public internet) and routing logic.
- **Holesail Integration** (`holesail.js`, `admin.js`):
- Manages Holesail clients and servers for P2P tunneling.
- Persists configurations and supports restarts/deletions.
- **Virtual Interfaces** (`virtual_interfaces.js`):
- Creates/removes IP aliases on loopback interfaces (e.g., `lo0` on macOS, `lo` on Linux, Windows via netsh).
- Assigns IPs from configured subnets (default: `192.168.3.x`).
- Supports multi-subnet configuration with CIDR notation.
- **Note**: On macOS, if "Stealth Mode" is enabled in System Settings > Network > Firewall Options, ICMP (ping) responses may not work for virtual interfaces. Disable stealth mode or allow ICMP in firewall settings if ping functionality is required.
- **Certificate Authority** (`certificate_authority.js`):
- Generates/installs root CA and domain certificates with SANs.
- Supports IP inclusion in SANs for flexible TLS.
- Manages certificate expiration monitoring and renewal.
- Provides certificate chain fetching for public internet domains.
- **Plugin System** (`plugin-handler.js`, `plugin-sites/`):
- Manages plugins for internal domains (e.g., `peer.directory`).
- Serves `https://peer.directory` via the peer.directory plugin with domain listing and search.
- Provides plugin lifecycle management (start, stop, restart) via admin interface.
- Supports plugin action registration and settings management.
- Real-time log viewing for each plugin with xterm.js terminals.
- **Admin Interface** (`admin.js`, `admin/index.html`):
- Web-based panel with real-time WebSocket updates.
- Manages domains, Holesail servers/clients, local DNS, certificates, interfaces, logs, settings, backups, diagnostics, stats (including health monitoring), DNS conflict preferences, and plugins.
- Plugin management tab with start/stop/restart, action execution, settings configuration, and real-time log viewing.
- **Domains Management** (`domains.js`):
- Adds domains and handles internal domains (e.g., `peer.directory`, `p2ns.admin`).
- **Maintenance Modules**:
- **Backup** (`backup.js`): Automatic and manual backup system with rotation and restore functionality.
- **Metrics** (`metrics.js`): System metrics collection, aggregation, and historical data tracking.
- **Resource Validation** (`resource_validation.js`): Validates and tracks system resources, peer connections, and cleanup operations. Automatically runs at configurable intervals to detect and clean up stale resources.
- **Cleanup** (`cleanup.js`): Closes servers, connections, and interfaces on shutdown.
- **Infrastructure Modules**:
- **Circuit Breaker** (`circuit_breaker.js`): Automatic failure detection and recovery with exponential backoff.
- **Rate Limiting** (`rate_limit.js`): API endpoint rate limiting to prevent abuse.
- **Error Handler** (`error_handler.js`): Comprehensive error handling with retry logic and graceful degradation.
- **Validation** (`validation.js`): Input validation and configuration validation.
- **Config** (`config.js`): Configuration validation on startup.
- **State Management** (`state.js`):
- Maintains global state for mappings, configurations, DNS version preferences, metrics, and peer tracking.
- **Logging** (`logger.js`):
- Provides prefixed, leveled logging.
The system operates in **Master** (initializes core, loads `cache/domains.json` or `DOMAINS_FILE`) or **Joiner** (requests invites) modes.
## Prerequisites
- **Node.js**: Version 18 or higher (tested up to 20.x).
- **Holesail CLI**: For generating connection hashes (`npm install -g holesail`).
- **Dependencies**: `corestore`, `hyperswarm`, `autopass`, `protomux`, `compact-encoding`, `hyperdb`, `hyperschema`, `hyperdrive`, `b4a`, `sodium-native`, `node-forge`, `dns-packet`, `holesail`, `holesail-logger`, `dotenv`, `ws`, `http-proxy`, `sharp`, `pidusage`, `selfsigned`, and dev tooling (`tailwindcss`, `postcss`). See `package.json` for pinned versions.
- **Platform**: macOS or Linux (sudo required for ports <1024 and interfaces). Windows support in progress.
- **Optional**: Configuration files (default locations in `cache/` directory): `cache/domains.json`, `cache/local_dns.json`, `cache/holesail_servers.json`, `cache/holesail_clients.json`, `cache/selector_cache.json`, `cache/subscriptions.json`, `cache/peer_history.json`, `cache/peer_metrics.json`, and `.env` for configuration. File paths can be customized via environment variables.
## Installation
1. **Clone the Repository**:
```bash
git clone https://git.ssh.surf/snxraven/p2ns.git
cd p2ns
```
2. **Install Dependencies**:
```bash
npm install
```
3. **Prepare Configuration Files** (Optional):
By default, configuration files are stored in the `cache/` directory. You can customize paths via environment variables (see [Environment Variables](#environment-variables)).
- **cache/domains.json** (or `DOMAINS_FILE`): Pre-load domains for master nodes:
```json
[
{ "domain": "example.tld", "hash": "hs://s00084bf87dfa89a3048fb081c0e6207eb5a", "ssl": false },
{ "domain": "another.example", "hash": "hs://s000bcc379b38f6d3a5cb4cfde23fa52392d", "ssl": true }
]
```
The `ssl` field (boolean, optional) indicates if the Holesail connection uses SSL/TLS. When `true`, proxy connections will use HTTPS/WSS instead of HTTP/WS. Defaults to `false` if not specified.
- **cache/local_dns.json** (or `LOCAL_DNS_FILE`): Custom DNS records (supports A, AAAA, CNAME, MX, TXT, SRV, SOA, CAA, NS, PTR, OTHER):
```json
[
{ "name": "local.example", "type": "A", "class": "IN", "ttl": 3600, "data": "192.168.1.100" },
{ "name": "mail.example", "type": "MX", "class": "IN", "ttl": 3600, "preference": 10, "exchange": "mx.example.com" },
{ "name": "_service._tcp.example", "type": "SRV", "class": "IN", "ttl": 3600, "priority": 0, "weight": 5, "port": 8080, "target": "server.example.com" },
{ "name": "example.com", "type": "SOA", "class": "IN", "ttl": 86400, "mname": "ns1.example.com", "rname": "admin.example.com", "serial": 2025090801, "refresh": 3600, "retry": 600, "expire": 604800, "minimum": 3600 },
{ "name": "example.com", "type": "CAA", "class": "IN", "ttl": 3600, "flags": 0, "tag": "issue", "value": "letsencrypt.org" }
]
```
- **cache/holesail_servers.json** (or `HOLESAIL_SERVERS_FILE`): Persistent Holesail servers:
```json
{
"servers": [
{ "id": "abc123", "opts": { "name": "server1", "port": 8080, "host": "0.0.0.0", "secure": true, "log": 1 } }
]
}
```
- **cache/holesail_clients.json** (or `HOLESAIL_CLIENTS_FILE`): Persistent Holesail clients:
```json
{
"clients": [
{ "id": "def456", "opts": { "domain": "example.tld", "key": "hs://s00084bf...", "port": 8080, "protocol": "tcp" } }
]
}
```
- **cache/selector_cache.json** (or `SELECTOR_CACHE_FILE`): DNS version preferences and hash preferences for domains with conflicts:
```json
{
"versionPreferences": {
"myspace.com": "public",
"example.com": "p2p"
},
"hashPreferences": {
"conflicted.com": "local"
}
}
```
4. **Configure Environment** (Optional):
Copy `default.env` to `.env` and customize (see [Environment Variables](#environment-variables)).
## Running the System
- **Genesis master** (creates the network Autopass and `cache/network.json` manifest; loads `domains.json` on first run):
```bash
sudo node p2ns.js --master --genesis
```
Or set `P2NS_MASTER=true` and `P2NS_GENESIS=true` in `.env` for Docker/PM2.
- **Secondary master** (same invite authority and reconnect policy; joins an existing network via invite — does not create a new Autopass on empty storage):
```bash
sudo node p2ns.js --master
```
Start with empty `STORAGE_DIR`, connect to a genesis or writer peer, and pair like a joiner. Set `MASTER_LOAD_DOMAINS=true` only if this node should import `domains.json`.
- **Legacy single master** (existing `my-storage` without manifest): first startup with new code auto-writes `cache/network.json` from the existing Autopass.
Master-capable nodes have enhanced peer management:
- **Proactive Invite Sending**: Automatically sends invites to peers when they connect, eliminating the need for peers to request invites.
- **Automatic Reconnection**: Actively attempts to reconnect to disconnected peers using Hyperswarm's peer discovery mechanisms.
- **Reconnection Tracking**: Tracks disconnected peers and attempts reconnection with exponential backoff (configurable via `MASTER_RECONNECT_INTERVAL` and `MASTER_MAX_RECONNECT_ATTEMPTS`).
- **Invite After Reconnection**: Automatically sends invites again when peers successfully reconnect.
- **Joiner Node** (connects to peers via invites):
```bash
sudo node p2ns.js
```
- **Clean Storage** (removes `./my-storage` for a fresh start):
```bash
sudo node p2ns.js --clean [--master] [--genesis]
```
The system binds to:
- **UDP 53**: DNS server (unless `DISABLE_DNS_SERVER=true`).
- **TCP 443**: HTTPS proxy (localhost, unless `DISABLE_PROXY_SERVER=true`).
- **TCP 80**: HTTP redirect (per domain, unless `DISABLE_PROXY_SERVER=true`).
- **Internal Ports**: Holesail clients (default: `8080`).
On startup, P2NS:
- Installs a root CA and generates certificates for internal domains (`peer.directory`, `peer.dir`, `p2ns.admin`).
- Loads `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) to initialize DNS version preferences and hash preferences for conflicting domains.
- Sets up DNS, proxy, and Holesail services.
- Loads persisted Holesail servers and clients from `cache/holesail_servers.json` and `cache/holesail_clients.json` (or respective environment variables).
- Monitors `cache/domains.json` (or `DOMAINS_FILE`) for changes (Master/Joiner).
- Initializes WebSocket for admin interface updates.
Access the admin interface at `https://p2ns.admin` and the peer directory at `https://peer.directory` (trust the root CA in your browser).
## Adding Domains
Domains are added as claims with Holesail hashes. **Note: Internal domains (such as `p2ns.admin`, `peer.directory`, and any domains defined in `plugin-sites/{domain}/config.json`) cannot be claimed and are reserved for local use only.**
1. **Generate a Hash**:
```bash
holesail --live 80 --public
```
Output: `Connection hash: hs://<hash>`
2. **Command Line** (Master only, interactive):
After starting P2NS, you can use the interactive command line interface to add domains. Type:
```
add example.tld hs://<hash>
```
The command format is: `add <domain> <hash>`
**Limitations:**
- Only available when running as Master node
- Must be typed in the terminal where P2NS is running
- Interactive input only (not suitable for scripts)
- For programmatic access, use the admin API or edit `cache/domains.json` directly
3. **cache/domains.json** (or `DOMAINS_FILE`):
Edit `cache/domains.json` and restart or let the watcher reload (Master/Joiner).
4. **Admin Interface**:
Use the "Domains" tab to add domains interactively. When adding a domain, you can check the "This Holesail Connection Uses SSL/TLS" checkbox if the service being tunneled uses SSL/TLS. This will ensure proxy connections use HTTPS/WSS instead of HTTP/WS.
Domains sync across peers via Autopass, with consensus resolved through the Autobase sidecar ([RFC 0001](docs/rfcs/0001-autobase-consensus.md)).
**Automatic Consensus Updates**: When domains are added, the system automatically:
- Invalidates DNS cache to ensure fresh consensus checks
- Recalculates consensus for the new domain
- Triggers network-wide consensus recalculation requests
- Updates conflict badges and UI immediately
## Domain Removal
Domain removal behavior depends on your relationship to the domain:
### Resolved Claimant Removal (Full Cleanup)
When you remove a domain where you are the resolved claimant:
- **Local**: Performs full cleanup (`atomicDomainCleanup`) removing all claims and votes for the domain
- **Broadcasts**: `consensus.removeDomain` RPC event to all peers (`{ domain }`)
- **Other Peers**: Each peer processes the removal request and removes their own claims if they have any
- **Result**: All claims and votes for the domain are removed from the network
### Conflict Claim Removal (Partial Removal)
When you remove a domain where you have a claim but aren't the resolved claimant:
- **Local**: Removes only your own claim and votes (`removeOwnClaimAndVotes`)
- **Broadcasts**: `removeConflictDomainClaim:${domain}` notification message
- **Other Peers**: Receive notification, update consensus state, but **do NOT remove their claims**
- **Result**: Only your conflict claim is removed, all other peers' claims remain intact
**Claim Ownership Protection**: The system enforces strict ownership validation:
- Only the peer that created a claim can modify or remove it
- Ownership is verified by matching claimant ID before any operation
- Other peers' claims are never affected by your modifications
- Prevents unauthorized claim overwriting or deletion
## Consensus and Voting
P2NS uses a decentralized consensus mechanism to resolve domain claims across the peer network. Resolution is computed by an **Autobase apply-based consensus sidecar** ([RFC 0001](docs/rfcs/0001-autobase-consensus.md)): Autopass KV remains the durable write store; every local claim/vote mutation dual-appends typed events to the sidecar; `getConsensusState` reads from the apply view only.
### How Consensus Works
**Claims**: When a domain is added, a claim is created in the Autopass ledger with the key format `claim:domain:claimant`, where `claimant` is the public key of the peer making the claim. The claim value is a JSON object containing the Holesail hash, timestamp, and optionally an SSL flag: `{hash, timestamp, ssl}`. The `ssl` field (boolean) indicates if the Holesail connection uses SSL/TLS, which determines whether proxy connections use HTTPS/WSS instead of HTTP/WS.
**Claim Ownership and Modification**:
- Each peer can only modify or remove their own claims (strict ownership validation)
- When updating a domain, the system checks if you already have a claim and updates it instead of creating duplicates
- Ownership is verified by matching the claimant ID in the claim key before any modification
- Other peers' claims are never affected by your modifications
- If a domain already has a claim with the same hash and SSL flag, the update is skipped (idempotent)
**Votes**: Peers vote on claims by creating vote entries with the key format `vote:domain:claimant:voter`. Each peer can vote for one claimant per domain. Votes are stored in the Autopass ledger and synchronized across all peers.
**Quorum**: For a domain to be resolved, a quorum must be met. The quorum is calculated as:
```
minVotes = max(CONSENSUS_MIN_VOTES, ceil(activePeers * CONSENSUS_QUORUM_THRESHOLD))
```
Where `activePeers` includes the local node. This ensures that a sufficient number of peers have participated in the consensus process.
**Resolution**: A domain is resolved when:
1. One claimant has the most votes
2. The quorum threshold is met
3. The resolved hash is used for DNS resolution and proxying
### Auto-Voting
P2NS automatically votes on domain claims based on the following logic:
1. **Own Claim**: If the local peer has a claim for a domain, it automatically votes for its own claim.
2. **Single Claim**: If there is only one claim for a domain, the peer automatically votes for it.
3. **Multiple Claims**: If there are multiple claims, the peer uses the tie-breaking strategy (see below) to determine which claimant to vote for.
Auto-voting runs periodically and when new claims or votes are detected. Peers will also remove votes for other claimants if their voting decision changes.
### Tie-Breaking Strategies
When multiple claimants have the same number of votes, a tie-breaking strategy is applied. The strategy is configurable via `CONSENSUS_TIE_BREAKER`:
- **`timestamp`** (default): Prefers the oldest claim (first-come-first-served). This rewards early domain registration.
- **`claimant_age`**: Prefers the local writer if they are among the tied claimants; otherwise uses lexicographic ordering of public keys.
- **`lexicographic`**: Prefers the local peer's claim if it's a candidate; otherwise uses lexicographic (alphabetical) ordering of claimant public keys.
### Consensus States
A domain can be in one of several consensus states:
- **`resolved`**: The domain has been successfully resolved with a clear winner and quorum met.
- **`conflict`**: Domain resolved to another claimant, but local peer has a competing claim.
- **`insufficient_quorum`**: There are claims but not enough votes to meet the quorum threshold.
- **`tie`**: Multiple claimants have the same number of votes (tie-breaker applied).
- **`no_claims`**: No claims exist for this domain.
Only domains in the `resolved` state will return a hash for DNS resolution. Other states indicate that consensus has not yet been reached. The `conflict` state indicates that while the domain is resolved to another claimant, the local peer still maintains a competing claim.
### Consensus Configuration
The consensus mechanism can be configured via environment variables:
- **`CONSENSUS_QUORUM_THRESHOLD`** (default: `0.5`): The percentage of active peers that must vote to meet quorum (0.0-1.0). For example, `0.5` means 50% of peers must vote.
- **`CONSENSUS_MIN_VOTES`** (default: `2`): The minimum number of votes required regardless of peer count. This ensures quorum even in small networks.
- **`CONSENSUS_TIE_BREAKER`** (default: `timestamp`): The strategy used to break ties between claimants with equal votes. Options: `timestamp`, `claimant_age`, `lexicographic`.
- **`CONSENSUS_VOTE_VALIDATION`** (default: `true`): Whether to validate that votes reference existing claims. Invalid votes are ignored.
- **`CONSENSUS_IMMEDIATE_UPDATE`** (default: `true`): Whether to trigger consensus recalculation immediately when claims or votes are added/removed, rather than waiting for the next periodic check.
### Consensus Metrics
The system tracks consensus metrics including:
- Total resolutions and failures
- Quorum failures
- Tie resolutions
- Vote validation failures
- Average votes per domain
- Per-domain resolution statistics
These metrics are available via the admin interface and the `/api/consensus/metrics` API endpoint (includes `sidecar` health and `bootstrapComplete`).
### Monitoring Consensus
You can monitor consensus state for domains via:
- **Admin Interface**: The Domains tab shows consensus status badges for each domain.
- **API**: Use `GET /api/consensus/:domain` to get detailed consensus state for a specific domain.
- **Sidecar health**: Use `GET /api/consensus/status` for sidecar readiness, bootstrap progress, and event counts.
- **Metrics**: Use `GET /api/consensus/metrics` to view overall consensus statistics (includes embedded sidecar status).
For more details, see the [REST API documentation](docs/RESTAPI.md#consensus-endpoints).
## Peer Management
P2NS tracks peer connections, metrics, and history to provide visibility into the network and enable peer management features.
### Peer Connection Lifecycle
When peers connect via Hyperswarm:
1. **Connection Established**: Peer connection is detected and tracked
2. **RPC Setup**: Core `p2ns.core-request-rpc` and plugin RPC muxes attach to the connection
3. **Replication**: Corestore replication begins automatically
4. **Invite Exchange**: Joiners receive Autopass invite wire via `invite.deliver` RPC when needed
5. **Metrics Tracking**: Connection metrics are recorded (duration, timestamps)
6. **Disconnection**: On disconnect, metrics are finalized and stored in history
### Peer Blocking
You can block peers to prevent them from connecting:
- **Via Admin Interface**: Go to the "Peers" tab, find the peer, and click "Block"
- **Via API**: `POST /api/peers/:peerId/block`
- **Via SDK**: `await sdk.peers.blockPeer('peer-id')`
Blocked peers:
- Are prevented from establishing new connections
- Have existing connections closed immediately
- Are stored in `cache/blocked_peers.json` (or `BLOCKED_PEERS_FILE`)
- Can be unblocked via admin interface, API, or SDK
### Peer Metrics and History
P2NS tracks comprehensive peer information:
- **Connection Metrics**: Number of connections, total duration, average duration, last seen timestamp
- **Connection History**: Timestamped log of all connection/disconnection events
- **Peer Information**: Peer ID, connection status, uptime, and metadata
Access peer information via:
- **Admin Interface**: "Peers" tab shows all connected peers with metrics and history
- **API**: `GET /api/peers` and `GET /api/peers/:peerId`
- **SDK**: `sdk.peers.getPeerInfo()`, `sdk.peers.getPeerMetrics()`, `sdk.peers.getPeerHistory()`
Peer data is persisted in:
- `cache/peer_history.json` (or `PEER_HISTORY_FILE`): Connection history
- `cache/peer_metrics.json` (or `PEER_METRICS_FILE`): Aggregated metrics
- `cache/blocked_peers.json` (or `BLOCKED_PEERS_FILE`): Blocked peer list
### Duplicate Connection Handling
P2NS automatically handles duplicate connections from the same peer:
- If a valid existing connection exists, new duplicate connections are rejected
- If the existing connection is stale (closed), it's cleaned up and the new connection is accepted
- This prevents resource leaks and ensures efficient peer management
## Using the Peer Directory
Access `https://peer.directory` to:
- View all claimed domains (sorted alphabetically).
- Search domains in real-time.
- Click links to visit domains (e.g., `https://example.tld`).
Served via the HTTPS proxy with internal resolution.
## Using the Admin Interface
The admin interface at `https://p2ns.admin` provides real-time management via WebSocket. Features include:
- **Domains**: View, search, add, and remove domains. Local claims marked with 🏠. Supports pagination and real-time updates. When adding domains, the system automatically checks for existing claims and updates them instead of creating duplicates. Domain removal behavior depends on consensus status:
- **Resolved Claimant**: Full cleanup removes all claims and votes for the domain
- **Conflict Claim**: Partial removal removes only your own claim and votes, preserving other peers' claims
- **Host**:
- **Holesail Servers**: Create, restart, delete, and view logs for servers (name, port, host, key, secure/UDP, log level). Persisted in `cache/holesail_servers.json` (or `HOLESAIL_SERVERS_FILE`).
- **Holesail Clients**: Create, restart, delete, and view logs for clients (domain, key, port, service name, protocol). Persisted in `cache/holesail_clients.json` (or `HOLESAIL_CLIENTS_FILE`). Only domains you own are shown in the "Create Client" modal.
- **Service Subscription**: Subscribe to services from other domains in the network. Auto-subscribe to all services for a domain, or manage individual service subscriptions. Subscriptions are automatically synchronized when services are added or removed by domain owners.
- **Local DNS**: Manage custom DNS records (A, AAAA, CNAME, MX, TXT, SRV, SOA, CAA, NS, PTR, OTHER) with edit/delete options. Persisted in `cache/local_dns.json` (or `LOCAL_DNS_FILE`). Includes DNS Conflict Selector for managing domains with both P2P and public records, and P2P Domain Conflicts for managing domains with consensus conflicts.
- **Entries**: View Autopass ledger entries (claims/votes) with search and pagination.
- **Peers**: List connected peers (public keys) with search and pagination.
- **Certificates**: Generate, delete, regenerate, and view domain certificates. Manage root CA (regenerate/install).
- **Interfaces**: View domain-to-IP mappings and clean up unused interfaces.
- **Backups**: Create manual backups, restore from backups, view backup metadata, and manage backup retention. Automatic backups run on a configurable interval.
- **Diagnostics**: Network diagnostics tools including DNS lookup, ping, traceroute, connection testing, and bandwidth monitoring. Supports streaming output for real-time results.
- **Stats**: Real-time system metrics including request statistics, Holesail connection status, process metrics (CPU, memory), and historical data with configurable time ranges. Includes system health monitoring with service status, dependency checks, and readiness/liveness probe information.
- **Logs**: Real-time terminal view of system logs (DEBUG/INFO/WARN/ERROR).
- **Settings**: Edit environment variables with persistence to `.env`. Includes subnet configurator for managing multiple subnet ranges with CIDR notation.
- **Plugins**: Manage plugins with start/stop/restart capabilities, execute plugin actions, configure plugin settings, and view real-time logs for each plugin. Plugins are automatically discovered from `plugin-sites/` directories. Each plugin card shows status, version, description, registered actions, settings, and a log terminal. Logs are hidden by default and appear when you start/stop/restart a plugin.
The interface uses Tailwind CSS and xterm.js for a polished, terminal-based experience. Modals handle actions, and notifications show success/errors. For programmatic access, see the [REST API documentation](docs/RESTAPI.md) for details on the backend API endpoints.
## DNS Conflict Selector
The DNS Conflict Selector, located in the "Local DNS" tab of the admin interface, allows users to manage domains that have both P2P and public DNS records. This feature addresses scenarios where a domain (e.g., `myspace.com`) is claimed in the P2NS network but also exists in public DNS.
- **Functionality**:
- Displays a table of domains with both P2P and public records, showing the domain name, public IP (from `state.publicIpForDomain`), and current mode (P2P or Public).
- Provides a toggle switch to select between P2P (default, resolves to internal IP with TTL 1) and Public (resolves to public IP with TTL 1).
- Preferences are stored in `state.versionPreferences` and persisted to `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) for durability across server restarts.
- When a preference is set (e.g., `myspace.com: public`), the middleware selection page (`p2p_domains_proxy.js`) is bypassed, and the system uses the specified mode for both DNS resolution and proxying.
- If no preference is set, a middleware page prompts the user to choose between P2P and Public versions, setting a cookie and updating `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`).
- Supports all DNS record types (A, AAAA, CNAME, MX, TXT, SRV, SOA, CAA, NS, PTR, OTHER) for local DNS records, allowing fine-grained control over resolution.
- **Usage**:
- Navigate to the "Local DNS" tab in the admin interface.
- View the "DNS Conflict Selector" section, which lists all domains from `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) and `state.domainsWithBoth` on startup.
- Use the search bar (`search-dns-conflicts`) to filter domains by name, version, or public IP.
- Toggle the switch to change a domain's mode (e.g., from P2P to Public). Changes are saved instantly to `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) and reflected in DNS responses and proxy routing.
- Add/edit/delete custom DNS records in the "Local DNS" section to override P2P or public resolutions (e.g., SRV for service discovery, CAA for certificate authority restrictions).
- Remove a domain via the "Domains" tab to clear its preference from `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) and the table.
- **Persistence**:
- Version preferences are stored in `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) (e.g., `{"myspace.com": "public", "example.com": "p2p"}`).
- Loaded on server startup into `state.versionPreferences` for immediate use.
- Updated automatically when toggling preferences or removing domains.
- **Behavior**:
- **DNS**: Defaults to P2P (internal IP, TTL 1) unless `state.versionPreferences` specifies `public` (public IP, TTL 1). Custom DNS records in `cache/local_dns.json` (or `LOCAL_DNS_FILE`) take precedence.
- **Proxying**: Routes to internal IPs for P2P or public IPs for Public mode, bypassing the middleware page when a preference is set.
- **Admin Interface**: Displays all cached preferences and local DNS records on startup, ensuring no site visit is required to populate the table.
- **Testing**:
- Create a `cache/selector_cache.json` (or set `SELECTOR_CACHE_FILE`) with entries (e.g., `{"myspace.com": "public"}`).
- Add custom DNS records to `cache/local_dns.json` (or `LOCAL_DNS_FILE`) with various types (e.g., SRV, SOA, CAA).
- Start the server and access the admin panel's Local DNS tab to verify the table shows all cached domains and records.
- Toggle a preference and check that `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) updates and DNS/proxy behavior changes (e.g., `dig @127.0.0.1 myspace.com` returns public IP).
- Add/edit/delete a DNS record (e.g., SRV for `_service._tcp.example`) and verify it appears in `cache/local_dns.json` (or `LOCAL_DNS_FILE`) and resolves correctly (`dig @127.0.0.1 _service._tcp.example SRV`).
- Remove a domain and confirm its preference is removed from `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) and the table.
## Hybrid DNS Resolution
- **P2P Domains**: Resolve to local IPs, start Holesail tunnels.
- **Public Domains**: Forward to `PUBLIC_DNS_SERVER` (default: `1.1.1.1`) with TTL 1 for domains with both records when public mode is selected.
- **Local DNS Records**: Served from `cache/local_dns.json` (or `LOCAL_DNS_FILE`) with support for all record types (A, AAAA, CNAME, MX, TXT, SRV, SOA, CAA, NS, PTR, OTHER).
- **Internal Domains**: Map to `127.0.0.1`.
- **Conflicting Domains**: Managed via the DNS Conflict Selector, with preferences stored in `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`).
Test with:
```bash
dig @127.0.0.1 example.tld
```
## P2P Domain Conflicts
The P2P Domain Conflicts feature, located in the "Local DNS" tab of the admin interface, allows users to manage domains where they have a local claim but another claimant won the P2P consensus. This addresses scenarios where multiple users claim the same domain, and the consensus algorithm selects a different winner than the local user.
- **Functionality**:
- Displays a table of domains with P2P consensus conflicts where the user has a local claim but consensus resolved to another claimant.
- Shows domain name, local claim hash, consensus-resolved hash, and current hash preference.
- Provides a toggle switch to select between using the local claim hash or the consensus-resolved hash.
- Preferences are stored in `state.hashPreferences` and persisted to `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) for durability across server restarts.
- When a preference is set (e.g., `example.com: local`), DNS resolution and Holesail client creation use the selected hash.
- Automatically restarts any active Holesail clients for the domain when preferences change to ensure immediate effect.
- **Usage**:
- Navigate to the "Local DNS" tab in the admin interface.
- Click the "P2P Conflicts" sub-tab.
- View domains where you have local claims but consensus resolved to another claimant.
- Use the search bar (`search-p2p-domain-conflicts`) to filter domains.
- Toggle the switch to change a domain's hash preference (Local vs Resolved).
- Changes are applied immediately - existing Holesail clients are restarted with the new hash.
- **Persistence**:
- Hash preferences are stored in `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) alongside version preferences.
- Example: `{"versionPreferences": {"myspace.com": "public"}, "hashPreferences": {"example.com": "local"}}`
- Loaded on server startup into `state.hashPreferences` for immediate use.
- **Automatic Client Management**:
- When hash preferences change, the system automatically:
1. Clears DNS cache for immediate effect
2. Closes existing Holesail connections for the domain
3. Creates new Holesail clients with the correct hash
4. Verifies connections are active before completing
- **Automatic Cleanup**:
- Hash preferences are automatically validated and cleaned up:
- On server startup after consensus completes
- When domains are removed
- Invalid preferences are removed (e.g., user no longer has local claim, conflict resolved)
- Preferences are validated against current consensus state to ensure accuracy
- **Testing**:
- Create domains with conflicting claims in the P2P network.
- Set hash preferences in the P2P Domain Conflicts interface.
- Verify that DNS resolution returns the correct hash: `dig @127.0.0.1 example.com`
- Check that Holesail clients connect to the correct hash by monitoring logs.
- Toggle preferences and confirm clients restart with new hashes.
## Proxying and Tunneling
- **HTTPS Proxy**: Routes requests to local IPs via Holesail (port 443, localhost) or public IPs based on DNS Conflict Selector preferences.
- **HTTP Redirect**: Redirects to HTTPS (port 80, per domain).
- **TLS Proxy**: Per-domain TLS servers with SNI support.
- **Holesail Clients**: Start lazily, timeout after `HOLESAIL_TIMEOUT` (default: 5 minutes) unless persistent. Support both TCP and UDP protocols. Clients created for subscribed services are automatically named using the format `domain_servicename` to avoid naming conflicts.
- **Tunnel Failure Recovery**: If proxying to the local Holesail endpoint fails with `ECONNREFUSED`, P2NS now restarts the Holesail client and performs a one-shot retry for idempotent HTTP methods (`GET`, `HEAD`, `OPTIONS`) before returning `503`.
- **Fast-Fail Timeout**: Internal proxy upstream requests use `INTERNAL_PROXY_TIMEOUT_MS` (default `12000`) to avoid hanging requests during tunnel failure states.
- **Holesail Servers**: Run persistently, configurable via admin interface.
- **WebSocket Support**: Handles upgrades for real-time applications.
## Service Subscriptions
P2NS supports automatic subscription to services published by domain owners across the network. This feature allows peers to automatically create Holesail clients for services they want to access, with dynamic synchronization when services are added or removed.
### Domain Ownership
Domain ownership is determined by consensus: a domain is considered "owned" by a peer if that peer is the resolved claimant for the domain (i.e., their claim has the most votes and meets quorum). Only domains you own are shown in the "Create Client" modal, ensuring you can only create clients for domains you control.
### Service Records in Claims
Domain claim records now include a `clients` array that lists all services configured for that domain. Each service entry contains:
- `serviceName`: Unique identifier for the service
- `key`: Holesail connection hash (`hs://...`)
- `port`: Port number for the service
- `protocol`: Either `tcp` or `udp`
The claim record structure is: `{hash, clients: [...], timestamp, ssl}`. This allows multiple services per domain while maintaining a single claim record. The `ssl` field (boolean, optional) indicates if the Holesail connection uses SSL/TLS for secure tunneling.
### Service Subscription Features
- **Service Subscription Modal**: Accessible from the "Host" tab via the "Service Subscription" button. Lists all domains in the network with their available services, searchable by domain name.
- **Individual Service Subscription**: Subscribe to specific services from any domain. Creates a Holesail client automatically with the naming format `domain_servicename`.
- **Subscribe All**: Per-domain option to automatically subscribe to all services for that domain. When enabled, new services added by the domain owner are automatically subscribed to by all peers with this option enabled.
- **Protocol Selection**: When creating clients manually or subscribing to services, you can specify the protocol (TCP or UDP). Defaults to TCP if not specified.
- **Auto-Subscription on Bootup**: On system startup, all subscribed services automatically create their Holesail clients (unless `DISABLE_AUTO_SUBSCRIPTION` is enabled).
- **Dynamic Synchronization**: The system automatically:
- **Unsubscribes** from services when they are removed from a domain's claim record by the domain owner
- **Subscribes** to new services when they are added to a domain's claim record, if "subscribe all" is enabled for that domain
- Monitors AutoPass `update` and `append` events to detect claim changes
- Uses debouncing to prevent excessive processing during rapid updates
### Subscription Storage
Subscriptions are persisted in `cache/subscriptions.json` with the following structure:
```json
[
{
"domain": "example.tld",
"subscribeAll": true,
"services": [
{
"serviceName": "web",
"key": "hs://s00084bf...",
"port": 8080,
"protocol": "tcp"
}
]
}
]
```
The `subscribeAll` flag indicates whether to automatically subscribe to all services for that domain. Individual service subscriptions are stored in the `services` array.
### Configuration
- **`DISABLE_AUTO_SUBSCRIPTION`**: Set to `true` to disable automatic starting of Holesail clients for subscribed services on bootup or when new services are detected. Default: `false`.
- **`SUBSCRIPTIONS_FILE`**: Path to subscriptions storage file. Default: `./cache/subscriptions.json`.
### Usage
1. **Subscribing to Services**:
- Open the "Service Subscription" modal from the "Host" tab
- Search for domains or browse the list
- Click "Subscribe" next to a service to subscribe individually
- Click "Subscribe to All" to enable automatic subscription to all services for that domain
2. **Managing Subscriptions**:
- View all subscriptions in the Service Subscription modal
- Unsubscribe from individual services or disable "subscribe all" for a domain
- Subscriptions are automatically synchronized when domain owners add or remove services
3. **Creating Clients Manually**:
- Use the "Create Client" button in the "Host" tab
- Only domains you own are shown in the domain list
- Specify service name and protocol (TCP/UDP) when creating
- The client ID will be `domain_servicename` for subscribed services, or a generated ID for manual clients
## Certificate Authority and TLS
- **Root CA**: Generated in `./certs/ca.cert.pem`, installed on macOS (Keychain), Linux (system CA + NSS for Chrome), and Windows (`certutil`).
- **Domain Certs**: Created in `./certs/<domain>/` with SANs for domain and IPs.
- **SNI Support**: Dynamic certificate selection for HTTPS proxy.
- **Auto-Install**: Checks/reinstalls CA if expired; supports macOS, Linux, and Windows (via `certutil`).
- **Conflict Handling**: Bypasses middleware page for domains with preferences in `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`).
### Manual Root CA Installation
If automatic SSL certificate installation fails, you can manually install the root CA certificate. The certificate is located at `./certs/ca.cert.pem` (or `CERTS_DIR/ca.cert.pem` if `CERTS_DIR` is set in your `.env`).
#### Operating System Level Installation
**macOS:**
1. Open **Keychain Access** (Applications > Utilities > Keychain Access).
2. Select **System** keychain from the left sidebar.
3. Go to **File** > **Import Items...** (or press `Cmd+O`).
4. Navigate to `./certs/ca.cert.pem` and select it.
5. Find "P2NS CA" in the list, double-click it.
6. Expand **Trust** and set "When using this certificate" to **Always Trust**.
7. Close the dialog and enter your password when prompted.
Alternatively, via command line (requires sudo):
```bash
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ./certs/ca.cert.pem
```
**Linux:**
Install **both** the system CA bundle and the **NSS** database (Chrome/Chromium on Linux ignore `update-ca-certificates` alone).
Prerequisite: `certutil` (`libnss3-tools` on Debian/Ubuntu, `nss-tools` on Fedora).
```bash
# System trust store
sudo cp ./certs/ca.cert.pem /usr/local/share/ca-certificates/p2ns-ca.crt
sudo update-ca-certificates
# User NSS DB (Chrome/Chromium)
mkdir -p ~/.pki/nssdb
certutil -d sql:$HOME/.pki/nssdb -D -n "P2NS CA" 2>/dev/null || true
certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n "P2NS CA" -i ./certs/ca.cert.pem
# Optional: system NSS DB (RHEL/Fedora)
sudo certutil -d sql:/etc/pki/nssdb -D -n "P2NS CA" 2>/dev/null || true
sudo certutil -d sql:/etc/pki/nssdb -A -t "C,," -n "P2NS CA" -i ./certs/ca.cert.pem
```
Restart Chrome after installing. See [docs/CERTIFICATES.md](CERTIFICATES.md) for verification commands.
**Windows:**
1. Open Command Prompt as Administrator.
2. Navigate to your P2NS directory:
```cmd
cd C:\path\to\p2ns
```
3. Install the certificate:
```cmd
certutil -addstore -f "ROOT" certs\ca.cert.pem
```
#### Browser-Level Installation
Even after OS-level installation, some browsers maintain their own certificate stores and may require separate installation:
**Chrome/Edge (Chromium-based):**
- Chrome and Edge use the OS certificate store on macOS and Windows, so OS-level installation should be sufficient.
- On Linux, use the **NSS `certutil` commands** in the Linux section above (`~/.pki/nssdb`). GUI import via `chrome://settings/security` → **Manage certificates** → **Authorities** is an alternative if `certutil` is unavailable.
**Firefox:**
Firefox uses its own certificate store and requires separate installation:
1. Open Firefox preferences (`about:preferences#privacy` or `about:preferences#security`).
2. Scroll down to **Certificates** section.
3. Click **View Certificates**.
4. Go to **Authorities** tab.
5. Click **Import...**.
6. Navigate to `./certs/ca.cert.pem` and select it.
7. Check **Trust this CA to identify websites** and click **OK**.
**Safari (macOS):**
- Safari uses the macOS System Keychain, so OS-level installation should be sufficient.
- If issues persist, verify the certificate is trusted in Keychain Access (see macOS instructions above).
**Verification:**
After installation, verify the certificate is trusted by accessing `https://p2ns.admin` or `https://peer.directory` in your browser. You should see a valid TLS connection without certificate warnings.
## Logging and Debugging
Logging uses leveled, prefixed messages (`[Main]`, `[DNS]`, `[Internal Proxy]`, `[HTTP Proxy]`, etc.). Set `LOG_LEVEL` in `.env` (0=DEBUG … 3=ERROR).
### Log files (`logs/`)
| File | Prefix / source | Console |
|------|-----------------|--------|
| `core.log` | Most subsystems (Swarm, Admin, Main, …) | Yes |
| `proxy.log` | `[Internal Proxy]` HTTPS plugin proxy | No |
| `http-proxy.log` | `[HTTP Proxy]` HTTP→HTTPS redirect server | No |
| `dns.log` | `[DNS]` | No |
| `plugins.log` | `Plugin:*`, PluginSDK, PluginChannels | No |
| `holesail.log` | `[Holesail]` | No |
Override directory with `LOG_DIR`. In-memory tail size per channel: `LOG_BUFFER_LINES` (default `2000`).
### Admin Logs tab
- Choose a log file from the dropdown.
- **Filter** lines (debounced substring or `/regex/flags`).
- Live tail via WebSocket (`subscribe-log` / `file-log` messages), not the legacy console stream.
Plugin `sdk.log.*` messages still appear in per-plugin terminals (`plugin-log` WebSocket) and `plugin-sites/{domain}/app.log`.
Example core line:
```
2026-05-28T12:00:00.000Z [INFO] [Main] Starting main function...
```
## Configuration Options
### Environment Variables
Configure via `.env` (copy from `default.env`):
- **STORAGE_DIR**: Corestore data directory (default: `./my-storage`).
- **DOMAINS_FILE**: Domains JSON file (default: `./cache/domains.json`).
- **LOCAL_DNS_FILE**: Local DNS records file (default: `cache/local_dns.json`).
- **HOLESAIL_SERVERS_FILE**: Holesail servers file (default: `./cache/holesail_servers.json`).
- **HOLESAIL_CLIENTS_FILE**: Holesail clients file (default: `./cache/holesail_clients.json`).
- **SELECTOR_CACHE_FILE**: DNS version preferences and hash preferences file (default: `./cache/selector_cache.json`).
- **SUBSCRIPTIONS_FILE**: Path to subscriptions storage file (default: `./cache/subscriptions.json`).
- **PEER_HISTORY_FILE**: Path to peer history storage file (default: `./cache/peer_history.json`).
- **PEER_METRICS_FILE**: Path to peer metrics storage file (default: `./cache/peer_metrics.json`).
- **TOPIC_SEED**: Hyperswarm topic seed (default: `p2ns-dns`).
- **CERTS_DIR**: Certificate storage directory (default: `./certs`).
- **INTERNAL_PORT**: Holesail client port (default: `8080`).
- **SUBNET_BASE**: Virtual interface IP base (default: `192.168.3`). Deprecated in favor of `SUBNETS` array.
- **INITIAL_IP_INDEX**: Starting IP index (default: `2`). Deprecated in favor of `SUBNETS` array.
- **SUBNETS**: JSON array of subnet configurations with `base`, `cidr`, `startIndex`, and optional `name` (default: single subnet from `SUBNET_BASE`). Example: `[{"base":"192.168.3.0","cidr":24,"startIndex":2,"name":"Primary Subnet"}]`.
- **SUBNET_NAME**: Interface for virtual IPs (default: `lo0` on macOS, `lo` on Linux, auto-detected on Windows). Can be configured via the admin interface Settings tab, which provides a dropdown list of all available network interfaces on the system. **Note**: On macOS, if "Stealth Mode" is enabled in firewall settings, ICMP responses for virtual interfaces may be blocked.
- **PUBLIC_DNS_SERVER**: Public DNS fallback server(s). Supports comma-separated list for failover (default: `1.1.1.1`). Example: `1.1.1.1,8.8.8.8,9.9.9.9`. The system will try each server in order until one succeeds. Can be managed via the admin interface Settings tab.
- **HOLESAIL_TIMEOUT**: Holesail client timeout in minutes (default: `5` minutes).
- **INTERNAL_PROXY_TIMEOUT_MS**: Timeout for internal HTTP proxy upstream requests to local Holesail endpoints (default: `12000` ms). Prevents long hangs during tunnel failures.
- **PORT_CHECK_TIMEOUT**: Port check timeout in seconds (default: `2` seconds).
- **LOG_LEVEL**: Logging level (0-3, default: `0`).
- **LOG_DIR**: Directory for split log files (default: `./logs`).
- **LOG_BUFFER_LINES**: In-memory tail lines per log channel for admin API/WS (default: `2000`).
- **Internal Domains**: Automatically discovered from `plugin-sites/{domain}/config.json` files. `p2ns.admin` is always internal.
- **DNS_PORT**: DNS server port (default: `53`).
- **HTTPS_PORT**: HTTPS proxy port (default: `443`).
- **HTTP_PORT**: HTTP redirect port (default: `80`).
- **DISABLE_DNS_SERVER**: Disable DNS server (default: `false`).
- **DISABLE_PROXY_SERVER**: Disable proxy servers (default: `false`).
- **DISABLE_VIRTUAL_INTERFACES**: Disable virtual interface creation (default: `false`).
- **ALLOW_ANY_WRITER_INVITES**: Allow joiners to issue invites (default: `true`).
- **CONSENSUS_QUORUM_THRESHOLD**: Percentage of active peers that must vote to meet quorum (0.0-1.0, default: `0.5`). For example, `0.5` means 50% of peers must vote.
- **CONSENSUS_MIN_VOTES**: Minimum number of votes required regardless of peer count (default: `2`). Ensures quorum even in small networks.
- **CONSENSUS_TIE_BREAKER**: Strategy for breaking ties between claimants with equal votes (default: `timestamp`). Options: `timestamp` (prefer oldest claim), `claimant_age` (prefer local writer among tied claimants, else lexicographic), `lexicographic` (prefer local writer, else alphabetical).
- **CONSENSUS_VOTE_VALIDATION**: Whether to validate that votes reference existing claims (default: `true`). Invalid votes are ignored when enabled.
- **RESOURCE_VALIDATION_INTERVAL**: Resource validation interval in minutes (default: `5`). The system periodically validates and cleans up stale resources like closed connections, inactive servers, and orphaned timeouts.
- **CONSENSUS_IMMEDIATE_UPDATE**: Whether to trigger consensus recalculation immediately when claims or votes change (default: `true`). When `false`, waits for next periodic check.
- **FULL_PERSISTENCE**: Keep Holesail connections persistent indefinitely (default: `false`).
- **BACKUP_DIR**: Backup storage directory (default: `./backups`).
- **BACKUP_RETENTION**: Number of backups to keep before rotation (default: `25`).
- **BACKUP_INTERVAL**: Automatic backup interval in minutes (default: `720` = 12 hours).
- **METRICS_RETENTION_MS**: Metrics data retention period in minutes (default: `60` = 1 hour).
- **METRICS_SAMPLING_RATE**: Metrics sampling rate (0.0-1.0, default: `1.0` for all samples).
- **METRICS_AGGREGATION_INTERVAL**: Metrics aggregation interval in seconds (default: `60` = 1 minute).
- **METRICS_MAX_BUFFER_SIZE**: Maximum number of metric samples to buffer (default: `1000`).
- **DISABLE_AUTO_SUBSCRIPTION**: Disable automatic starting of Holesail clients for subscribed services on bootup or when new services are detected (default: `false`).
#### Master Node Settings
These settings only apply when running with the `--master` flag:
- **MASTER_RECONNECT_INTERVAL**: Base interval in seconds for reconnection attempts when peers disconnect (default: `5`). Master nodes will attempt to reconnect to disconnected peers with exponential backoff. The delay between attempts increases exponentially (5s, 10s, 20s, etc.) up to a maximum of 60 seconds.
- **MASTER_MAX_RECONNECT_ATTEMPTS**: Maximum number of reconnection attempts per peer before giving up (default: `10`). After this limit is reached, the peer is removed from the reconnection tracking set.
- **MASTER_PROACTIVE_INVITE_DELAY**: Delay in milliseconds before sending proactive invite to new peers (default: `500`). Master nodes automatically send invites to peers when they connect, ensuring new peers can join the network immediately without needing to request an invite.
**Master Node Behavior:**
- **Proactive Invite Sending**: Master nodes automatically send invites to peers when they connect, eliminating the need for peers to request invites.
- **Automatic Reconnection**: When a peer disconnects, the master node actively attempts to reconnect using Hyperswarm's peer discovery mechanisms.
- **Reconnection Tracking**: Master nodes track disconnected peers and attempt reconnection with exponential backoff to avoid overwhelming the network.
- **Invite After Reconnection**: When a peer successfully reconnects, the master node automatically sends an invite again to ensure the peer can rejoin the network.
Example `.env`:
```env
LOG_LEVEL=1
STORAGE_DIR=./my-storage
# Internal domains are automatically discovered from plugin-sites/{domain}/config.json
# p2ns.admin is always treated as an internal domain
SUBNETS=[{"base":"192.168.3.0","cidr":24,"startIndex":2,"name":"Primary Subnet"}]
SELECTOR_CACHE_FILE=cache/selector_cache.json
BACKUP_INTERVAL=720
METRICS_RETENTION_MS=60
```
## Backup and Recovery
P2NS includes an automatic backup system that periodically saves critical configuration files and data. Backups are stored in the directory specified by `BACKUP_DIR` (default: `./backups`).
### Automatic Backups
- Backups run automatically at intervals specified by `BACKUP_INTERVAL` in minutes (default: 720 minutes = 12 hours).
- The system maintains up to `BACKUP_RETENTION` backups (default: 25), automatically rotating older backups.
- Each backup includes:
- Configuration files (default locations: `cache/domains.json`, `cache/local_dns.json`, `cache/holesail_servers.json`, `cache/holesail_clients.json`, `cache/selector_cache.json`, `cache/subscriptions.json`, `cache/peer_history.json`, `cache/peer_metrics.json`)
- Certificate directory (`certs/` or `CERTS_DIR`)
- Cache directory (`cache/`)
- Metadata including timestamp, file list, and sizes
### Manual Backups
Create manual backups via:
- **Admin Interface**: Use the "Backups" tab to create, restore, view metadata, or delete backups.
- **API**: `POST /api/backups/create` endpoint.
### Restoring Backups
Restore backups via:
- **Admin Interface**: Select a backup from the list and click "Restore".
- **API**: `POST /api/backups/restore` with the backup name.
Restoration replaces current configuration files and certificates with the backup contents. The system should be restarted after restoration for changes to take full effect.
## Health Checks and Diagnostics
P2NS provides comprehensive health monitoring and diagnostic tools accessible via the admin interface and API.
### Health Endpoints
- **Liveness Probe**: `GET /api/health` - Checks if the process is alive and basic services are running.
- **Readiness Probe**: `GET /api/health?probe=readiness` - Checks if all services are ready to accept traffic.
Health status includes:
- Service status (DNS, Proxy, Swarm)
- Dependency health (Corestore, Hyperswarm)
- Connected peers count
- System uptime
### Diagnostic Tools
Available via the "Diagnostics" tab in the admin interface or API endpoints:
- **DNS Lookup** (`POST /api/diagnostics/dns-lookup`): Resolve DNS records (A, AAAA, MX, TXT, NS, CNAME, SRV, PTR, SOA).
- **Ping** (`POST /api/diagnostics/ping`): Test network connectivity with configurable packet count. Supports streaming output for real-time results.
- **Traceroute** (`POST /api/diagnostics/traceroute`): Trace network path to a target. Supports streaming output.
- **Connection Test** (`POST /api/diagnostics/connection-test`): Test TCP connectivity to a domain and port.
- **Bandwidth** (`GET /api/diagnostics/bandwidth`): View network interface information and configuration.
All diagnostic tools support both streaming (real-time) and non-streaming modes, with detailed error reporting and response time metrics.
## Resource Validation and Cleanup
P2NS includes an automatic resource validation system that periodically checks and cleans up stale resources to maintain system health and prevent resource leaks.
### Resource Validation System
The resource validation system runs at configurable intervals (default: 5 minutes, set via `RESOURCE_VALIDATION_INTERVAL`) and validates:
- **Holesail Connections**: Detects and removes stale Holesail server/client connections
- **TLS/HTTP Servers**: Identifies and closes inactive TLS and HTTP server instances
- **Plugin RPC**: Validates protomux-rpc plugin protocols and recreates stale peer RPC sessions
- **Timeout Handles**: Cleans up orphaned timeout handles that are no longer needed
- **State Maps**: Ensures state maps (`holesails`, `tlsServers`, `httpServers`, `holesailClientTimeouts`) are consistent
### What Gets Validated
1. **Holesail Servers/Clients**: Checks if processes are still running and connections are active
2. **TLS/HTTP Servers**: Verifies servers are still listening and haven't been closed unexpectedly
3. **Plugin RPC**: Validates plugin RPC sessions and recreates closed muxes when peers are still connected
4. **Timeouts**: Removes timeout handles for connections that no longer exist
### Configuration
- **RESOURCE_VALIDATION_INTERVAL**: Interval in minutes between validation runs (default: `5`)
- Validation runs automatically in the background
- No manual intervention required
- Logs validation actions for debugging
### Benefits
- **Prevents Resource Leaks**: Automatically cleans up stale resources
- **Maintains System Health**: Ensures state maps stay consistent
- **Improves Performance**: Removes unused connections and servers
- **Reduces Memory Usage**: Cleans up orphaned objects and handles
## System Metrics and Monitoring
P2NS collects comprehensive system metrics for monitoring and troubleshooting.
### Metrics Collection
- **Real-time Metrics**: Current system state including request statistics, Holesail connections, process metrics (CPU, memory), and peer information.
- **Historical Metrics**: Time-series data with configurable retention (`METRICS_RETENTION_MS`, default: 1 hour).
- **Sampling**: Configurable sampling rate (`METRICS_SAMPLING_RATE`, default: 1.0 for all samples).
- **Aggregation**: Automatic aggregation at intervals (`METRICS_AGGREGATION_INTERVAL`, default: 1 minute).
### Metrics Endpoints
- **Current Stats**: `GET /api/stats` - One-shot snapshot (used for initial Stats tab load).
- **Historical Data**: `GET /api/stats/historical?minutes=N` - Historical metrics (1-1440 minutes).
- **Live updates**: Admin Stats tab subscribes over WebSocket with `subscribe-stats` and receives `stats-snapshot` payloads (includes `stats`, `historical`, `health`, `status`, **Core** RPC invite diagnostics, and **Plugin RPC** protocol stats). `update-stats` notifies subscribers only; it does not replace the snapshot stream.
### Metrics Configuration
Configure metrics behavior via environment variables:
- `METRICS_RETENTION_MS`: How long to keep historical data
- `METRICS_SAMPLING_RATE`: Fraction of requests to sample (0.0-1.0)
- `METRICS_AGGREGATION_INTERVAL`: Aggregation frequency
- `METRICS_MAX_BUFFER_SIZE`: Maximum samples to buffer
View metrics in the admin **Stats** tab (charts + **Core** / **Plugin RPC** sections). Open the **Diagnostics** tab or `GET /api/diagnostics/invites` for detailed invite RPC diagnostics.
## Subnet Configuration
P2NS supports flexible subnet configuration for IP allocation to domains. You can configure multiple subnets with CIDR notation for better IP management.
### Single Subnet (Legacy)
The legacy `SUBNET_BASE` and `INITIAL_IP_INDEX` variables configure a single subnet:
- `SUBNET_BASE`: Base IP address (e.g., `192.168.3`)
- `INITIAL_IP_INDEX`: Starting IP index (e.g., `2` for `192.168.3.2`)
### Multi-Subnet Configuration
Use the `SUBNETS` environment variable (JSON array) for advanced subnet configuration:
```json
[
{
"base": "192.168.3.0",
"cidr": 24,
"startIndex": 2,
"name": "Primary Subnet"
},
{
"base": "10.0.0.0",
"cidr": 24,
"startIndex": 2,
"name": "Secondary Subnet"
}
]
```
Each subnet object contains:
- **base**: Network base IP address (e.g., `192.168.3.0`)
- **cidr**: CIDR notation (1-32, typically 24 for `/24` networks)
- **startIndex**: First usable IP index (1-254, typically 2 to avoid gateway)
- **name**: Optional subnet name/description
### Subnet Management
- **Admin Interface**: Use the "Settings" tab's subnet configurator to view and manage subnets.
- **API**:
- `GET /api/subnets` - List all configured subnets with capacity information
- `POST /api/subnets` - Update subnet configuration (requires restart)
The system automatically assigns IPs from available subnets, tracking usage and capacity. Subnet changes require a system restart to take full effect.
## Troubleshooting
- **Port Conflicts**: Use `netstat -an | grep LISTEN` or `lsof -i :53` to check ports. Set `DISABLE_DNS_SERVER` or `DISABLE_PROXY_SERVER` to `true`.
- **CA Installation**: Verify in Keychain Access (macOS), `/usr/local/share/ca-certificates` and `certutil -d sql:$HOME/.pki/nssdb -L` (Linux Chrome), or Windows ROOT store. Import manually if needed.
- **Holesail Failures**: Test hashes with `holesail --live 80 --public`. Ensure CLI is installed.
- **DNS Errors**: Check logs in admin interface or console. Verify peer connections, `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`), and `cache/local_dns.json` (or `LOCAL_DNS_FILE`) for conflicting domains and custom records.
- **Interface Issues**: Requires sudo; check with `ifconfig lo0` or `ip addr show lo`.
- **ICMP/Ping Not Working on macOS**: If virtual interfaces don't respond to ping, check if "Stealth Mode" is enabled in System Settings > Network > Firewall Options. Stealth mode blocks ICMP responses. Disable it or configure firewall to allow ICMP if ping functionality is needed.
- **Admin Interface**: Ensure `p2ns.admin` resolves and WebSocket connects (`wss://p2ns.admin/ws`). Check DNS Conflict Selector table loads all cached domains and DNS records on startup.
- **Sync Issues**: Use `--clean` to reset storage. Monitor `[Swarm]` logs for peer events.
- **DNS Conflict Selector**: If domains or records don't appear in the table, verify `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) and `cache/local_dns.json` (or `LOCAL_DNS_FILE`). Check logs for errors in `/api/local-dns` or `/api/selector-cache`.
- **P2P Domain Conflicts**: If conflicting domains don't appear, check that you have local claims for domains where consensus resolved to other claimants. Verify `cache/selector_cache.json` has hashPreferences. Check logs for errors in `/api/p2p-domain-conflicts`.
## Security Considerations
- **P2P Exposure**: Hyperswarm uses public keys; join trusted networks only.
- **CA Security**: Protect `./certs` as root CA enables local MITM.
- **Consensus**: Voting prevents tampering, but monitor for claim disputes.
- **Sudo**: Run in isolated environments due to privileged access.
- **Dependencies**: Audit `node-forge`, `holesail`, and others for vulnerabilities.
- **Selector Cache and DNS Records**: Ensure `cache/selector_cache.json` (or `SELECTOR_CACHE_FILE`) and `cache/local_dns.json` (or `LOCAL_DNS_FILE`) have restricted permissions, as they control DNS routing and hash preferences.
## Recent Enhancements
- **Claim Ownership and Safety**: Strict ownership validation prevents unauthorized claim modifications. Each peer can only modify their own claims, with multi-layer verification before any remove operation. Domain additions check for existing claims and update instead of creating duplicates.
- **Conflict Claim Removal**: Separate message type (`removeConflictDomainClaim`) for removing conflict claims that preserves other peers' claims. Only resolved claimants trigger full cleanup that removes all claims.
- **Hash Preferences Cleanup**: Automatic validation and cleanup of hash preferences on startup and domain removal. Invalid preferences are automatically removed (e.g., when conflicts resolve or local claims are removed).
- **Automatic Consensus Updates**: Domain additions automatically trigger consensus recalculation and network-wide updates, ensuring conflict badges and UI reflect current state immediately.
- **Certificate Security**: Automatic expiration monitoring and renewal, certificate revocation list (CRL) support, improved certificate details modal UI
- **Backup and Recovery**: Automatic backups with rotation, restore functionality, scheduled backups
- **Enhanced Error Recovery**: Retry logic with exponential backoff, automatic reconnection, circuit breaker pattern, graceful degradation
- **Health Check Enhancements**: Detailed service health checks, readiness/liveness probes, dependency health monitoring
- **Metrics Optimization**: Metric aggregation, configurable sampling, retention periods, optimized memory usage
- **Windows Support**: Partial support exists for virtual interfaces (netsh) and CA installation (certutil); broader runtime support is still in progress.
- **API Documentation**: OpenAPI/Swagger specification, usage examples, troubleshooting guide
## Additional Documentation
For more detailed information on specific topics, see:
- **[Plugin System](docs/plugins/README.md)**: Complete guide to creating and managing plugins
- **[Plugin SDK Reference](docs/plugins/PLUGIN_SDK.md)**: Comprehensive API reference for plugin development
- **[REST API Documentation](docs/RESTAPI.md)**: Complete API endpoint reference including plugin management endpoints
- **[HyperDB Integration](docs/plugins/HYPERDB.md)**: Database operations for plugins
- **[Plugin RPC](docs/plugins/PLUGIN_CHANNELS.md)**: Plugin peer protocols via protomux-rpc
- **[Hyperdrive Integration](docs/plugins/HYPERDRIVE.md)**: Distributed file system for plugins
- **[Proxy Server Documentation](proxy-server/README.md)**: Standalone proxy server configuration and usage
- **[Test Scripts](test-scripts/README.md)**: Smoke tests and `npm run test:plugin-rpc` for plugin RPC contract
- **[Architecture RFCs](docs/rfcs/README.md)**: RFC 0001 (implemented Autobase consensus sidecar) plus proposed designs for discovery, packaging, and admin security
### Holepunch-Native Defaults
The following modernization features are enabled by default:
- Shared Corestore namespaces for plugin databases (`USE_SHARED_CORESTORE_NAMESPACES=true`)
- Plugin protomux-rpc (`ENABLE_PROTOMUX_RPC=true`; RPC-only plugin protocols — see [PLUGIN_CHANNELS.md](plugins/PLUGIN_CHANNELS.md))
- Hypercore/Corestore stats collection in `/api/stats` (`ENABLE_HYPERCORE_STATS=true`)
Each can be explicitly disabled by setting its flag to `false`.
## Contributing
Fork, branch, and submit PRs for bug fixes, features, or docs. Test on macOS/Linux. Focus on stability and cross-platform compatibility.
Reference in New Issue View Git Blame Copy Permalink