forked from snxraven/p2ns
EXPERIMENTAL: replace KV-scan consensus with Autobase sidecar engine
Cut over domain resolution to an Autobase apply-based sidecar fed by claim/vote dual-writes from dnsPass. Remove the legacy full-KV scan getConsensusState path and wire all reads through consensus-view. - Add consensus-resolver, consensus-events, consensus-apply, consensus-autobase, and consensus-view modules - Dual-append consensus events on claim/vote dnsPassAdd/Remove - Bootstrap sidecar from existing KV entries; persist consensusAutobaseKey in network manifest - Expose sidecar health via GET /api/consensus/status and metrics - Add consensus-resolver and consensus-apply unit tests - Expand RFC 0001 to Implemented; update CONSENSUS.md Rollback: deploy prior release; KV data unchanged, sidecar rebuilds on next startup.
This commit is contained in:
+12
-12
@@ -6,6 +6,8 @@ This document provides a deep dive into the P2NS consensus mechanism for domain
|
|||||||
|
|
||||||
P2NS uses a quorum-based consensus mechanism to resolve domain ownership disputes. When multiple peers claim the same domain, the network votes to determine the legitimate owner. This enables decentralized domain management without a central authority.
|
P2NS uses a quorum-based consensus mechanism to resolve domain ownership disputes. When multiple peers claim the same domain, the network votes to determine the legitimate owner. This enables decentralized domain management without a central authority.
|
||||||
|
|
||||||
|
Resolution is computed by an **Autobase apply-based consensus sidecar** ([RFC 0001](rfcs/0001-autobase-consensus.md)). Claim and vote KV writes in Autopass dual-append typed events to the sidecar; the apply handler replays events in linearized order and `getConsensusState` reads from that view.
|
||||||
|
|
||||||
**Key goals:**
|
**Key goals:**
|
||||||
- Prevent domain squatting through voting
|
- Prevent domain squatting through voting
|
||||||
- Handle network partitions gracefully
|
- Handle network partitions gracefully
|
||||||
@@ -108,7 +110,7 @@ Where:
|
|||||||
| Status | Description |
|
| Status | Description |
|
||||||
|--------|-------------|
|
|--------|-------------|
|
||||||
| `resolved` | Single winner determined by vote count or tie-breaker |
|
| `resolved` | Single winner determined by vote count or tie-breaker |
|
||||||
| `conflict` | Domain resolved to another claimant, but local peer has a competing claim |
|
| `conflict` | Domain resolved to another claimant, but local peer has a competing claim (computed in admin/UI layers, not returned by `getConsensusState`) |
|
||||||
| `insufficient_quorum` | Not enough votes to reach consensus |
|
| `insufficient_quorum` | Not enough votes to reach consensus |
|
||||||
| `tie` | Multiple claimants tied; resolved via tie-breaker |
|
| `tie` | Multiple claimants tied; resolved via tie-breaker |
|
||||||
| `no_claims` | No claims exist for this domain |
|
| `no_claims` | No claims exist for this domain |
|
||||||
@@ -169,23 +171,21 @@ P2NS automatically casts votes under certain conditions.
|
|||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
async function autoVoteForDomain(domain, entries) {
|
async function autoVoteForDomain(domain, entries) {
|
||||||
// Get claims for this domain
|
|
||||||
const claims = getClaimsForDomain(domain, entries);
|
const claims = getClaimsForDomain(domain, entries);
|
||||||
|
|
||||||
// Check if local peer already voted
|
|
||||||
if (hasLocalVote(domain, entries)) return;
|
|
||||||
|
|
||||||
// Vote for local claim if exists
|
|
||||||
if (claims.has(localWriter)) {
|
if (claims.has(localWriter)) {
|
||||||
await castVote(domain, localWriter);
|
await castVote(domain, localWriter);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Vote for claim with most existing votes
|
if (Object.keys(claims).length === 1) {
|
||||||
const winner = getLeadingClaimant(domain, entries);
|
await castVote(domain, Object.keys(claims)[0]);
|
||||||
if (winner) {
|
return;
|
||||||
await castVote(domain, winner);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Multiple claimants: use configured tie-breaker among claimants
|
||||||
|
const winner = applyTieBreaker(Object.keys(claims), claimTimestamps, localWriter);
|
||||||
|
await castVote(domain, winner);
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -1,40 +1,102 @@
|
|||||||
# RFC 0001: Autobase Consensus Evaluation
|
# RFC 0001: Autobase Consensus
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Draft
|
Implemented
|
||||||
|
|
||||||
## Problem
|
## Background
|
||||||
|
|
||||||
P2NS currently resolves domain ownership with custom vote/quorum logic on top of Autopass entries. This works, but it carries maintenance overhead and bespoke reconciliation behavior.
|
P2NS resolves domain ownership with quorum-based voting over claim and vote records stored in the Autopass KV ledger. The previous resolver scanned all KV entries on every query, which caused maintenance overhead, non-deterministic duplicate-vote handling (iteration order), and legacy timestamp instability.
|
||||||
|
|
||||||
## Proposal
|
## Goals
|
||||||
|
|
||||||
Evaluate an Autobase-backed consensus stream for claim and vote events, with deterministic apply logic for ownership resolution.
|
- Replace the KV-scan resolver with an Autobase apply-based consensus sidecar as the **sole** production read path.
|
||||||
|
- Deterministic event replay in Autobase linearized order.
|
||||||
## Scope
|
- Bootstrap replay from existing dnsPass claim/vote entries for network migration.
|
||||||
|
- Sidecar health diagnostics in admin API and domain.consensus plugin.
|
||||||
- Prototype an isolated consensus view for domain claims and votes.
|
|
||||||
- Compare convergence behavior with the current implementation.
|
|
||||||
- Keep existing Autopass paths as source-of-truth during evaluation.
|
|
||||||
|
|
||||||
## Non-Goals
|
## Non-Goals
|
||||||
|
|
||||||
- Immediate migration of production consensus logic.
|
- Changing claim/vote KV key format or plugin SDK method signatures.
|
||||||
- Changing plugin APIs in this phase.
|
- In-process legacy engine fallback or feature-flagged dual resolver.
|
||||||
|
- Removing Autopass KV as the durable write store for claims and votes.
|
||||||
|
|
||||||
## Migration Plan
|
## Architecture
|
||||||
|
|
||||||
1. Build sidecar Autobase view fed from current claim/vote entries.
|
```mermaid
|
||||||
2. Add parity checks against current resolver output.
|
flowchart TD
|
||||||
3. Gate optional read path via feature flag.
|
Writes[domains.js voteForDomain] --> Queue[dns-pass-queue.js]
|
||||||
4. Decide go/no-go after parity and performance testing.
|
Queue --> DnsPass[Autopass KV]
|
||||||
|
Queue --> Events[Consensus Autobase append]
|
||||||
|
Events --> Apply[consensus-apply.js]
|
||||||
|
Apply --> View[consensus-view.js]
|
||||||
|
View --> GetState[getConsensusState]
|
||||||
|
Resolver[consensus-resolver.js] --> View
|
||||||
|
```
|
||||||
|
|
||||||
## Risks
|
Autopass KV remains the durable store. Every local claim/vote mutation dual-writes a typed event to the consensus Autobase. The apply handler maintains per-domain state; `getConsensusState` reads from the view and runs the pure resolver.
|
||||||
|
|
||||||
- Reordering semantics may expose hidden assumptions in current handlers.
|
## Event Model
|
||||||
- Requires deterministic apply handlers and strong replay discipline.
|
|
||||||
|
| Type | Payload | Source KV |
|
||||||
|
|------|---------|-----------|
|
||||||
|
| `claim_upsert` | domain, claimant, hash, timestamp, ssl, clients | `claim:{domain}:{claimant}` add |
|
||||||
|
| `claim_remove` | domain, claimant | `claim:{domain}:{claimant}` remove |
|
||||||
|
| `vote_upsert` | domain, claimant, voter | `vote:{domain}:{claimant}:{voter}` add |
|
||||||
|
| `vote_remove` | domain, claimant, voter | `vote:{domain}:{claimant}:{voter}` remove |
|
||||||
|
|
||||||
|
Events are JSON-encoded buffers appended to the consensus Autobase.
|
||||||
|
|
||||||
|
## Apply Semantics
|
||||||
|
|
||||||
|
Events are processed in Autobase linearized order:
|
||||||
|
|
||||||
|
1. **claim_upsert** — set or replace claim for claimant on domain.
|
||||||
|
2. **claim_remove** — remove claim; drop votes for that claimant on the domain.
|
||||||
|
3. **vote_upsert** — set voter's vote (last event per voter wins).
|
||||||
|
4. **vote_remove** — remove voter's vote.
|
||||||
|
|
||||||
|
Resolution (`consensus-resolver.js`) runs at read time with the same quorum, tie-break, vote-validation, and single-local-claim rules as the prior implementation.
|
||||||
|
|
||||||
|
## Determinism Fixes
|
||||||
|
|
||||||
|
- Legacy hash-only claims use `timestamp: 0` (not runtime `Date.now()`).
|
||||||
|
- Duplicate votes resolved by event order in the Autobase stream, not KV list iteration order.
|
||||||
|
|
||||||
|
## Migration
|
||||||
|
|
||||||
|
1. Genesis or first peer with empty sidecar bootstraps by replaying dnsPass claim/vote entries sorted by `(type, domain, timestamp, claimant, voter)`.
|
||||||
|
2. Network manifest stores `consensusAutobaseKey` for joiners.
|
||||||
|
3. Joiners open the sidecar with the manifest bootstrap key and replicate via `consensusBase.replicate(connection)` on swarm connections.
|
||||||
|
4. Local claim/vote writes dual-append events after successful dnsPass operations.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Existing consensus env vars unchanged (`CONSENSUS_QUORUM_THRESHOLD`, `CONSENSUS_MIN_VOTES`, `CONSENSUS_TIE_BREAKER`, `CONSENSUS_VOTE_VALIDATION`).
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- Legacy KV-scan `getConsensusState` removed from codebase.
|
||||||
|
- `test-scripts/consensus-resolver.test.js` and `test-scripts/consensus-apply.test.js` pass in CI.
|
||||||
|
- Admin `GET /api/consensus/status` reports sidecar health.
|
||||||
|
- `docs/CONSENSUS.md` aligned with implementation.
|
||||||
|
|
||||||
## Rollback
|
## Rollback
|
||||||
|
|
||||||
Disable feature flag and keep existing consensus resolver.
|
Deploy the previous P2NS release. Autopass KV data is unchanged; upgrading again rebuilds the sidecar from bootstrap replay.
|
||||||
|
|
||||||
|
## Implementation Map
|
||||||
|
|
||||||
|
| Module | Path |
|
||||||
|
|--------|------|
|
||||||
|
| Pure resolver | `includes/core/consensus-resolver.js` |
|
||||||
|
| Event codec | `includes/core/consensus-events.js` |
|
||||||
|
| Apply handler | `includes/core/consensus-apply.js` |
|
||||||
|
| Autobase lifecycle | `includes/core/consensus-autobase.js` |
|
||||||
|
| Read API | `includes/core/consensus-view.js` |
|
||||||
|
| Dual-write | `includes/core/dns-pass-queue.js` |
|
||||||
|
| Manifest key | `includes/infrastructure/network-manifest.js` |
|
||||||
|
|
||||||
|
## Changelog
|
||||||
|
|
||||||
|
- **2026-05-30**: Production cutover — Autobase sidecar replaces KV-scan resolver; RFC expanded from evaluation draft to implemented spec.
|
||||||
|
|||||||
@@ -10,6 +10,25 @@ async function handleConsensusRoutes(req, res) {
|
|||||||
const method = req.method;
|
const method = req.method;
|
||||||
const url = new URL(req.url, `https://${req.headers.host}`);
|
const url = new URL(req.url, `https://${req.headers.host}`);
|
||||||
|
|
||||||
|
// GET /api/consensus/status - Sidecar health
|
||||||
|
if (method === 'GET' && urlPath === '/api/consensus/status') {
|
||||||
|
try {
|
||||||
|
trackRequest('/api/consensus/status', true);
|
||||||
|
const { getConsensusStatus } = require('../../../core/consensus-autobase');
|
||||||
|
const status = getConsensusStatus();
|
||||||
|
res.writeHead(200, { 'Content-Type': 'application/json' });
|
||||||
|
res.end(JSON.stringify(status));
|
||||||
|
return true;
|
||||||
|
} catch (err) {
|
||||||
|
logError('Consensus', `Failed to get consensus status: ${err.message}`);
|
||||||
|
trackRequest('/api/consensus/status', false);
|
||||||
|
const errorResponse = createErrorResponse(err, 500);
|
||||||
|
res.writeHead(errorResponse.statusCode, errorResponse.headers);
|
||||||
|
res.end(errorResponse.body);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// GET /api/consensus/metrics - Get consensus metrics
|
// GET /api/consensus/metrics - Get consensus metrics
|
||||||
// Check this BEFORE the domain route to avoid matching "metrics" as a domain
|
// Check this BEFORE the domain route to avoid matching "metrics" as a domain
|
||||||
if (method === 'GET' && urlPath === '/api/consensus/metrics') {
|
if (method === 'GET' && urlPath === '/api/consensus/metrics') {
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
/**
|
||||||
|
* Autobase apply handler and in-memory consensus view state.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const { decodeEvent, EVENT_TYPES } = require('./consensus-events');
|
||||||
|
|
||||||
|
function createEmptyViewState() {
|
||||||
|
return {
|
||||||
|
domains: new Map(),
|
||||||
|
eventCount: 0,
|
||||||
|
lastApplyAt: 0
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function getOrCreateDomain(view, domain) {
|
||||||
|
if (!view.domains.has(domain)) {
|
||||||
|
view.domains.set(domain, {
|
||||||
|
claims: new Map(),
|
||||||
|
voterVotes: new Map()
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return view.domains.get(domain);
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyEventToView(view, event) {
|
||||||
|
switch (event.type) {
|
||||||
|
case EVENT_TYPES.CLAIM_UPSERT: {
|
||||||
|
const domainState = getOrCreateDomain(view, event.domain);
|
||||||
|
domainState.claims.set(event.claimant, {
|
||||||
|
hash: event.hash,
|
||||||
|
timestamp: event.timestamp,
|
||||||
|
ssl: event.ssl === true,
|
||||||
|
clients: event.clients || []
|
||||||
|
});
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case EVENT_TYPES.CLAIM_REMOVE: {
|
||||||
|
const domainState = view.domains.get(event.domain);
|
||||||
|
if (!domainState) break;
|
||||||
|
domainState.claims.delete(event.claimant);
|
||||||
|
for (const [voter, claimant] of domainState.voterVotes) {
|
||||||
|
if (claimant === event.claimant) {
|
||||||
|
domainState.voterVotes.delete(voter);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (domainState.claims.size === 0 && domainState.voterVotes.size === 0) {
|
||||||
|
view.domains.delete(event.domain);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case EVENT_TYPES.VOTE_UPSERT: {
|
||||||
|
const domainState = getOrCreateDomain(view, event.domain);
|
||||||
|
domainState.voterVotes.set(event.voter, event.claimant);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case EVENT_TYPES.VOTE_REMOVE: {
|
||||||
|
const domainState = view.domains.get(event.domain);
|
||||||
|
if (!domainState) break;
|
||||||
|
domainState.voterVotes.delete(event.voter);
|
||||||
|
if (domainState.claims.size === 0 && domainState.voterVotes.size === 0) {
|
||||||
|
view.domains.delete(event.domain);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function openConsensusView() {
|
||||||
|
return createEmptyViewState();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function applyConsensusNodes(nodes, view, host) {
|
||||||
|
for (const node of nodes) {
|
||||||
|
let event;
|
||||||
|
try {
|
||||||
|
event = decodeEvent(node.value);
|
||||||
|
} catch (err) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (event.type === 'add_writer' && event.key && host && typeof host.addWriter === 'function') {
|
||||||
|
await host.addWriter(Buffer.from(event.key, 'hex'), { indexer: true });
|
||||||
|
view.eventCount++;
|
||||||
|
view.lastApplyAt = Date.now();
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
applyEventToView(view, event);
|
||||||
|
view.eventCount++;
|
||||||
|
view.lastApplyAt = Date.now();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
module.exports = {
|
||||||
|
createEmptyViewState,
|
||||||
|
getOrCreateDomain,
|
||||||
|
applyEventToView,
|
||||||
|
openConsensusView,
|
||||||
|
applyConsensusNodes
|
||||||
|
};
|
||||||
@@ -0,0 +1,271 @@
|
|||||||
|
/**
|
||||||
|
* Consensus Autobase sidecar lifecycle.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const Autobase = require('autobase');
|
||||||
|
const state = require('../infrastructure/state');
|
||||||
|
const { logDebug, logInfo, logWarn, logError } = require('../infrastructure/logger');
|
||||||
|
const { openConsensusView, applyConsensusNodes } = require('./consensus-apply');
|
||||||
|
const { encodeEvent, entriesToBootstrapEvents } = require('./consensus-events');
|
||||||
|
const { listAllEntries, waitForAutobaseIdle } = require('./dns-pass-queue');
|
||||||
|
|
||||||
|
let consensusBase = null;
|
||||||
|
let bootstrapComplete = false;
|
||||||
|
let initPromise = null;
|
||||||
|
|
||||||
|
function getConsensusBase() {
|
||||||
|
return consensusBase;
|
||||||
|
}
|
||||||
|
|
||||||
|
function isBootstrapComplete() {
|
||||||
|
return bootstrapComplete;
|
||||||
|
}
|
||||||
|
|
||||||
|
function createConsensusAutobase(store, bootstrapKey) {
|
||||||
|
const opts = {
|
||||||
|
open() {
|
||||||
|
return openConsensusView();
|
||||||
|
},
|
||||||
|
async apply(nodes, view, host) {
|
||||||
|
await applyConsensusNodes(nodes, view, host);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if (bootstrapKey) {
|
||||||
|
return new Autobase(store, bootstrapKey, opts);
|
||||||
|
}
|
||||||
|
return new Autobase(store, opts);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function appendConsensusEvent(event) {
|
||||||
|
if (!consensusBase || consensusBase.closed) {
|
||||||
|
throw new Error('consensus autobase not open');
|
||||||
|
}
|
||||||
|
await waitForAutobaseIdle({ base: consensusBase });
|
||||||
|
await consensusBase.append(encodeEvent(event));
|
||||||
|
await consensusBase.update();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function appendConsensusEvents(events) {
|
||||||
|
for (const event of events) {
|
||||||
|
await appendConsensusEvent(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function ensureLocalWriter() {
|
||||||
|
if (!consensusBase || consensusBase.closed) return;
|
||||||
|
await consensusBase.ready();
|
||||||
|
await consensusBase.update();
|
||||||
|
if (consensusBase.writable) return;
|
||||||
|
|
||||||
|
const localKey = consensusBase.local?.key;
|
||||||
|
if (!localKey) return;
|
||||||
|
|
||||||
|
try {
|
||||||
|
await appendConsensusEvent({ type: 'add_writer', key: localKey.toString('hex') });
|
||||||
|
await consensusBase.update();
|
||||||
|
} catch (err) {
|
||||||
|
logWarn('ConsensusAutobase', `Could not request writer access: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function bootstrapFromDnsPass(pass, options = {}) {
|
||||||
|
const { allowLocalBootstrap = true, syncWaitMs = 3000 } = options;
|
||||||
|
if (!consensusBase || bootstrapComplete) return;
|
||||||
|
await consensusBase.ready();
|
||||||
|
await consensusBase.update();
|
||||||
|
|
||||||
|
if (!allowLocalBootstrap && syncWaitMs > 0) {
|
||||||
|
const deadline = Date.now() + syncWaitMs;
|
||||||
|
while (Date.now() < deadline) {
|
||||||
|
await consensusBase.update();
|
||||||
|
if (consensusBase.view && consensusBase.view.eventCount > 0) break;
|
||||||
|
await sleep(200);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const view = consensusBase.view;
|
||||||
|
if (view && view.eventCount > 0) {
|
||||||
|
bootstrapComplete = true;
|
||||||
|
logInfo('ConsensusAutobase', `Sidecar already has ${view.eventCount} events, skipping bootstrap`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!allowLocalBootstrap) {
|
||||||
|
logInfo('ConsensusAutobase', 'Joiner sidecar empty after sync — waiting for replication before local bootstrap');
|
||||||
|
bootstrapComplete = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!pass) {
|
||||||
|
logWarn('ConsensusAutobase', 'dnsPass not available for bootstrap');
|
||||||
|
bootstrapComplete = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const entries = await listAllEntries(pass);
|
||||||
|
const events = entriesToBootstrapEvents(entries);
|
||||||
|
if (events.length === 0) {
|
||||||
|
bootstrapComplete = true;
|
||||||
|
logInfo('ConsensusAutobase', 'No claim/vote entries to bootstrap');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
logInfo('ConsensusAutobase', `Bootstrapping sidecar with ${events.length} events from dnsPass`);
|
||||||
|
await appendConsensusEvents(events);
|
||||||
|
bootstrapComplete = true;
|
||||||
|
logInfo('ConsensusAutobase', 'Bootstrap complete');
|
||||||
|
}
|
||||||
|
|
||||||
|
function sleep(ms) {
|
||||||
|
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {object} options
|
||||||
|
* @param {import('corestore')} options.store
|
||||||
|
* @param {string|null} options.bootstrapKey
|
||||||
|
* @param {import('autopass')} options.dnsPass
|
||||||
|
* @param {boolean} options.isGenesis
|
||||||
|
*/
|
||||||
|
async function initializeConsensusAutobase({ store, bootstrapKey, dnsPass, isGenesis }) {
|
||||||
|
if (!store || typeof store.get !== 'function') {
|
||||||
|
logWarn('ConsensusAutobase', 'Valid corestore required — skipping consensus sidecar init');
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (initPromise) return initPromise;
|
||||||
|
|
||||||
|
initPromise = (async () => {
|
||||||
|
if (consensusBase && !consensusBase.closed) {
|
||||||
|
return consensusBase;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
consensusBase = createConsensusAutobase(store, bootstrapKey || null);
|
||||||
|
state.consensusAutobase = consensusBase;
|
||||||
|
|
||||||
|
await consensusBase.ready();
|
||||||
|
await consensusBase.update();
|
||||||
|
|
||||||
|
if (!bootstrapKey && isGenesis) {
|
||||||
|
const keyHex = consensusBase.key.toString('hex');
|
||||||
|
logInfo('ConsensusAutobase', `Genesis consensus autobase created: ${keyHex.slice(0, 16)}...`);
|
||||||
|
}
|
||||||
|
|
||||||
|
await ensureLocalWriter();
|
||||||
|
await bootstrapFromDnsPass(dnsPass, {
|
||||||
|
allowLocalBootstrap: !bootstrapKey || isGenesis,
|
||||||
|
syncWaitMs: bootstrapKey ? 3000 : 0
|
||||||
|
});
|
||||||
|
|
||||||
|
consensusBase.on('update', () => {
|
||||||
|
try {
|
||||||
|
const { invalidateConsensusCache } = require('./consensus-view');
|
||||||
|
invalidateConsensusCache();
|
||||||
|
} catch (err) {
|
||||||
|
logDebug('ConsensusAutobase', `Cache invalidation on update: ${err.message}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
return consensusBase;
|
||||||
|
} catch (err) {
|
||||||
|
initPromise = null;
|
||||||
|
if (consensusBase && !consensusBase.closed) {
|
||||||
|
try {
|
||||||
|
await consensusBase.close();
|
||||||
|
} catch (closeErr) {
|
||||||
|
logDebug('ConsensusAutobase', `Close after failed init: ${closeErr.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
consensusBase = null;
|
||||||
|
state.consensusAutobase = null;
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
|
||||||
|
try {
|
||||||
|
return await initPromise;
|
||||||
|
} catch (err) {
|
||||||
|
initPromise = null;
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function replicateConsensus(connection) {
|
||||||
|
if (!consensusBase || consensusBase.closed || !connection) return;
|
||||||
|
try {
|
||||||
|
consensusBase.replicate(connection);
|
||||||
|
} catch (err) {
|
||||||
|
logDebug('ConsensusAutobase', `Replicate failed: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function closeConsensusAutobase() {
|
||||||
|
bootstrapComplete = false;
|
||||||
|
initPromise = null;
|
||||||
|
if (!consensusBase) return;
|
||||||
|
try {
|
||||||
|
if (!consensusBase.closed) {
|
||||||
|
await consensusBase.close();
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
logError('ConsensusAutobase', `Error closing consensus autobase: ${err.message}`);
|
||||||
|
} finally {
|
||||||
|
consensusBase = null;
|
||||||
|
state.consensusAutobase = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function getConsensusStatus() {
|
||||||
|
const base = consensusBase;
|
||||||
|
const view = base && !base.closed ? base.view : null;
|
||||||
|
return {
|
||||||
|
open: !!(base && !base.closed),
|
||||||
|
ready: !!(base && base.opened !== false && !base.closed),
|
||||||
|
writable: !!(base && base.writable),
|
||||||
|
bootstrapComplete,
|
||||||
|
eventCount: view ? view.eventCount : 0,
|
||||||
|
domainCount: view ? view.domains.size : 0,
|
||||||
|
lastApplyAt: view ? view.lastApplyAt : null,
|
||||||
|
indexedLength: base ? base.indexedLength : 0,
|
||||||
|
length: base ? base.length : 0,
|
||||||
|
key: base && base.key ? base.key.toString('hex') : null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function initConsensusForNetwork({ store, dnsPass, networkManifest, manifestPath }) {
|
||||||
|
const base = await initializeConsensusAutobase({
|
||||||
|
store,
|
||||||
|
bootstrapKey: networkManifest?.consensusAutobaseKey || null,
|
||||||
|
dnsPass,
|
||||||
|
isGenesis: !networkManifest?.consensusAutobaseKey
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!base) return null;
|
||||||
|
|
||||||
|
const keyHex = base.key ? base.key.toString('hex') : null;
|
||||||
|
if (keyHex && !networkManifest?.consensusAutobaseKey && manifestPath) {
|
||||||
|
const networkManifestModule = require('../infrastructure/network-manifest');
|
||||||
|
await networkManifestModule.updateManifestConsensusKey(manifestPath, keyHex);
|
||||||
|
if (state.networkManifest) {
|
||||||
|
state.networkManifest.consensusAutobaseKey = keyHex;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return base;
|
||||||
|
}
|
||||||
|
|
||||||
|
module.exports = {
|
||||||
|
getConsensusBase,
|
||||||
|
isBootstrapComplete,
|
||||||
|
initializeConsensusAutobase,
|
||||||
|
initConsensusForNetwork,
|
||||||
|
appendConsensusEvent,
|
||||||
|
appendConsensusEvents,
|
||||||
|
replicateConsensus,
|
||||||
|
closeConsensusAutobase,
|
||||||
|
getConsensusStatus,
|
||||||
|
bootstrapFromDnsPass,
|
||||||
|
ensureLocalWriter
|
||||||
|
};
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
/**
|
||||||
|
* Consensus event encoding and KV key translation.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const { parseClaimValue } = require('./consensus-resolver');
|
||||||
|
|
||||||
|
const EVENT_TYPES = {
|
||||||
|
CLAIM_UPSERT: 'claim_upsert',
|
||||||
|
CLAIM_REMOVE: 'claim_remove',
|
||||||
|
VOTE_UPSERT: 'vote_upsert',
|
||||||
|
VOTE_REMOVE: 'vote_remove'
|
||||||
|
};
|
||||||
|
|
||||||
|
function encodeEvent(event) {
|
||||||
|
return Buffer.from(JSON.stringify(event), 'utf8');
|
||||||
|
}
|
||||||
|
|
||||||
|
function decodeEvent(buffer) {
|
||||||
|
const raw = Buffer.isBuffer(buffer) ? buffer.toString('utf8') : String(buffer);
|
||||||
|
return JSON.parse(raw);
|
||||||
|
}
|
||||||
|
|
||||||
|
function isConsensusKvKey(key) {
|
||||||
|
return key.startsWith('claim:') || key.startsWith('vote:');
|
||||||
|
}
|
||||||
|
|
||||||
|
function kvMutationToEvent(key, value, isRemove) {
|
||||||
|
if (key.startsWith('claim:')) {
|
||||||
|
const parts = key.split(':');
|
||||||
|
if (parts.length !== 3) return null;
|
||||||
|
const domain = parts[1];
|
||||||
|
const claimant = parts[2];
|
||||||
|
if (isRemove) {
|
||||||
|
return { type: EVENT_TYPES.CLAIM_REMOVE, domain, claimant };
|
||||||
|
}
|
||||||
|
const parsed = parseClaimValue(value);
|
||||||
|
return {
|
||||||
|
type: EVENT_TYPES.CLAIM_UPSERT,
|
||||||
|
domain,
|
||||||
|
claimant,
|
||||||
|
hash: parsed.hash,
|
||||||
|
timestamp: parsed.timestamp,
|
||||||
|
ssl: parsed.ssl,
|
||||||
|
clients: parsed.clients || []
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (key.startsWith('vote:')) {
|
||||||
|
const parts = key.split(':');
|
||||||
|
if (parts.length !== 4) return null;
|
||||||
|
const domain = parts[1];
|
||||||
|
const claimant = parts[2];
|
||||||
|
const voter = parts[3];
|
||||||
|
if (isRemove) {
|
||||||
|
return { type: EVENT_TYPES.VOTE_REMOVE, domain, claimant, voter };
|
||||||
|
}
|
||||||
|
return { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant, voter };
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Deterministic bootstrap ordering for dnsPass KV entries.
|
||||||
|
* @param {Array<{ key: string, value: string }>} entries
|
||||||
|
* @returns {Array<object>}
|
||||||
|
*/
|
||||||
|
function entriesToBootstrapEvents(entries) {
|
||||||
|
const events = [];
|
||||||
|
for (const entry of entries) {
|
||||||
|
if (!isConsensusKvKey(entry.key)) continue;
|
||||||
|
const event = kvMutationToEvent(entry.key, entry.value, false);
|
||||||
|
if (event) events.push(event);
|
||||||
|
}
|
||||||
|
|
||||||
|
events.sort((a, b) => {
|
||||||
|
const typeOrder = (t) => {
|
||||||
|
if (t.startsWith('claim_')) return 0;
|
||||||
|
return 1;
|
||||||
|
};
|
||||||
|
const ta = typeOrder(a.type);
|
||||||
|
const tb = typeOrder(b.type);
|
||||||
|
if (ta !== tb) return ta - tb;
|
||||||
|
if (a.domain !== b.domain) return a.domain.localeCompare(b.domain);
|
||||||
|
const tsA = a.timestamp ?? 0;
|
||||||
|
const tsB = b.timestamp ?? 0;
|
||||||
|
if (tsA !== tsB) return tsA - tsB;
|
||||||
|
const ca = a.claimant || '';
|
||||||
|
const cb = b.claimant || '';
|
||||||
|
if (ca !== cb) return ca.localeCompare(cb);
|
||||||
|
const va = a.voter || '';
|
||||||
|
const vb = b.voter || '';
|
||||||
|
return va.localeCompare(vb);
|
||||||
|
});
|
||||||
|
|
||||||
|
return events;
|
||||||
|
}
|
||||||
|
|
||||||
|
module.exports = {
|
||||||
|
EVENT_TYPES,
|
||||||
|
encodeEvent,
|
||||||
|
decodeEvent,
|
||||||
|
isConsensusKvKey,
|
||||||
|
kvMutationToEvent,
|
||||||
|
entriesToBootstrapEvents
|
||||||
|
};
|
||||||
@@ -0,0 +1,181 @@
|
|||||||
|
/**
|
||||||
|
* Pure consensus resolution logic for domain ownership.
|
||||||
|
* Used by the Autobase apply view at read time.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const LEGACY_CLAIM_TIMESTAMP = 0;
|
||||||
|
|
||||||
|
function parseClaimValue(value) {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(value);
|
||||||
|
if (parsed.hash && parsed.timestamp !== undefined) {
|
||||||
|
return {
|
||||||
|
hash: parsed.hash,
|
||||||
|
clients: parsed.clients || [],
|
||||||
|
timestamp: parsed.timestamp,
|
||||||
|
ssl: parsed.ssl === true,
|
||||||
|
legacy: false
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (parsed.hash) {
|
||||||
|
return {
|
||||||
|
hash: parsed.hash,
|
||||||
|
clients: parsed.clients || [],
|
||||||
|
timestamp: parsed.timestamp || LEGACY_CLAIM_TIMESTAMP,
|
||||||
|
ssl: parsed.ssl === true,
|
||||||
|
legacy: false
|
||||||
|
};
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// legacy hash-only string
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
hash: value,
|
||||||
|
clients: [],
|
||||||
|
timestamp: LEGACY_CLAIM_TIMESTAMP,
|
||||||
|
ssl: false,
|
||||||
|
legacy: true
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function validateVote(claimant, claims, voteValidationEnabled) {
|
||||||
|
if (!voteValidationEnabled) return true;
|
||||||
|
return Object.prototype.hasOwnProperty.call(claims, claimant);
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyTieBreaker(candidates, claimTimestamps, localWriter, strategy) {
|
||||||
|
switch (strategy) {
|
||||||
|
case 'timestamp':
|
||||||
|
return candidates.sort((a, b) => {
|
||||||
|
const tsA = claimTimestamps[a] || 0;
|
||||||
|
const tsB = claimTimestamps[b] || 0;
|
||||||
|
return tsA - tsB;
|
||||||
|
})[0];
|
||||||
|
|
||||||
|
case 'claimant_age':
|
||||||
|
if (candidates.includes(localWriter)) return localWriter;
|
||||||
|
return candidates.sort((a, b) => a.localeCompare(b))[0];
|
||||||
|
|
||||||
|
case 'lexicographic':
|
||||||
|
default:
|
||||||
|
if (candidates.includes(localWriter)) return localWriter;
|
||||||
|
return candidates.sort((a, b) => a.localeCompare(b))[0];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {object} params
|
||||||
|
* @param {string} params.domain
|
||||||
|
* @param {Record<string, { hash: string, timestamp: number, ssl?: boolean, clients?: array }>} params.claims
|
||||||
|
* @param {Record<string, string>} params.voterVotes - voter -> claimant
|
||||||
|
* @param {object} params.config
|
||||||
|
* @param {string|null} params.localWriter
|
||||||
|
* @param {number} params.activePeers
|
||||||
|
*/
|
||||||
|
function resolveDomainConsensus({
|
||||||
|
domain,
|
||||||
|
claims,
|
||||||
|
voterVotes,
|
||||||
|
config,
|
||||||
|
localWriter,
|
||||||
|
activePeers
|
||||||
|
}) {
|
||||||
|
const claimTimestamps = {};
|
||||||
|
const claimHashes = {};
|
||||||
|
const voteCounts = {};
|
||||||
|
|
||||||
|
for (const [claimant, claim] of Object.entries(claims || {})) {
|
||||||
|
claimHashes[claimant] = claim.hash;
|
||||||
|
claimTimestamps[claimant] = claim.timestamp;
|
||||||
|
voteCounts[claimant] = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
const voters = new Set();
|
||||||
|
const voteValidation = config.CONSENSUS_VOTE_VALIDATION !== false;
|
||||||
|
|
||||||
|
for (const [voter, claimant] of Object.entries(voterVotes || {})) {
|
||||||
|
if (!validateVote(claimant, claimHashes, voteValidation)) continue;
|
||||||
|
voteCounts[claimant] = (voteCounts[claimant] || 0) + 1;
|
||||||
|
voters.add(voter);
|
||||||
|
}
|
||||||
|
|
||||||
|
const minVotes = Math.max(
|
||||||
|
config.CONSENSUS_MIN_VOTES || 2,
|
||||||
|
Math.ceil(activePeers * (config.CONSENSUS_QUORUM_THRESHOLD ?? 0.5))
|
||||||
|
);
|
||||||
|
const totalVotes = Object.values(voteCounts).reduce((sum, count) => sum + count, 0);
|
||||||
|
const quorumMet = totalVotes >= minVotes;
|
||||||
|
|
||||||
|
let status = 'no_claims';
|
||||||
|
let resolvedClaimant = null;
|
||||||
|
let hash = null;
|
||||||
|
|
||||||
|
const claimants = Object.keys(claimHashes);
|
||||||
|
|
||||||
|
if (claimants.length === 0) {
|
||||||
|
status = 'no_claims';
|
||||||
|
} else if (!quorumMet) {
|
||||||
|
const isSingleLocalClaim =
|
||||||
|
claimants.length === 1 && localWriter && Object.prototype.hasOwnProperty.call(claimHashes, localWriter);
|
||||||
|
const onlyWeVoted = totalVotes === 0 || (totalVotes === 1 && voters.has(localWriter));
|
||||||
|
|
||||||
|
if (isSingleLocalClaim && (totalVotes === 0 || onlyWeVoted)) {
|
||||||
|
status = 'resolved';
|
||||||
|
resolvedClaimant = localWriter;
|
||||||
|
hash = claimHashes[localWriter];
|
||||||
|
} else {
|
||||||
|
status = 'insufficient_quorum';
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
const maxVotes = Math.max(...Object.values(voteCounts));
|
||||||
|
const candidates = Object.keys(voteCounts).filter((c) => voteCounts[c] === maxVotes);
|
||||||
|
|
||||||
|
if (candidates.length === 1) {
|
||||||
|
status = 'resolved';
|
||||||
|
resolvedClaimant = candidates[0];
|
||||||
|
hash = claimHashes[resolvedClaimant];
|
||||||
|
} else {
|
||||||
|
status = 'tie';
|
||||||
|
const strategy = config.CONSENSUS_TIE_BREAKER || 'timestamp';
|
||||||
|
resolvedClaimant = applyTieBreaker(candidates, claimTimestamps, localWriter, strategy);
|
||||||
|
hash = claimHashes[resolvedClaimant];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
status,
|
||||||
|
hash,
|
||||||
|
resolvedClaimant,
|
||||||
|
voteCounts: { ...voteCounts },
|
||||||
|
activePeers,
|
||||||
|
quorumMet,
|
||||||
|
minVotes,
|
||||||
|
totalVotes,
|
||||||
|
lastResolution: Date.now()
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildDomainSnapshotFromView(viewState, domain) {
|
||||||
|
const domainData = viewState.domains.get(domain);
|
||||||
|
if (!domainData) {
|
||||||
|
return { claims: {}, voterVotes: {} };
|
||||||
|
}
|
||||||
|
const claims = {};
|
||||||
|
for (const [claimant, claim] of domainData.claims) {
|
||||||
|
claims[claimant] = { ...claim };
|
||||||
|
}
|
||||||
|
const voterVotes = {};
|
||||||
|
for (const [voter, claimant] of domainData.voterVotes) {
|
||||||
|
voterVotes[voter] = claimant;
|
||||||
|
}
|
||||||
|
return { claims, voterVotes };
|
||||||
|
}
|
||||||
|
|
||||||
|
module.exports = {
|
||||||
|
LEGACY_CLAIM_TIMESTAMP,
|
||||||
|
parseClaimValue,
|
||||||
|
validateVote,
|
||||||
|
applyTieBreaker,
|
||||||
|
resolveDomainConsensus,
|
||||||
|
buildDomainSnapshotFromView
|
||||||
|
};
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
/**
|
||||||
|
* Consensus read API backed by the Autobase sidecar view.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const state = require('../infrastructure/state');
|
||||||
|
const { validateConfig } = require('../infrastructure/config');
|
||||||
|
const { getPersistentPublicKey } = require('../infrastructure/utils');
|
||||||
|
const { logDebug, logInfo, logWarn } = require('../infrastructure/logger');
|
||||||
|
const { trackConsensusEvent } = require('../maintenance/metrics');
|
||||||
|
const {
|
||||||
|
resolveDomainConsensus,
|
||||||
|
buildDomainSnapshotFromView
|
||||||
|
} = require('./consensus-resolver');
|
||||||
|
const { getConsensusBase, getConsensusStatus, isBootstrapComplete } = require('./consensus-autobase');
|
||||||
|
|
||||||
|
function getActivePeerCount() {
|
||||||
|
const writers = state.networkWriterPeers;
|
||||||
|
if (writers && writers.size > 0) {
|
||||||
|
return writers.size + 1;
|
||||||
|
}
|
||||||
|
return (state.connectedPeers?.size || 0) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
const consensusStateCache = new Map();
|
||||||
|
const CONSENSUS_CACHE_TTL = 10000;
|
||||||
|
|
||||||
|
const consensusMetrics = {
|
||||||
|
resolutions: 0,
|
||||||
|
quorumFailures: 0,
|
||||||
|
ties: 0,
|
||||||
|
validationFailures: 0,
|
||||||
|
totalVotes: 0,
|
||||||
|
avgVotesPerDomain: 0,
|
||||||
|
domainResolutions: new Map()
|
||||||
|
};
|
||||||
|
|
||||||
|
const countedVotes = new Set();
|
||||||
|
const countedResolutions = new Set();
|
||||||
|
const countedQuorumFailures = new Set();
|
||||||
|
const countedTies = new Set();
|
||||||
|
|
||||||
|
let consensusConfig = null;
|
||||||
|
|
||||||
|
function getConsensusConfig() {
|
||||||
|
if (!consensusConfig) {
|
||||||
|
try {
|
||||||
|
consensusConfig = validateConfig();
|
||||||
|
} catch (err) {
|
||||||
|
consensusConfig = {
|
||||||
|
CONSENSUS_QUORUM_THRESHOLD: 0.5,
|
||||||
|
CONSENSUS_MIN_VOTES: 2,
|
||||||
|
CONSENSUS_TIE_BREAKER: 'timestamp',
|
||||||
|
CONSENSUS_VOTE_VALIDATION: true,
|
||||||
|
CONSENSUS_IMMEDIATE_UPDATE: true
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return consensusConfig;
|
||||||
|
}
|
||||||
|
|
||||||
|
function invalidateConsensusCache(domain) {
|
||||||
|
if (domain) {
|
||||||
|
consensusStateCache.delete(domain);
|
||||||
|
} else {
|
||||||
|
consensusStateCache.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function recordResolutionMetrics(domain, status, resolvedClaimant, config) {
|
||||||
|
if (status === 'resolved' && resolvedClaimant) {
|
||||||
|
const resolutionKey = `${domain}:${resolvedClaimant}`;
|
||||||
|
if (!countedResolutions.has(resolutionKey)) {
|
||||||
|
countedResolutions.add(resolutionKey);
|
||||||
|
consensusMetrics.resolutions++;
|
||||||
|
}
|
||||||
|
} else if (status === 'insufficient_quorum') {
|
||||||
|
if (!countedQuorumFailures.has(domain)) {
|
||||||
|
countedQuorumFailures.add(domain);
|
||||||
|
consensusMetrics.quorumFailures++;
|
||||||
|
}
|
||||||
|
} else if (status === 'tie' && resolvedClaimant) {
|
||||||
|
const tieKey = `${domain}:${resolvedClaimant}`;
|
||||||
|
if (!countedTies.has(tieKey)) {
|
||||||
|
countedTies.add(tieKey);
|
||||||
|
consensusMetrics.ties++;
|
||||||
|
}
|
||||||
|
if (trackConsensusEvent) {
|
||||||
|
trackConsensusEvent('tie', { domain });
|
||||||
|
}
|
||||||
|
logInfo('ConsensusView', `Tie resolved for ${domain} using ${config.CONSENSUS_TIE_BREAKER}: ${resolvedClaimant}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!consensusMetrics.domainResolutions.has(domain)) {
|
||||||
|
consensusMetrics.domainResolutions.set(domain, { resolved: 0, failed: 0 });
|
||||||
|
}
|
||||||
|
const domainMetrics = consensusMetrics.domainResolutions.get(domain);
|
||||||
|
if (status === 'resolved') {
|
||||||
|
domainMetrics.resolved++;
|
||||||
|
} else if (status !== 'no_claims') {
|
||||||
|
domainMetrics.failed++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function getConsensusState(domain) {
|
||||||
|
const pass = state.dnsPass;
|
||||||
|
if (!pass) {
|
||||||
|
return {
|
||||||
|
status: 'error',
|
||||||
|
error: 'dnsPass not initialized',
|
||||||
|
hash: null,
|
||||||
|
resolvedClaimant: null,
|
||||||
|
voteCounts: {},
|
||||||
|
activePeers: 0,
|
||||||
|
quorumMet: false,
|
||||||
|
lastResolution: null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const cached = consensusStateCache.get(domain);
|
||||||
|
if (cached && Date.now() - cached.timestamp < CONSENSUS_CACHE_TTL) {
|
||||||
|
return cached.state;
|
||||||
|
}
|
||||||
|
|
||||||
|
const base = getConsensusBase();
|
||||||
|
if (!base || base.closed) {
|
||||||
|
return {
|
||||||
|
status: 'error',
|
||||||
|
error: 'consensus sidecar not initialized',
|
||||||
|
hash: null,
|
||||||
|
resolvedClaimant: null,
|
||||||
|
voteCounts: {},
|
||||||
|
activePeers: 0,
|
||||||
|
quorumMet: false,
|
||||||
|
lastResolution: null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
await base.ready();
|
||||||
|
await base.update();
|
||||||
|
|
||||||
|
const localWriter = getPersistentPublicKey();
|
||||||
|
if (!localWriter) {
|
||||||
|
return {
|
||||||
|
status: 'error',
|
||||||
|
error: 'Persistent public key not available',
|
||||||
|
hash: null,
|
||||||
|
resolvedClaimant: null,
|
||||||
|
voteCounts: {},
|
||||||
|
activePeers: 0,
|
||||||
|
quorumMet: false,
|
||||||
|
lastResolution: null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const view = base.view;
|
||||||
|
if (!view) {
|
||||||
|
return {
|
||||||
|
status: 'error',
|
||||||
|
error: 'consensus view not available',
|
||||||
|
hash: null,
|
||||||
|
resolvedClaimant: null,
|
||||||
|
voteCounts: {},
|
||||||
|
activePeers: 0,
|
||||||
|
quorumMet: false,
|
||||||
|
lastResolution: null
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const snapshot = buildDomainSnapshotFromView(view, domain);
|
||||||
|
const config = getConsensusConfig();
|
||||||
|
const activePeers = getActivePeerCount();
|
||||||
|
|
||||||
|
const consensusState = resolveDomainConsensus({
|
||||||
|
domain,
|
||||||
|
claims: snapshot.claims,
|
||||||
|
voterVotes: snapshot.voterVotes,
|
||||||
|
config,
|
||||||
|
localWriter,
|
||||||
|
activePeers
|
||||||
|
});
|
||||||
|
|
||||||
|
recordResolutionMetrics(domain, consensusState.status, consensusState.resolvedClaimant, config);
|
||||||
|
|
||||||
|
for (const [claimant, count] of Object.entries(consensusState.voteCounts)) {
|
||||||
|
if (count <= 0) continue;
|
||||||
|
for (const voter of Object.keys(snapshot.voterVotes)) {
|
||||||
|
if (snapshot.voterVotes[voter] !== claimant) continue;
|
||||||
|
const voteKey = `${domain}:${claimant}:${voter}`;
|
||||||
|
if (!countedVotes.has(voteKey)) {
|
||||||
|
countedVotes.add(voteKey);
|
||||||
|
consensusMetrics.totalVotes++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const domainsWithVotes = consensusMetrics.domainResolutions.size;
|
||||||
|
if (domainsWithVotes > 0) {
|
||||||
|
consensusMetrics.avgVotesPerDomain = consensusMetrics.totalVotes / domainsWithVotes;
|
||||||
|
}
|
||||||
|
|
||||||
|
consensusStateCache.set(domain, { state: consensusState, timestamp: Date.now() });
|
||||||
|
return consensusState;
|
||||||
|
}
|
||||||
|
|
||||||
|
function getConsensusMetrics() {
|
||||||
|
const sidecar = getConsensusStatus();
|
||||||
|
return {
|
||||||
|
...consensusMetrics,
|
||||||
|
sidecar,
|
||||||
|
bootstrapComplete: isBootstrapComplete(),
|
||||||
|
domainResolutions: Array.from(consensusMetrics.domainResolutions.entries()).map(([d, metrics]) => ({
|
||||||
|
domain: d,
|
||||||
|
...metrics
|
||||||
|
}))
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function getAllDomainNamesFromView() {
|
||||||
|
const base = getConsensusBase();
|
||||||
|
if (!base || !base.view) return [];
|
||||||
|
return Array.from(base.view.domains.keys());
|
||||||
|
}
|
||||||
|
|
||||||
|
module.exports = {
|
||||||
|
getConsensusState,
|
||||||
|
getConsensusMetrics,
|
||||||
|
getConsensusConfig,
|
||||||
|
invalidateConsensusCache,
|
||||||
|
getAllDomainNamesFromView
|
||||||
|
};
|
||||||
@@ -106,6 +106,29 @@ function createCoreSwarmHandlers(ctx) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
setupListeners();
|
setupListeners();
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { initConsensusForNetwork } = require('./consensus-autobase');
|
||||||
|
const networkManifest = require('../infrastructure/network-manifest');
|
||||||
|
const manifestPath = networkManifest.getManifestPath(process.env.NETWORK_MANIFEST_FILE);
|
||||||
|
let manifest = state.networkManifest;
|
||||||
|
if (!manifest) {
|
||||||
|
manifest = await networkManifest.readManifest(manifestPath);
|
||||||
|
if (manifest) state.networkManifest = manifest;
|
||||||
|
}
|
||||||
|
const base = await initConsensusForNetwork({
|
||||||
|
store,
|
||||||
|
dnsPass: newPass,
|
||||||
|
networkManifest: manifest,
|
||||||
|
manifestPath
|
||||||
|
});
|
||||||
|
if (base) {
|
||||||
|
logInfo('Swarm', 'Consensus sidecar initialized after pairing');
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
logError('Swarm', `Failed to initialize consensus sidecar: ${err.message}`);
|
||||||
|
}
|
||||||
|
|
||||||
if (isMaster) {
|
if (isMaster) {
|
||||||
state.masterPendingPass = false;
|
state.masterPendingPass = false;
|
||||||
logInfo('Swarm', 'Secondary master paired — dnsPass ready, master invite policy active');
|
logInfo('Swarm', 'Secondary master paired — dnsPass ready, master invite policy active');
|
||||||
|
|||||||
+427
-770
File diff suppressed because it is too large
Load Diff
@@ -156,14 +156,6 @@ async function runSerialized(pass, fn) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
async function dnsPassAdd(pass, key, value, file) {
|
|
||||||
return runSerialized(pass, () => pass.add(key, value, file));
|
|
||||||
}
|
|
||||||
|
|
||||||
async function dnsPassRemove(pass, key) {
|
|
||||||
return runSerialized(pass, () => pass.remove(key));
|
|
||||||
}
|
|
||||||
|
|
||||||
async function dnsPassGet(pass, key) {
|
async function dnsPassGet(pass, key) {
|
||||||
return runSerialized(pass, () => pass.get(key));
|
return runSerialized(pass, () => pass.get(key));
|
||||||
}
|
}
|
||||||
@@ -182,10 +174,47 @@ async function listAllEntries(pass) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function appendConsensusEventForKvMutation(key, value, isRemove) {
|
||||||
|
try {
|
||||||
|
const { kvMutationToEvent, isConsensusKvKey } = require('./consensus-events');
|
||||||
|
const { appendConsensusEvent } = require('./consensus-autobase');
|
||||||
|
if (!isConsensusKvKey(key)) return;
|
||||||
|
const event = kvMutationToEvent(key, value, isRemove);
|
||||||
|
if (!event) return;
|
||||||
|
await appendConsensusEvent(event);
|
||||||
|
} catch (err) {
|
||||||
|
logWarn('DnsPassQueue', `Consensus event append failed for ${key}: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function dnsPassAdd(pass, key, value, file) {
|
||||||
|
return runSerialized(pass, async () => {
|
||||||
|
const result = await pass.add(key, value, file);
|
||||||
|
await appendConsensusEventForKvMutation(key, value, false);
|
||||||
|
return result;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function dnsPassRemove(pass, key) {
|
||||||
|
return runSerialized(pass, async () => {
|
||||||
|
let existingValue = null;
|
||||||
|
try {
|
||||||
|
existingValue = await pass.get(key);
|
||||||
|
} catch (err) {
|
||||||
|
// ignore lookup errors before remove
|
||||||
|
}
|
||||||
|
const result = await pass.remove(key);
|
||||||
|
const valueStr = existingValue != null ? existingValue.toString('utf8') : '';
|
||||||
|
await appendConsensusEventForKvMutation(key, valueStr, true);
|
||||||
|
return result;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
module.exports = {
|
module.exports = {
|
||||||
enqueueDnsPass,
|
enqueueDnsPass,
|
||||||
whenDnsPassIdle,
|
whenDnsPassIdle,
|
||||||
ensureDnsPassOpen,
|
ensureDnsPassOpen,
|
||||||
|
waitForAutobaseIdle,
|
||||||
syncDnsPassView,
|
syncDnsPassView,
|
||||||
isInviteCreationActive,
|
isInviteCreationActive,
|
||||||
isAtomicDnsPassError,
|
isAtomicDnsPassError,
|
||||||
|
|||||||
@@ -93,16 +93,30 @@ async function writeManifest(manifestPath, fields) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function adoptManifestFromPass(pass, topicSeed, manifestPath, options = {}) {
|
async function adoptManifestFromPass(pass, topicSeed, manifestPath, options = {}) {
|
||||||
const { genesisPublicKey, isGenesisRun = false } = options;
|
const { genesisPublicKey, isGenesisRun = false, consensusAutobaseKey = null } = options;
|
||||||
const identity = extractNetworkIdentity(pass, genesisPublicKey);
|
const identity = extractNetworkIdentity(pass, genesisPublicKey);
|
||||||
const doc = await writeManifest(manifestPath, {
|
const doc = await writeManifest(manifestPath, {
|
||||||
...identity,
|
...identity,
|
||||||
topicSeed: topicSeed || process.env.TOPIC_SEED || 'p2ns-dns',
|
topicSeed: topicSeed || process.env.TOPIC_SEED || 'p2ns-dns',
|
||||||
isGenesis: isGenesisRun
|
isGenesis: isGenesisRun,
|
||||||
|
...(consensusAutobaseKey ? { consensusAutobaseKey } : {})
|
||||||
});
|
});
|
||||||
return doc;
|
return doc;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function updateManifestConsensusKey(manifestPath, consensusAutobaseKey) {
|
||||||
|
const filePath = getManifestPath(manifestPath);
|
||||||
|
const existing = await readManifest(manifestPath);
|
||||||
|
if (!existing) return null;
|
||||||
|
const doc = {
|
||||||
|
...existing,
|
||||||
|
consensusAutobaseKey
|
||||||
|
};
|
||||||
|
await fs.writeFile(filePath, `${JSON.stringify(doc, null, 2)}\n`, 'utf8');
|
||||||
|
logInfo('NetworkManifest', `Updated manifest with consensusAutobaseKey: ${consensusAutobaseKey.slice(0, 16)}...`);
|
||||||
|
return doc;
|
||||||
|
}
|
||||||
|
|
||||||
function recordPeerNetworkStatus(peerId, status) {
|
function recordPeerNetworkStatus(peerId, status) {
|
||||||
if (!peerId || !status) return;
|
if (!peerId || !status) return;
|
||||||
const state = require('./state');
|
const state = require('./state');
|
||||||
@@ -163,6 +177,7 @@ module.exports = {
|
|||||||
readManifest,
|
readManifest,
|
||||||
writeManifest,
|
writeManifest,
|
||||||
adoptManifestFromPass,
|
adoptManifestFromPass,
|
||||||
|
updateManifestConsensusKey,
|
||||||
recordPeerNetworkStatus,
|
recordPeerNetworkStatus,
|
||||||
checkSplitBrain,
|
checkSplitBrain,
|
||||||
getLocalNetworkSummary
|
getLocalNetworkSummary
|
||||||
|
|||||||
@@ -42,6 +42,7 @@ const currentSubnetIndex = 0; // Start with first subnet for round-robin
|
|||||||
|
|
||||||
module.exports = {
|
module.exports = {
|
||||||
dnsPass: null,
|
dnsPass: null,
|
||||||
|
consensusAutobase: null,
|
||||||
corestore: null,
|
corestore: null,
|
||||||
hypercoreStats: null,
|
hypercoreStats: null,
|
||||||
/** Incremented while dns-pass-queue runs a serialized Autopass op (defer update handlers) */
|
/** Incremented while dns-pass-queue runs a serialized Autopass op (defer update handlers) */
|
||||||
|
|||||||
@@ -449,6 +449,12 @@ async function main() {
|
|||||||
function scheduleConnectionReplication(conn) {
|
function scheduleConnectionReplication(conn) {
|
||||||
if (!conn || conn.destroyed) return;
|
if (!conn || conn.destroyed) return;
|
||||||
logDebug('Swarm', 'Skipping p2ns-topic Autopass replication; Autopass swarm handles it');
|
logDebug('Swarm', 'Skipping p2ns-topic Autopass replication; Autopass swarm handles it');
|
||||||
|
try {
|
||||||
|
const { replicateConsensus } = require('./includes/core/consensus-autobase');
|
||||||
|
replicateConsensus(conn);
|
||||||
|
} catch (err) {
|
||||||
|
logDebug('Swarm', `Consensus sidecar replication: ${err.message}`);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Helper function to safely set dnsPass and ensure state consistency
|
// Helper function to safely set dnsPass and ensure state consistency
|
||||||
@@ -1556,6 +1562,21 @@ async function main() {
|
|||||||
logInfo('Main', 'Skipping domains.json watcher (secondary master; MASTER_LOAD_DOMAINS=false)');
|
logInfo('Main', 'Skipping domains.json watcher (secondary master; MASTER_LOAD_DOMAINS=false)');
|
||||||
}
|
}
|
||||||
setupListeners();
|
setupListeners();
|
||||||
|
|
||||||
|
try {
|
||||||
|
const { initConsensusForNetwork } = require('./includes/core/consensus-autobase');
|
||||||
|
const base = await initConsensusForNetwork({
|
||||||
|
store,
|
||||||
|
dnsPass: newPass,
|
||||||
|
networkManifest: state.networkManifest,
|
||||||
|
manifestPath
|
||||||
|
});
|
||||||
|
if (base) {
|
||||||
|
logInfo('Main', 'Consensus sidecar initialized');
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
logError('Main', `Failed to initialize consensus sidecar: ${err.message}`);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (isMaster) {
|
if (isMaster) {
|
||||||
@@ -2686,6 +2707,15 @@ async function main() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Close consensus sidecar before dnsPass
|
||||||
|
try {
|
||||||
|
const { closeConsensusAutobase } = require('./includes/core/consensus-autobase');
|
||||||
|
await closeConsensusAutobase();
|
||||||
|
logDebug('Main', 'Consensus sidecar closed');
|
||||||
|
} catch (err) {
|
||||||
|
logWarn('Main', `Error closing consensus sidecar: ${err.message}`);
|
||||||
|
}
|
||||||
|
|
||||||
// Close DNSPass after pairing operations (main component)
|
// Close DNSPass after pairing operations (main component)
|
||||||
if (state.dnsPass && typeof state.dnsPass.close === 'function') {
|
if (state.dnsPass && typeof state.dnsPass.close === 'function') {
|
||||||
logDebug('Main', 'Closing dnsPass (main component)...');
|
logDebug('Main', 'Closing dnsPass (main component)...');
|
||||||
|
|||||||
+2
-1
@@ -9,7 +9,8 @@
|
|||||||
"test:admin": "node test-scripts/admin-smoke.js",
|
"test:admin": "node test-scripts/admin-smoke.js",
|
||||||
"test:peer-paste": "node test-scripts/peer-paste-smoke.js",
|
"test:peer-paste": "node test-scripts/peer-paste-smoke.js",
|
||||||
"test:plugins": "node test-scripts/plugins-smoke.js",
|
"test:plugins": "node test-scripts/plugins-smoke.js",
|
||||||
"test:core-rpc": "node test-scripts/core-rpc-smoke.js && node test-scripts/network-manifest.test.js && node test-scripts/core-swarm-handlers.test.js && node test-scripts/multi-master.test.js && node test-scripts/core-invite-rpc-integration.js",
|
"test:core-rpc": "node test-scripts/core-rpc-smoke.js && node test-scripts/network-manifest.test.js && node test-scripts/core-swarm-handlers.test.js && node test-scripts/multi-master.test.js && node test-scripts/core-invite-rpc-integration.js && npm run test:consensus",
|
||||||
|
"test:consensus": "node test-scripts/consensus-resolver.test.js && node test-scripts/consensus-apply.test.js",
|
||||||
"test:multi-master": "node test-scripts/multi-master.test.js",
|
"test:multi-master": "node test-scripts/multi-master.test.js",
|
||||||
"test:plugin-rpc": "node test-scripts/plugin-channel-rpc.test.js",
|
"test:plugin-rpc": "node test-scripts/plugin-channel-rpc.test.js",
|
||||||
"audit": "npm audit --audit-level=high",
|
"audit": "npm audit --audit-level=high",
|
||||||
|
|||||||
@@ -146,6 +146,10 @@ async function handler(req, res) {
|
|||||||
totalVotes: metrics.totalVotes || 0,
|
totalVotes: metrics.totalVotes || 0,
|
||||||
avgVotesPerDomain: metrics.avgVotesPerDomain || 0
|
avgVotesPerDomain: metrics.avgVotesPerDomain || 0
|
||||||
},
|
},
|
||||||
|
sidecar: metrics.sidecar || {
|
||||||
|
open: false,
|
||||||
|
bootstrapComplete: false
|
||||||
|
},
|
||||||
activePeers: connectedPeers,
|
activePeers: connectedPeers,
|
||||||
timestamp: Date.now()
|
timestamp: Date.now()
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -495,14 +495,12 @@ async function getAllEntries(pass = state.dnsPass, useCache = true) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function getHashForDomain(domain) {
|
async function getHashForDomain(domain) {
|
||||||
// Check cache first
|
|
||||||
const cached = getCachedHash(domain);
|
const cached = getCachedHash(domain);
|
||||||
if (cached) {
|
if (cached) {
|
||||||
logDebug('Core', `Cache hit for domain: ${domain}`, { domain });
|
logDebug('Core', `Cache hit for domain: ${domain}`, { domain });
|
||||||
return cached;
|
return cached;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Try to use core consensus logic if available
|
|
||||||
try {
|
try {
|
||||||
const { getHashForDomain: coreGetHashForDomain } = require('../includes/core/core');
|
const { getHashForDomain: coreGetHashForDomain } = require('../includes/core/core');
|
||||||
const hash = await coreGetHashForDomain(domain);
|
const hash = await coreGetHashForDomain(domain);
|
||||||
@@ -511,12 +509,13 @@ async function getHashForDomain(domain) {
|
|||||||
return hash;
|
return hash;
|
||||||
}
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
logDebug('Core', `Core consensus not available, using fallback: ${err.message}`, { domain });
|
logDebug('Core', `Core consensus not available: ${err.message}`, { domain });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function getDomainSSLStatus(domain) {
|
async function getDomainSSLStatus(domain) {
|
||||||
// Try to use core consensus logic if available
|
|
||||||
try {
|
try {
|
||||||
const { getDomainSSLStatus: coreGetDomainSSLStatus } = require('../includes/core/core');
|
const { getDomainSSLStatus: coreGetDomainSSLStatus } = require('../includes/core/core');
|
||||||
return await coreGetDomainSSLStatus(domain);
|
return await coreGetDomainSSLStatus(domain);
|
||||||
@@ -524,75 +523,6 @@ async function getDomainSSLStatus(domain) {
|
|||||||
logDebug('Core', `Core SSL status not available, defaulting to false: ${err.message}`, { domain });
|
logDebug('Core', `Core SSL status not available, defaulting to false: ${err.message}`, { domain });
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Fallback to simplified logic if core not available
|
|
||||||
const pass = state.dnsPass;
|
|
||||||
if (!pass) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
await pass.ready();
|
|
||||||
logDebug('Core', `Querying hash for domain: ${domain}`, { domain });
|
|
||||||
|
|
||||||
const localWriter = getPersistentPublicKey();
|
|
||||||
if (!localWriter) {
|
|
||||||
logWarn('Core', 'Cannot query hash: persistent public key not available', { domain });
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
const allEntries = await getAllEntries(pass, true);
|
|
||||||
const claims = {};
|
|
||||||
const voteCounts = {};
|
|
||||||
|
|
||||||
for (const entry of allEntries) {
|
|
||||||
if (entry.key.startsWith(`claim:${domain}:`)) {
|
|
||||||
const claimant = entry.key.slice(`claim:${domain}:`.length);
|
|
||||||
// Parse claim value (may be JSON with timestamp or legacy format)
|
|
||||||
try {
|
|
||||||
const parsed = JSON.parse(entry.value);
|
|
||||||
claims[claimant] = parsed.hash || entry.value;
|
|
||||||
} catch (e) {
|
|
||||||
claims[claimant] = entry.value; // Legacy format
|
|
||||||
}
|
|
||||||
voteCounts[claimant] = 0;
|
|
||||||
}
|
|
||||||
if (entry.key.startsWith(`vote:${domain}:`)) {
|
|
||||||
const parts = entry.key.split(':');
|
|
||||||
if (parts.length === 4) {
|
|
||||||
const claimant = parts[2];
|
|
||||||
// Validate vote references existing claim
|
|
||||||
if (voteCounts[claimant] !== undefined) {
|
|
||||||
voteCounts[claimant]++;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if (Object.keys(claims).length === 0) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
const maxVotes = Math.max(...Object.values(voteCounts));
|
|
||||||
if (maxVotes === 0) {
|
|
||||||
if (Object.keys(claims).length === 1 && claims.hasOwnProperty(localWriter)) {
|
|
||||||
const hash = claims[localWriter];
|
|
||||||
setCachedHash(domain, hash);
|
|
||||||
return hash;
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
const candidates = Object.keys(voteCounts).filter(claimant => voteCounts[claimant] === maxVotes);
|
|
||||||
let resolvedClaimant;
|
|
||||||
if (candidates.length === 1) {
|
|
||||||
resolvedClaimant = candidates[0];
|
|
||||||
} else {
|
|
||||||
resolvedClaimant = candidates.includes(localWriter) ? localWriter : candidates.sort((a, b) => a.localeCompare(b))[0];
|
|
||||||
}
|
|
||||||
|
|
||||||
const hash = claims[resolvedClaimant];
|
|
||||||
logInfo('Core', `Resolved claimant for ${domain}`, { domain, claimant: resolvedClaimant });
|
|
||||||
setCachedHash(domain, hash);
|
|
||||||
return hash;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ============================================================================
|
// ============================================================================
|
||||||
|
|||||||
@@ -0,0 +1,133 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* Unit tests for consensus apply + bootstrap event ordering.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const assert = require('assert');
|
||||||
|
const { createEmptyViewState, applyEventToView } = require('../includes/core/consensus-apply');
|
||||||
|
const { EVENT_TYPES, entriesToBootstrapEvents } = require('../includes/core/consensus-events');
|
||||||
|
const {
|
||||||
|
resolveDomainConsensus,
|
||||||
|
buildDomainSnapshotFromView
|
||||||
|
} = require('../includes/core/consensus-resolver');
|
||||||
|
|
||||||
|
const CONFIG = {
|
||||||
|
CONSENSUS_QUORUM_THRESHOLD: 0.5,
|
||||||
|
CONSENSUS_MIN_VOTES: 2,
|
||||||
|
CONSENSUS_TIE_BREAKER: 'timestamp',
|
||||||
|
CONSENSUS_VOTE_VALIDATION: true
|
||||||
|
};
|
||||||
|
|
||||||
|
function replayEvents(events) {
|
||||||
|
const view = createEmptyViewState();
|
||||||
|
for (const event of events) {
|
||||||
|
applyEventToView(view, event);
|
||||||
|
}
|
||||||
|
return view;
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveFromView(view, domain, localWriter = 'local', activePeers = 4) {
|
||||||
|
const snapshot = buildDomainSnapshotFromView(view, domain);
|
||||||
|
return resolveDomainConsensus({
|
||||||
|
domain,
|
||||||
|
claims: snapshot.claims,
|
||||||
|
voterVotes: snapshot.voterVotes,
|
||||||
|
config: CONFIG,
|
||||||
|
localWriter,
|
||||||
|
activePeers
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function testBootstrapOrdering() {
|
||||||
|
const entries = [
|
||||||
|
{ key: 'vote:demo.test:peer-b:voter-1', value: '1' },
|
||||||
|
{ key: 'claim:demo.test:peer-b', value: JSON.stringify({ hash: 'hs://b', timestamp: 2000 }) },
|
||||||
|
{ key: 'claim:demo.test:peer-a', value: JSON.stringify({ hash: 'hs://a', timestamp: 1000 }) },
|
||||||
|
{ key: 'vote:demo.test:peer-a:voter-1', value: '1' },
|
||||||
|
{ key: 'vote:demo.test:peer-a:voter-2', value: '1' }
|
||||||
|
];
|
||||||
|
const events = entriesToBootstrapEvents(entries);
|
||||||
|
assert.strictEqual(events[0].type, EVENT_TYPES.CLAIM_UPSERT);
|
||||||
|
assert.strictEqual(events[0].claimant, 'peer-a');
|
||||||
|
assert.strictEqual(events[1].claimant, 'peer-b');
|
||||||
|
assert.ok(events[2].type.startsWith('vote_'));
|
||||||
|
}
|
||||||
|
|
||||||
|
function testClaimRemoveDropsVotes() {
|
||||||
|
const view = replayEvents([
|
||||||
|
{
|
||||||
|
type: EVENT_TYPES.CLAIM_UPSERT,
|
||||||
|
domain: 'x.test',
|
||||||
|
claimant: 'a',
|
||||||
|
hash: 'hs://a',
|
||||||
|
timestamp: 1,
|
||||||
|
ssl: false,
|
||||||
|
clients: []
|
||||||
|
},
|
||||||
|
{ type: EVENT_TYPES.VOTE_UPSERT, domain: 'x.test', claimant: 'a', voter: 'v1' },
|
||||||
|
{ type: EVENT_TYPES.CLAIM_REMOVE, domain: 'x.test', claimant: 'a' }
|
||||||
|
]);
|
||||||
|
assert.strictEqual(view.domains.has('x.test'), false);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testApplyMatchesResolver() {
|
||||||
|
const domain = 'apply.test';
|
||||||
|
const events = [
|
||||||
|
{
|
||||||
|
type: EVENT_TYPES.CLAIM_UPSERT,
|
||||||
|
domain,
|
||||||
|
claimant: 'peer-a',
|
||||||
|
hash: 'hs://a',
|
||||||
|
timestamp: 100,
|
||||||
|
ssl: false,
|
||||||
|
clients: []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: EVENT_TYPES.CLAIM_UPSERT,
|
||||||
|
domain,
|
||||||
|
claimant: 'peer-b',
|
||||||
|
hash: 'hs://b',
|
||||||
|
timestamp: 200,
|
||||||
|
ssl: false,
|
||||||
|
clients: []
|
||||||
|
},
|
||||||
|
{ type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: 'peer-a', voter: 'v1' },
|
||||||
|
{ type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: 'peer-a', voter: 'v2' },
|
||||||
|
{ type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: 'peer-a', voter: 'v3' }
|
||||||
|
];
|
||||||
|
const view = replayEvents(events);
|
||||||
|
const state = resolveFromView(view, domain, 'peer-z', 4);
|
||||||
|
assert.strictEqual(state.status, 'resolved');
|
||||||
|
assert.strictEqual(state.resolvedClaimant, 'peer-a');
|
||||||
|
assert.strictEqual(state.hash, 'hs://a');
|
||||||
|
}
|
||||||
|
|
||||||
|
function testKvBootstrapFixture() {
|
||||||
|
const entries = [
|
||||||
|
{
|
||||||
|
key: 'claim:fixture.test:alpha',
|
||||||
|
value: JSON.stringify({ hash: 'hs://alpha', timestamp: 10, ssl: false })
|
||||||
|
},
|
||||||
|
{
|
||||||
|
key: 'claim:fixture.test:beta',
|
||||||
|
value: JSON.stringify({ hash: 'hs://beta', timestamp: 20, ssl: false })
|
||||||
|
},
|
||||||
|
{ key: 'vote:fixture.test:alpha:voter-1', value: 'hs://alpha' },
|
||||||
|
{ key: 'vote:fixture.test:alpha:voter-2', value: 'hs://alpha' },
|
||||||
|
{ key: 'vote:fixture.test:alpha:voter-3', value: 'hs://alpha' }
|
||||||
|
];
|
||||||
|
const view = replayEvents(entriesToBootstrapEvents(entries));
|
||||||
|
const state = resolveFromView(view, 'fixture.test', 'gamma', 4);
|
||||||
|
assert.strictEqual(state.status, 'resolved');
|
||||||
|
assert.strictEqual(state.resolvedClaimant, 'alpha');
|
||||||
|
}
|
||||||
|
|
||||||
|
function run() {
|
||||||
|
testBootstrapOrdering();
|
||||||
|
testClaimRemoveDropsVotes();
|
||||||
|
testApplyMatchesResolver();
|
||||||
|
testKvBootstrapFixture();
|
||||||
|
console.log('consensus-apply.test.js: all passed');
|
||||||
|
}
|
||||||
|
|
||||||
|
run();
|
||||||
@@ -0,0 +1,183 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* Unit tests for consensus-resolver.js
|
||||||
|
*/
|
||||||
|
|
||||||
|
const assert = require('assert');
|
||||||
|
const {
|
||||||
|
parseClaimValue,
|
||||||
|
resolveDomainConsensus,
|
||||||
|
applyTieBreaker,
|
||||||
|
LEGACY_CLAIM_TIMESTAMP
|
||||||
|
} = require('../includes/core/consensus-resolver');
|
||||||
|
|
||||||
|
const DEFAULT_CONFIG = {
|
||||||
|
CONSENSUS_QUORUM_THRESHOLD: 0.5,
|
||||||
|
CONSENSUS_MIN_VOTES: 2,
|
||||||
|
CONSENSUS_TIE_BREAKER: 'timestamp',
|
||||||
|
CONSENSUS_VOTE_VALIDATION: true
|
||||||
|
};
|
||||||
|
|
||||||
|
const LOCAL = 'local-peer-key';
|
||||||
|
const REMOTE_A = 'remote-a-key';
|
||||||
|
const REMOTE_B = 'remote-b-key';
|
||||||
|
|
||||||
|
function resolve(domain, claims, voterVotes, overrides = {}) {
|
||||||
|
return resolveDomainConsensus({
|
||||||
|
domain,
|
||||||
|
claims,
|
||||||
|
voterVotes,
|
||||||
|
config: { ...DEFAULT_CONFIG, ...overrides.config },
|
||||||
|
localWriter: overrides.localWriter ?? LOCAL,
|
||||||
|
activePeers: overrides.activePeers ?? 4
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function testLegacyClaimTimestamp() {
|
||||||
|
const parsed = parseClaimValue('hs://legacy-hash');
|
||||||
|
assert.strictEqual(parsed.timestamp, LEGACY_CLAIM_TIMESTAMP);
|
||||||
|
assert.strictEqual(parsed.legacy, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testNoClaims() {
|
||||||
|
const state = resolve('example.test', {}, {});
|
||||||
|
assert.strictEqual(state.status, 'no_claims');
|
||||||
|
assert.strictEqual(state.hash, null);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testSingleLocalClaim() {
|
||||||
|
const state = resolve(
|
||||||
|
'solo.test',
|
||||||
|
{ [LOCAL]: { hash: 'hs://solo', timestamp: 1000 } },
|
||||||
|
{}
|
||||||
|
);
|
||||||
|
assert.strictEqual(state.status, 'resolved');
|
||||||
|
assert.strictEqual(state.resolvedClaimant, LOCAL);
|
||||||
|
assert.strictEqual(state.hash, 'hs://solo');
|
||||||
|
}
|
||||||
|
|
||||||
|
function testInsufficientQuorum() {
|
||||||
|
const state = resolve(
|
||||||
|
'quorum.test',
|
||||||
|
{
|
||||||
|
[LOCAL]: { hash: 'hs://local', timestamp: 1000 },
|
||||||
|
[REMOTE_A]: { hash: 'hs://remote', timestamp: 2000 }
|
||||||
|
},
|
||||||
|
{ [LOCAL]: LOCAL },
|
||||||
|
{ activePeers: 10 }
|
||||||
|
);
|
||||||
|
assert.strictEqual(state.status, 'insufficient_quorum');
|
||||||
|
assert.strictEqual(state.quorumMet, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testQuorumResolved() {
|
||||||
|
const claims = {
|
||||||
|
[LOCAL]: { hash: 'hs://local', timestamp: 1000 },
|
||||||
|
[REMOTE_A]: { hash: 'hs://remote', timestamp: 2000 }
|
||||||
|
};
|
||||||
|
const votes = {
|
||||||
|
voter1: LOCAL,
|
||||||
|
voter2: LOCAL,
|
||||||
|
voter3: LOCAL
|
||||||
|
};
|
||||||
|
const state = resolve('winner.test', claims, votes, { activePeers: 4 });
|
||||||
|
assert.strictEqual(state.status, 'resolved');
|
||||||
|
assert.strictEqual(state.resolvedClaimant, LOCAL);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testVoteValidationDisabled() {
|
||||||
|
const state = resolve(
|
||||||
|
'noval.test',
|
||||||
|
{ [LOCAL]: { hash: 'hs://local', timestamp: 1 } },
|
||||||
|
{ voter1: REMOTE_A, voter2: REMOTE_A, voter3: REMOTE_A },
|
||||||
|
{ config: { CONSENSUS_VOTE_VALIDATION: false }, activePeers: 4 }
|
||||||
|
);
|
||||||
|
assert.strictEqual(state.status, 'resolved');
|
||||||
|
assert.strictEqual(state.resolvedClaimant, REMOTE_A);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testVoteValidationEnabled() {
|
||||||
|
const state = resolve(
|
||||||
|
'val.test',
|
||||||
|
{
|
||||||
|
[LOCAL]: { hash: 'hs://local', timestamp: 1 },
|
||||||
|
[REMOTE_A]: { hash: 'hs://remote', timestamp: 2 }
|
||||||
|
},
|
||||||
|
{ voter1: REMOTE_B, voter2: REMOTE_B, voter3: REMOTE_B },
|
||||||
|
{ activePeers: 4 }
|
||||||
|
);
|
||||||
|
assert.strictEqual(state.status, 'insufficient_quorum');
|
||||||
|
}
|
||||||
|
|
||||||
|
function testTieTimestamp() {
|
||||||
|
const claims = {
|
||||||
|
[REMOTE_A]: { hash: 'hs://a', timestamp: 5000 },
|
||||||
|
[REMOTE_B]: { hash: 'hs://b', timestamp: 1000 }
|
||||||
|
};
|
||||||
|
const votes = { v1: REMOTE_A, v2: REMOTE_B, v3: REMOTE_A, v4: REMOTE_B };
|
||||||
|
const state = resolve('tie.test', claims, votes, {
|
||||||
|
activePeers: 4,
|
||||||
|
localWriter: 'other-peer'
|
||||||
|
});
|
||||||
|
assert.strictEqual(state.status, 'tie');
|
||||||
|
assert.strictEqual(state.resolvedClaimant, REMOTE_B);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testTieBreakerLexicographicPrefersLocal() {
|
||||||
|
const winner = applyTieBreaker([REMOTE_B, LOCAL], {}, LOCAL, 'lexicographic');
|
||||||
|
assert.strictEqual(winner, LOCAL);
|
||||||
|
}
|
||||||
|
|
||||||
|
function testDuplicateVoteLastWinsViaOrderedReplay() {
|
||||||
|
const { createEmptyViewState, applyEventToView } = require('../includes/core/consensus-apply');
|
||||||
|
const { EVENT_TYPES } = require('../includes/core/consensus-events');
|
||||||
|
const view = createEmptyViewState();
|
||||||
|
const domain = 'dup.test';
|
||||||
|
|
||||||
|
applyEventToView(view, {
|
||||||
|
type: EVENT_TYPES.CLAIM_UPSERT,
|
||||||
|
domain,
|
||||||
|
claimant: REMOTE_A,
|
||||||
|
hash: 'hs://a',
|
||||||
|
timestamp: 1,
|
||||||
|
ssl: false,
|
||||||
|
clients: []
|
||||||
|
});
|
||||||
|
applyEventToView(view, {
|
||||||
|
type: EVENT_TYPES.CLAIM_UPSERT,
|
||||||
|
domain,
|
||||||
|
claimant: REMOTE_B,
|
||||||
|
hash: 'hs://b',
|
||||||
|
timestamp: 2,
|
||||||
|
ssl: false,
|
||||||
|
clients: []
|
||||||
|
});
|
||||||
|
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_A, voter: 'voter1' });
|
||||||
|
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_B, voter: 'voter1' });
|
||||||
|
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_A, voter: 'voter2' });
|
||||||
|
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_A, voter: 'voter3' });
|
||||||
|
|
||||||
|
const domainState = view.domains.get(domain);
|
||||||
|
assert.strictEqual(domainState.voterVotes.get('voter1'), REMOTE_B);
|
||||||
|
|
||||||
|
const { buildDomainSnapshotFromView } = require('../includes/core/consensus-resolver');
|
||||||
|
const snapshot = buildDomainSnapshotFromView(view, domain);
|
||||||
|
const state = resolve(domain, snapshot.claims, snapshot.voterVotes, { activePeers: 4, localWriter: 'x' });
|
||||||
|
assert.strictEqual(state.resolvedClaimant, REMOTE_A);
|
||||||
|
}
|
||||||
|
|
||||||
|
function run() {
|
||||||
|
testLegacyClaimTimestamp();
|
||||||
|
testNoClaims();
|
||||||
|
testSingleLocalClaim();
|
||||||
|
testInsufficientQuorum();
|
||||||
|
testQuorumResolved();
|
||||||
|
testVoteValidationDisabled();
|
||||||
|
testVoteValidationEnabled();
|
||||||
|
testTieTimestamp();
|
||||||
|
testTieBreakerLexicographicPrefersLocal();
|
||||||
|
testDuplicateVoteLastWinsViaOrderedReplay();
|
||||||
|
console.log('consensus-resolver.test.js: all passed');
|
||||||
|
}
|
||||||
|
|
||||||
|
run();
|
||||||
Reference in New Issue
Block a user