EXPERIMENTAL: replace KV-scan consensus with Autobase sidecar engine

Cut over domain resolution to an Autobase apply-based sidecar fed by
claim/vote dual-writes from dnsPass. Remove the legacy full-KV scan
getConsensusState path and wire all reads through consensus-view.

- Add consensus-resolver, consensus-events, consensus-apply,
  consensus-autobase, and consensus-view modules
- Dual-append consensus events on claim/vote dnsPassAdd/Remove
- Bootstrap sidecar from existing KV entries; persist
  consensusAutobaseKey in network manifest
- Expose sidecar health via GET /api/consensus/status and metrics
- Add consensus-resolver and consensus-apply unit tests
- Expand RFC 0001 to Implemented; update CONSENSUS.md

Rollback: deploy prior release; KV data unchanged, sidecar rebuilds on
next startup.
This commit is contained in:
Raven Scott
2026-05-30 23:49:48 -04:00
parent 3a8daefc30
commit d230fb3360
19 changed files with 1867 additions and 890 deletions
+12 -12
View File
@@ -6,6 +6,8 @@ This document provides a deep dive into the P2NS consensus mechanism for domain
P2NS uses a quorum-based consensus mechanism to resolve domain ownership disputes. When multiple peers claim the same domain, the network votes to determine the legitimate owner. This enables decentralized domain management without a central authority. P2NS uses a quorum-based consensus mechanism to resolve domain ownership disputes. When multiple peers claim the same domain, the network votes to determine the legitimate owner. This enables decentralized domain management without a central authority.
Resolution is computed by an **Autobase apply-based consensus sidecar** ([RFC 0001](rfcs/0001-autobase-consensus.md)). Claim and vote KV writes in Autopass dual-append typed events to the sidecar; the apply handler replays events in linearized order and `getConsensusState` reads from that view.
**Key goals:** **Key goals:**
- Prevent domain squatting through voting - Prevent domain squatting through voting
- Handle network partitions gracefully - Handle network partitions gracefully
@@ -108,7 +110,7 @@ Where:
| Status | Description | | Status | Description |
|--------|-------------| |--------|-------------|
| `resolved` | Single winner determined by vote count or tie-breaker | | `resolved` | Single winner determined by vote count or tie-breaker |
| `conflict` | Domain resolved to another claimant, but local peer has a competing claim | | `conflict` | Domain resolved to another claimant, but local peer has a competing claim (computed in admin/UI layers, not returned by `getConsensusState`) |
| `insufficient_quorum` | Not enough votes to reach consensus | | `insufficient_quorum` | Not enough votes to reach consensus |
| `tie` | Multiple claimants tied; resolved via tie-breaker | | `tie` | Multiple claimants tied; resolved via tie-breaker |
| `no_claims` | No claims exist for this domain | | `no_claims` | No claims exist for this domain |
@@ -169,23 +171,21 @@ P2NS automatically casts votes under certain conditions.
```javascript ```javascript
async function autoVoteForDomain(domain, entries) { async function autoVoteForDomain(domain, entries) {
// Get claims for this domain
const claims = getClaimsForDomain(domain, entries); const claims = getClaimsForDomain(domain, entries);
// Check if local peer already voted
if (hasLocalVote(domain, entries)) return;
// Vote for local claim if exists
if (claims.has(localWriter)) { if (claims.has(localWriter)) {
await castVote(domain, localWriter); await castVote(domain, localWriter);
return; return;
} }
// Vote for claim with most existing votes if (Object.keys(claims).length === 1) {
const winner = getLeadingClaimant(domain, entries); await castVote(domain, Object.keys(claims)[0]);
if (winner) { return;
await castVote(domain, winner);
} }
// Multiple claimants: use configured tie-breaker among claimants
const winner = applyTieBreaker(Object.keys(claims), claimTimestamps, localWriter);
await castVote(domain, winner);
} }
``` ```
+85 -23
View File
@@ -1,40 +1,102 @@
# RFC 0001: Autobase Consensus Evaluation # RFC 0001: Autobase Consensus
## Status ## Status
Draft Implemented
## Problem ## Background
P2NS currently resolves domain ownership with custom vote/quorum logic on top of Autopass entries. This works, but it carries maintenance overhead and bespoke reconciliation behavior. P2NS resolves domain ownership with quorum-based voting over claim and vote records stored in the Autopass KV ledger. The previous resolver scanned all KV entries on every query, which caused maintenance overhead, non-deterministic duplicate-vote handling (iteration order), and legacy timestamp instability.
## Proposal ## Goals
Evaluate an Autobase-backed consensus stream for claim and vote events, with deterministic apply logic for ownership resolution. - Replace the KV-scan resolver with an Autobase apply-based consensus sidecar as the **sole** production read path.
- Deterministic event replay in Autobase linearized order.
## Scope - Bootstrap replay from existing dnsPass claim/vote entries for network migration.
- Sidecar health diagnostics in admin API and domain.consensus plugin.
- Prototype an isolated consensus view for domain claims and votes.
- Compare convergence behavior with the current implementation.
- Keep existing Autopass paths as source-of-truth during evaluation.
## Non-Goals ## Non-Goals
- Immediate migration of production consensus logic. - Changing claim/vote KV key format or plugin SDK method signatures.
- Changing plugin APIs in this phase. - In-process legacy engine fallback or feature-flagged dual resolver.
- Removing Autopass KV as the durable write store for claims and votes.
## Migration Plan ## Architecture
1. Build sidecar Autobase view fed from current claim/vote entries. ```mermaid
2. Add parity checks against current resolver output. flowchart TD
3. Gate optional read path via feature flag. Writes[domains.js voteForDomain] --> Queue[dns-pass-queue.js]
4. Decide go/no-go after parity and performance testing. Queue --> DnsPass[Autopass KV]
Queue --> Events[Consensus Autobase append]
Events --> Apply[consensus-apply.js]
Apply --> View[consensus-view.js]
View --> GetState[getConsensusState]
Resolver[consensus-resolver.js] --> View
```
## Risks Autopass KV remains the durable store. Every local claim/vote mutation dual-writes a typed event to the consensus Autobase. The apply handler maintains per-domain state; `getConsensusState` reads from the view and runs the pure resolver.
- Reordering semantics may expose hidden assumptions in current handlers. ## Event Model
- Requires deterministic apply handlers and strong replay discipline.
| Type | Payload | Source KV |
|------|---------|-----------|
| `claim_upsert` | domain, claimant, hash, timestamp, ssl, clients | `claim:{domain}:{claimant}` add |
| `claim_remove` | domain, claimant | `claim:{domain}:{claimant}` remove |
| `vote_upsert` | domain, claimant, voter | `vote:{domain}:{claimant}:{voter}` add |
| `vote_remove` | domain, claimant, voter | `vote:{domain}:{claimant}:{voter}` remove |
Events are JSON-encoded buffers appended to the consensus Autobase.
## Apply Semantics
Events are processed in Autobase linearized order:
1. **claim_upsert** — set or replace claim for claimant on domain.
2. **claim_remove** — remove claim; drop votes for that claimant on the domain.
3. **vote_upsert** — set voter's vote (last event per voter wins).
4. **vote_remove** — remove voter's vote.
Resolution (`consensus-resolver.js`) runs at read time with the same quorum, tie-break, vote-validation, and single-local-claim rules as the prior implementation.
## Determinism Fixes
- Legacy hash-only claims use `timestamp: 0` (not runtime `Date.now()`).
- Duplicate votes resolved by event order in the Autobase stream, not KV list iteration order.
## Migration
1. Genesis or first peer with empty sidecar bootstraps by replaying dnsPass claim/vote entries sorted by `(type, domain, timestamp, claimant, voter)`.
2. Network manifest stores `consensusAutobaseKey` for joiners.
3. Joiners open the sidecar with the manifest bootstrap key and replicate via `consensusBase.replicate(connection)` on swarm connections.
4. Local claim/vote writes dual-append events after successful dnsPass operations.
## Configuration
Existing consensus env vars unchanged (`CONSENSUS_QUORUM_THRESHOLD`, `CONSENSUS_MIN_VOTES`, `CONSENSUS_TIE_BREAKER`, `CONSENSUS_VOTE_VALIDATION`).
## Acceptance Criteria
- Legacy KV-scan `getConsensusState` removed from codebase.
- `test-scripts/consensus-resolver.test.js` and `test-scripts/consensus-apply.test.js` pass in CI.
- Admin `GET /api/consensus/status` reports sidecar health.
- `docs/CONSENSUS.md` aligned with implementation.
## Rollback ## Rollback
Disable feature flag and keep existing consensus resolver. Deploy the previous P2NS release. Autopass KV data is unchanged; upgrading again rebuilds the sidecar from bootstrap replay.
## Implementation Map
| Module | Path |
|--------|------|
| Pure resolver | `includes/core/consensus-resolver.js` |
| Event codec | `includes/core/consensus-events.js` |
| Apply handler | `includes/core/consensus-apply.js` |
| Autobase lifecycle | `includes/core/consensus-autobase.js` |
| Read API | `includes/core/consensus-view.js` |
| Dual-write | `includes/core/dns-pass-queue.js` |
| Manifest key | `includes/infrastructure/network-manifest.js` |
## Changelog
- **2026-05-30**: Production cutover — Autobase sidecar replaces KV-scan resolver; RFC expanded from evaluation draft to implemented spec.
@@ -10,6 +10,25 @@ async function handleConsensusRoutes(req, res) {
const method = req.method; const method = req.method;
const url = new URL(req.url, `https://${req.headers.host}`); const url = new URL(req.url, `https://${req.headers.host}`);
// GET /api/consensus/status - Sidecar health
if (method === 'GET' && urlPath === '/api/consensus/status') {
try {
trackRequest('/api/consensus/status', true);
const { getConsensusStatus } = require('../../../core/consensus-autobase');
const status = getConsensusStatus();
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(status));
return true;
} catch (err) {
logError('Consensus', `Failed to get consensus status: ${err.message}`);
trackRequest('/api/consensus/status', false);
const errorResponse = createErrorResponse(err, 500);
res.writeHead(errorResponse.statusCode, errorResponse.headers);
res.end(errorResponse.body);
return true;
}
}
// GET /api/consensus/metrics - Get consensus metrics // GET /api/consensus/metrics - Get consensus metrics
// Check this BEFORE the domain route to avoid matching "metrics" as a domain // Check this BEFORE the domain route to avoid matching "metrics" as a domain
if (method === 'GET' && urlPath === '/api/consensus/metrics') { if (method === 'GET' && urlPath === '/api/consensus/metrics') {
+102
View File
@@ -0,0 +1,102 @@
/**
* Autobase apply handler and in-memory consensus view state.
*/
const { decodeEvent, EVENT_TYPES } = require('./consensus-events');
function createEmptyViewState() {
return {
domains: new Map(),
eventCount: 0,
lastApplyAt: 0
};
}
function getOrCreateDomain(view, domain) {
if (!view.domains.has(domain)) {
view.domains.set(domain, {
claims: new Map(),
voterVotes: new Map()
});
}
return view.domains.get(domain);
}
function applyEventToView(view, event) {
switch (event.type) {
case EVENT_TYPES.CLAIM_UPSERT: {
const domainState = getOrCreateDomain(view, event.domain);
domainState.claims.set(event.claimant, {
hash: event.hash,
timestamp: event.timestamp,
ssl: event.ssl === true,
clients: event.clients || []
});
break;
}
case EVENT_TYPES.CLAIM_REMOVE: {
const domainState = view.domains.get(event.domain);
if (!domainState) break;
domainState.claims.delete(event.claimant);
for (const [voter, claimant] of domainState.voterVotes) {
if (claimant === event.claimant) {
domainState.voterVotes.delete(voter);
}
}
if (domainState.claims.size === 0 && domainState.voterVotes.size === 0) {
view.domains.delete(event.domain);
}
break;
}
case EVENT_TYPES.VOTE_UPSERT: {
const domainState = getOrCreateDomain(view, event.domain);
domainState.voterVotes.set(event.voter, event.claimant);
break;
}
case EVENT_TYPES.VOTE_REMOVE: {
const domainState = view.domains.get(event.domain);
if (!domainState) break;
domainState.voterVotes.delete(event.voter);
if (domainState.claims.size === 0 && domainState.voterVotes.size === 0) {
view.domains.delete(event.domain);
}
break;
}
default:
break;
}
}
function openConsensusView() {
return createEmptyViewState();
}
async function applyConsensusNodes(nodes, view, host) {
for (const node of nodes) {
let event;
try {
event = decodeEvent(node.value);
} catch (err) {
continue;
}
if (event.type === 'add_writer' && event.key && host && typeof host.addWriter === 'function') {
await host.addWriter(Buffer.from(event.key, 'hex'), { indexer: true });
view.eventCount++;
view.lastApplyAt = Date.now();
continue;
}
applyEventToView(view, event);
view.eventCount++;
view.lastApplyAt = Date.now();
}
}
module.exports = {
createEmptyViewState,
getOrCreateDomain,
applyEventToView,
openConsensusView,
applyConsensusNodes
};
+271
View File
@@ -0,0 +1,271 @@
/**
* Consensus Autobase sidecar lifecycle.
*/
const Autobase = require('autobase');
const state = require('../infrastructure/state');
const { logDebug, logInfo, logWarn, logError } = require('../infrastructure/logger');
const { openConsensusView, applyConsensusNodes } = require('./consensus-apply');
const { encodeEvent, entriesToBootstrapEvents } = require('./consensus-events');
const { listAllEntries, waitForAutobaseIdle } = require('./dns-pass-queue');
let consensusBase = null;
let bootstrapComplete = false;
let initPromise = null;
function getConsensusBase() {
return consensusBase;
}
function isBootstrapComplete() {
return bootstrapComplete;
}
function createConsensusAutobase(store, bootstrapKey) {
const opts = {
open() {
return openConsensusView();
},
async apply(nodes, view, host) {
await applyConsensusNodes(nodes, view, host);
}
};
if (bootstrapKey) {
return new Autobase(store, bootstrapKey, opts);
}
return new Autobase(store, opts);
}
async function appendConsensusEvent(event) {
if (!consensusBase || consensusBase.closed) {
throw new Error('consensus autobase not open');
}
await waitForAutobaseIdle({ base: consensusBase });
await consensusBase.append(encodeEvent(event));
await consensusBase.update();
}
async function appendConsensusEvents(events) {
for (const event of events) {
await appendConsensusEvent(event);
}
}
async function ensureLocalWriter() {
if (!consensusBase || consensusBase.closed) return;
await consensusBase.ready();
await consensusBase.update();
if (consensusBase.writable) return;
const localKey = consensusBase.local?.key;
if (!localKey) return;
try {
await appendConsensusEvent({ type: 'add_writer', key: localKey.toString('hex') });
await consensusBase.update();
} catch (err) {
logWarn('ConsensusAutobase', `Could not request writer access: ${err.message}`);
}
}
async function bootstrapFromDnsPass(pass, options = {}) {
const { allowLocalBootstrap = true, syncWaitMs = 3000 } = options;
if (!consensusBase || bootstrapComplete) return;
await consensusBase.ready();
await consensusBase.update();
if (!allowLocalBootstrap && syncWaitMs > 0) {
const deadline = Date.now() + syncWaitMs;
while (Date.now() < deadline) {
await consensusBase.update();
if (consensusBase.view && consensusBase.view.eventCount > 0) break;
await sleep(200);
}
}
const view = consensusBase.view;
if (view && view.eventCount > 0) {
bootstrapComplete = true;
logInfo('ConsensusAutobase', `Sidecar already has ${view.eventCount} events, skipping bootstrap`);
return;
}
if (!allowLocalBootstrap) {
logInfo('ConsensusAutobase', 'Joiner sidecar empty after sync — waiting for replication before local bootstrap');
bootstrapComplete = true;
return;
}
if (!pass) {
logWarn('ConsensusAutobase', 'dnsPass not available for bootstrap');
bootstrapComplete = true;
return;
}
const entries = await listAllEntries(pass);
const events = entriesToBootstrapEvents(entries);
if (events.length === 0) {
bootstrapComplete = true;
logInfo('ConsensusAutobase', 'No claim/vote entries to bootstrap');
return;
}
logInfo('ConsensusAutobase', `Bootstrapping sidecar with ${events.length} events from dnsPass`);
await appendConsensusEvents(events);
bootstrapComplete = true;
logInfo('ConsensusAutobase', 'Bootstrap complete');
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
/**
* @param {object} options
* @param {import('corestore')} options.store
* @param {string|null} options.bootstrapKey
* @param {import('autopass')} options.dnsPass
* @param {boolean} options.isGenesis
*/
async function initializeConsensusAutobase({ store, bootstrapKey, dnsPass, isGenesis }) {
if (!store || typeof store.get !== 'function') {
logWarn('ConsensusAutobase', 'Valid corestore required — skipping consensus sidecar init');
return null;
}
if (initPromise) return initPromise;
initPromise = (async () => {
if (consensusBase && !consensusBase.closed) {
return consensusBase;
}
try {
consensusBase = createConsensusAutobase(store, bootstrapKey || null);
state.consensusAutobase = consensusBase;
await consensusBase.ready();
await consensusBase.update();
if (!bootstrapKey && isGenesis) {
const keyHex = consensusBase.key.toString('hex');
logInfo('ConsensusAutobase', `Genesis consensus autobase created: ${keyHex.slice(0, 16)}...`);
}
await ensureLocalWriter();
await bootstrapFromDnsPass(dnsPass, {
allowLocalBootstrap: !bootstrapKey || isGenesis,
syncWaitMs: bootstrapKey ? 3000 : 0
});
consensusBase.on('update', () => {
try {
const { invalidateConsensusCache } = require('./consensus-view');
invalidateConsensusCache();
} catch (err) {
logDebug('ConsensusAutobase', `Cache invalidation on update: ${err.message}`);
}
});
return consensusBase;
} catch (err) {
initPromise = null;
if (consensusBase && !consensusBase.closed) {
try {
await consensusBase.close();
} catch (closeErr) {
logDebug('ConsensusAutobase', `Close after failed init: ${closeErr.message}`);
}
}
consensusBase = null;
state.consensusAutobase = null;
throw err;
}
})();
try {
return await initPromise;
} catch (err) {
initPromise = null;
throw err;
}
}
function replicateConsensus(connection) {
if (!consensusBase || consensusBase.closed || !connection) return;
try {
consensusBase.replicate(connection);
} catch (err) {
logDebug('ConsensusAutobase', `Replicate failed: ${err.message}`);
}
}
async function closeConsensusAutobase() {
bootstrapComplete = false;
initPromise = null;
if (!consensusBase) return;
try {
if (!consensusBase.closed) {
await consensusBase.close();
}
} catch (err) {
logError('ConsensusAutobase', `Error closing consensus autobase: ${err.message}`);
} finally {
consensusBase = null;
state.consensusAutobase = null;
}
}
function getConsensusStatus() {
const base = consensusBase;
const view = base && !base.closed ? base.view : null;
return {
open: !!(base && !base.closed),
ready: !!(base && base.opened !== false && !base.closed),
writable: !!(base && base.writable),
bootstrapComplete,
eventCount: view ? view.eventCount : 0,
domainCount: view ? view.domains.size : 0,
lastApplyAt: view ? view.lastApplyAt : null,
indexedLength: base ? base.indexedLength : 0,
length: base ? base.length : 0,
key: base && base.key ? base.key.toString('hex') : null
};
}
async function initConsensusForNetwork({ store, dnsPass, networkManifest, manifestPath }) {
const base = await initializeConsensusAutobase({
store,
bootstrapKey: networkManifest?.consensusAutobaseKey || null,
dnsPass,
isGenesis: !networkManifest?.consensusAutobaseKey
});
if (!base) return null;
const keyHex = base.key ? base.key.toString('hex') : null;
if (keyHex && !networkManifest?.consensusAutobaseKey && manifestPath) {
const networkManifestModule = require('../infrastructure/network-manifest');
await networkManifestModule.updateManifestConsensusKey(manifestPath, keyHex);
if (state.networkManifest) {
state.networkManifest.consensusAutobaseKey = keyHex;
}
}
return base;
}
module.exports = {
getConsensusBase,
isBootstrapComplete,
initializeConsensusAutobase,
initConsensusForNetwork,
appendConsensusEvent,
appendConsensusEvents,
replicateConsensus,
closeConsensusAutobase,
getConsensusStatus,
bootstrapFromDnsPass,
ensureLocalWriter
};
+106
View File
@@ -0,0 +1,106 @@
/**
* Consensus event encoding and KV key translation.
*/
const { parseClaimValue } = require('./consensus-resolver');
const EVENT_TYPES = {
CLAIM_UPSERT: 'claim_upsert',
CLAIM_REMOVE: 'claim_remove',
VOTE_UPSERT: 'vote_upsert',
VOTE_REMOVE: 'vote_remove'
};
function encodeEvent(event) {
return Buffer.from(JSON.stringify(event), 'utf8');
}
function decodeEvent(buffer) {
const raw = Buffer.isBuffer(buffer) ? buffer.toString('utf8') : String(buffer);
return JSON.parse(raw);
}
function isConsensusKvKey(key) {
return key.startsWith('claim:') || key.startsWith('vote:');
}
function kvMutationToEvent(key, value, isRemove) {
if (key.startsWith('claim:')) {
const parts = key.split(':');
if (parts.length !== 3) return null;
const domain = parts[1];
const claimant = parts[2];
if (isRemove) {
return { type: EVENT_TYPES.CLAIM_REMOVE, domain, claimant };
}
const parsed = parseClaimValue(value);
return {
type: EVENT_TYPES.CLAIM_UPSERT,
domain,
claimant,
hash: parsed.hash,
timestamp: parsed.timestamp,
ssl: parsed.ssl,
clients: parsed.clients || []
};
}
if (key.startsWith('vote:')) {
const parts = key.split(':');
if (parts.length !== 4) return null;
const domain = parts[1];
const claimant = parts[2];
const voter = parts[3];
if (isRemove) {
return { type: EVENT_TYPES.VOTE_REMOVE, domain, claimant, voter };
}
return { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant, voter };
}
return null;
}
/**
* Deterministic bootstrap ordering for dnsPass KV entries.
* @param {Array<{ key: string, value: string }>} entries
* @returns {Array<object>}
*/
function entriesToBootstrapEvents(entries) {
const events = [];
for (const entry of entries) {
if (!isConsensusKvKey(entry.key)) continue;
const event = kvMutationToEvent(entry.key, entry.value, false);
if (event) events.push(event);
}
events.sort((a, b) => {
const typeOrder = (t) => {
if (t.startsWith('claim_')) return 0;
return 1;
};
const ta = typeOrder(a.type);
const tb = typeOrder(b.type);
if (ta !== tb) return ta - tb;
if (a.domain !== b.domain) return a.domain.localeCompare(b.domain);
const tsA = a.timestamp ?? 0;
const tsB = b.timestamp ?? 0;
if (tsA !== tsB) return tsA - tsB;
const ca = a.claimant || '';
const cb = b.claimant || '';
if (ca !== cb) return ca.localeCompare(cb);
const va = a.voter || '';
const vb = b.voter || '';
return va.localeCompare(vb);
});
return events;
}
module.exports = {
EVENT_TYPES,
encodeEvent,
decodeEvent,
isConsensusKvKey,
kvMutationToEvent,
entriesToBootstrapEvents
};
+181
View File
@@ -0,0 +1,181 @@
/**
* Pure consensus resolution logic for domain ownership.
* Used by the Autobase apply view at read time.
*/
const LEGACY_CLAIM_TIMESTAMP = 0;
function parseClaimValue(value) {
try {
const parsed = JSON.parse(value);
if (parsed.hash && parsed.timestamp !== undefined) {
return {
hash: parsed.hash,
clients: parsed.clients || [],
timestamp: parsed.timestamp,
ssl: parsed.ssl === true,
legacy: false
};
}
if (parsed.hash) {
return {
hash: parsed.hash,
clients: parsed.clients || [],
timestamp: parsed.timestamp || LEGACY_CLAIM_TIMESTAMP,
ssl: parsed.ssl === true,
legacy: false
};
}
} catch (e) {
// legacy hash-only string
}
return {
hash: value,
clients: [],
timestamp: LEGACY_CLAIM_TIMESTAMP,
ssl: false,
legacy: true
};
}
function validateVote(claimant, claims, voteValidationEnabled) {
if (!voteValidationEnabled) return true;
return Object.prototype.hasOwnProperty.call(claims, claimant);
}
function applyTieBreaker(candidates, claimTimestamps, localWriter, strategy) {
switch (strategy) {
case 'timestamp':
return candidates.sort((a, b) => {
const tsA = claimTimestamps[a] || 0;
const tsB = claimTimestamps[b] || 0;
return tsA - tsB;
})[0];
case 'claimant_age':
if (candidates.includes(localWriter)) return localWriter;
return candidates.sort((a, b) => a.localeCompare(b))[0];
case 'lexicographic':
default:
if (candidates.includes(localWriter)) return localWriter;
return candidates.sort((a, b) => a.localeCompare(b))[0];
}
}
/**
* @param {object} params
* @param {string} params.domain
* @param {Record<string, { hash: string, timestamp: number, ssl?: boolean, clients?: array }>} params.claims
* @param {Record<string, string>} params.voterVotes - voter -> claimant
* @param {object} params.config
* @param {string|null} params.localWriter
* @param {number} params.activePeers
*/
function resolveDomainConsensus({
domain,
claims,
voterVotes,
config,
localWriter,
activePeers
}) {
const claimTimestamps = {};
const claimHashes = {};
const voteCounts = {};
for (const [claimant, claim] of Object.entries(claims || {})) {
claimHashes[claimant] = claim.hash;
claimTimestamps[claimant] = claim.timestamp;
voteCounts[claimant] = 0;
}
const voters = new Set();
const voteValidation = config.CONSENSUS_VOTE_VALIDATION !== false;
for (const [voter, claimant] of Object.entries(voterVotes || {})) {
if (!validateVote(claimant, claimHashes, voteValidation)) continue;
voteCounts[claimant] = (voteCounts[claimant] || 0) + 1;
voters.add(voter);
}
const minVotes = Math.max(
config.CONSENSUS_MIN_VOTES || 2,
Math.ceil(activePeers * (config.CONSENSUS_QUORUM_THRESHOLD ?? 0.5))
);
const totalVotes = Object.values(voteCounts).reduce((sum, count) => sum + count, 0);
const quorumMet = totalVotes >= minVotes;
let status = 'no_claims';
let resolvedClaimant = null;
let hash = null;
const claimants = Object.keys(claimHashes);
if (claimants.length === 0) {
status = 'no_claims';
} else if (!quorumMet) {
const isSingleLocalClaim =
claimants.length === 1 && localWriter && Object.prototype.hasOwnProperty.call(claimHashes, localWriter);
const onlyWeVoted = totalVotes === 0 || (totalVotes === 1 && voters.has(localWriter));
if (isSingleLocalClaim && (totalVotes === 0 || onlyWeVoted)) {
status = 'resolved';
resolvedClaimant = localWriter;
hash = claimHashes[localWriter];
} else {
status = 'insufficient_quorum';
}
} else {
const maxVotes = Math.max(...Object.values(voteCounts));
const candidates = Object.keys(voteCounts).filter((c) => voteCounts[c] === maxVotes);
if (candidates.length === 1) {
status = 'resolved';
resolvedClaimant = candidates[0];
hash = claimHashes[resolvedClaimant];
} else {
status = 'tie';
const strategy = config.CONSENSUS_TIE_BREAKER || 'timestamp';
resolvedClaimant = applyTieBreaker(candidates, claimTimestamps, localWriter, strategy);
hash = claimHashes[resolvedClaimant];
}
}
return {
status,
hash,
resolvedClaimant,
voteCounts: { ...voteCounts },
activePeers,
quorumMet,
minVotes,
totalVotes,
lastResolution: Date.now()
};
}
function buildDomainSnapshotFromView(viewState, domain) {
const domainData = viewState.domains.get(domain);
if (!domainData) {
return { claims: {}, voterVotes: {} };
}
const claims = {};
for (const [claimant, claim] of domainData.claims) {
claims[claimant] = { ...claim };
}
const voterVotes = {};
for (const [voter, claimant] of domainData.voterVotes) {
voterVotes[voter] = claimant;
}
return { claims, voterVotes };
}
module.exports = {
LEGACY_CLAIM_TIMESTAMP,
parseClaimValue,
validateVote,
applyTieBreaker,
resolveDomainConsensus,
buildDomainSnapshotFromView
};
+230
View File
@@ -0,0 +1,230 @@
/**
* Consensus read API backed by the Autobase sidecar view.
*/
const state = require('../infrastructure/state');
const { validateConfig } = require('../infrastructure/config');
const { getPersistentPublicKey } = require('../infrastructure/utils');
const { logDebug, logInfo, logWarn } = require('../infrastructure/logger');
const { trackConsensusEvent } = require('../maintenance/metrics');
const {
resolveDomainConsensus,
buildDomainSnapshotFromView
} = require('./consensus-resolver');
const { getConsensusBase, getConsensusStatus, isBootstrapComplete } = require('./consensus-autobase');
function getActivePeerCount() {
const writers = state.networkWriterPeers;
if (writers && writers.size > 0) {
return writers.size + 1;
}
return (state.connectedPeers?.size || 0) + 1;
}
const consensusStateCache = new Map();
const CONSENSUS_CACHE_TTL = 10000;
const consensusMetrics = {
resolutions: 0,
quorumFailures: 0,
ties: 0,
validationFailures: 0,
totalVotes: 0,
avgVotesPerDomain: 0,
domainResolutions: new Map()
};
const countedVotes = new Set();
const countedResolutions = new Set();
const countedQuorumFailures = new Set();
const countedTies = new Set();
let consensusConfig = null;
function getConsensusConfig() {
if (!consensusConfig) {
try {
consensusConfig = validateConfig();
} catch (err) {
consensusConfig = {
CONSENSUS_QUORUM_THRESHOLD: 0.5,
CONSENSUS_MIN_VOTES: 2,
CONSENSUS_TIE_BREAKER: 'timestamp',
CONSENSUS_VOTE_VALIDATION: true,
CONSENSUS_IMMEDIATE_UPDATE: true
};
}
}
return consensusConfig;
}
function invalidateConsensusCache(domain) {
if (domain) {
consensusStateCache.delete(domain);
} else {
consensusStateCache.clear();
}
}
function recordResolutionMetrics(domain, status, resolvedClaimant, config) {
if (status === 'resolved' && resolvedClaimant) {
const resolutionKey = `${domain}:${resolvedClaimant}`;
if (!countedResolutions.has(resolutionKey)) {
countedResolutions.add(resolutionKey);
consensusMetrics.resolutions++;
}
} else if (status === 'insufficient_quorum') {
if (!countedQuorumFailures.has(domain)) {
countedQuorumFailures.add(domain);
consensusMetrics.quorumFailures++;
}
} else if (status === 'tie' && resolvedClaimant) {
const tieKey = `${domain}:${resolvedClaimant}`;
if (!countedTies.has(tieKey)) {
countedTies.add(tieKey);
consensusMetrics.ties++;
}
if (trackConsensusEvent) {
trackConsensusEvent('tie', { domain });
}
logInfo('ConsensusView', `Tie resolved for ${domain} using ${config.CONSENSUS_TIE_BREAKER}: ${resolvedClaimant}`);
}
if (!consensusMetrics.domainResolutions.has(domain)) {
consensusMetrics.domainResolutions.set(domain, { resolved: 0, failed: 0 });
}
const domainMetrics = consensusMetrics.domainResolutions.get(domain);
if (status === 'resolved') {
domainMetrics.resolved++;
} else if (status !== 'no_claims') {
domainMetrics.failed++;
}
}
async function getConsensusState(domain) {
const pass = state.dnsPass;
if (!pass) {
return {
status: 'error',
error: 'dnsPass not initialized',
hash: null,
resolvedClaimant: null,
voteCounts: {},
activePeers: 0,
quorumMet: false,
lastResolution: null
};
}
const cached = consensusStateCache.get(domain);
if (cached && Date.now() - cached.timestamp < CONSENSUS_CACHE_TTL) {
return cached.state;
}
const base = getConsensusBase();
if (!base || base.closed) {
return {
status: 'error',
error: 'consensus sidecar not initialized',
hash: null,
resolvedClaimant: null,
voteCounts: {},
activePeers: 0,
quorumMet: false,
lastResolution: null
};
}
await base.ready();
await base.update();
const localWriter = getPersistentPublicKey();
if (!localWriter) {
return {
status: 'error',
error: 'Persistent public key not available',
hash: null,
resolvedClaimant: null,
voteCounts: {},
activePeers: 0,
quorumMet: false,
lastResolution: null
};
}
const view = base.view;
if (!view) {
return {
status: 'error',
error: 'consensus view not available',
hash: null,
resolvedClaimant: null,
voteCounts: {},
activePeers: 0,
quorumMet: false,
lastResolution: null
};
}
const snapshot = buildDomainSnapshotFromView(view, domain);
const config = getConsensusConfig();
const activePeers = getActivePeerCount();
const consensusState = resolveDomainConsensus({
domain,
claims: snapshot.claims,
voterVotes: snapshot.voterVotes,
config,
localWriter,
activePeers
});
recordResolutionMetrics(domain, consensusState.status, consensusState.resolvedClaimant, config);
for (const [claimant, count] of Object.entries(consensusState.voteCounts)) {
if (count <= 0) continue;
for (const voter of Object.keys(snapshot.voterVotes)) {
if (snapshot.voterVotes[voter] !== claimant) continue;
const voteKey = `${domain}:${claimant}:${voter}`;
if (!countedVotes.has(voteKey)) {
countedVotes.add(voteKey);
consensusMetrics.totalVotes++;
}
}
}
const domainsWithVotes = consensusMetrics.domainResolutions.size;
if (domainsWithVotes > 0) {
consensusMetrics.avgVotesPerDomain = consensusMetrics.totalVotes / domainsWithVotes;
}
consensusStateCache.set(domain, { state: consensusState, timestamp: Date.now() });
return consensusState;
}
function getConsensusMetrics() {
const sidecar = getConsensusStatus();
return {
...consensusMetrics,
sidecar,
bootstrapComplete: isBootstrapComplete(),
domainResolutions: Array.from(consensusMetrics.domainResolutions.entries()).map(([d, metrics]) => ({
domain: d,
...metrics
}))
};
}
function getAllDomainNamesFromView() {
const base = getConsensusBase();
if (!base || !base.view) return [];
return Array.from(base.view.domains.keys());
}
module.exports = {
getConsensusState,
getConsensusMetrics,
getConsensusConfig,
invalidateConsensusCache,
getAllDomainNamesFromView
};
+23
View File
@@ -106,6 +106,29 @@ function createCoreSwarmHandlers(ctx) {
} }
setupListeners(); setupListeners();
try {
const { initConsensusForNetwork } = require('./consensus-autobase');
const networkManifest = require('../infrastructure/network-manifest');
const manifestPath = networkManifest.getManifestPath(process.env.NETWORK_MANIFEST_FILE);
let manifest = state.networkManifest;
if (!manifest) {
manifest = await networkManifest.readManifest(manifestPath);
if (manifest) state.networkManifest = manifest;
}
const base = await initConsensusForNetwork({
store,
dnsPass: newPass,
networkManifest: manifest,
manifestPath
});
if (base) {
logInfo('Swarm', 'Consensus sidecar initialized after pairing');
}
} catch (err) {
logError('Swarm', `Failed to initialize consensus sidecar: ${err.message}`);
}
if (isMaster) { if (isMaster) {
state.masterPendingPass = false; state.masterPendingPass = false;
logInfo('Swarm', 'Secondary master paired — dnsPass ready, master invite policy active'); logInfo('Swarm', 'Secondary master paired — dnsPass ready, master invite policy active');
+427 -770
View File
File diff suppressed because it is too large Load Diff
+37 -8
View File
@@ -156,14 +156,6 @@ async function runSerialized(pass, fn) {
}); });
} }
async function dnsPassAdd(pass, key, value, file) {
return runSerialized(pass, () => pass.add(key, value, file));
}
async function dnsPassRemove(pass, key) {
return runSerialized(pass, () => pass.remove(key));
}
async function dnsPassGet(pass, key) { async function dnsPassGet(pass, key) {
return runSerialized(pass, () => pass.get(key)); return runSerialized(pass, () => pass.get(key));
} }
@@ -182,10 +174,47 @@ async function listAllEntries(pass) {
}); });
} }
async function appendConsensusEventForKvMutation(key, value, isRemove) {
try {
const { kvMutationToEvent, isConsensusKvKey } = require('./consensus-events');
const { appendConsensusEvent } = require('./consensus-autobase');
if (!isConsensusKvKey(key)) return;
const event = kvMutationToEvent(key, value, isRemove);
if (!event) return;
await appendConsensusEvent(event);
} catch (err) {
logWarn('DnsPassQueue', `Consensus event append failed for ${key}: ${err.message}`);
}
}
async function dnsPassAdd(pass, key, value, file) {
return runSerialized(pass, async () => {
const result = await pass.add(key, value, file);
await appendConsensusEventForKvMutation(key, value, false);
return result;
});
}
async function dnsPassRemove(pass, key) {
return runSerialized(pass, async () => {
let existingValue = null;
try {
existingValue = await pass.get(key);
} catch (err) {
// ignore lookup errors before remove
}
const result = await pass.remove(key);
const valueStr = existingValue != null ? existingValue.toString('utf8') : '';
await appendConsensusEventForKvMutation(key, valueStr, true);
return result;
});
}
module.exports = { module.exports = {
enqueueDnsPass, enqueueDnsPass,
whenDnsPassIdle, whenDnsPassIdle,
ensureDnsPassOpen, ensureDnsPassOpen,
waitForAutobaseIdle,
syncDnsPassView, syncDnsPassView,
isInviteCreationActive, isInviteCreationActive,
isAtomicDnsPassError, isAtomicDnsPassError,
+17 -2
View File
@@ -93,16 +93,30 @@ async function writeManifest(manifestPath, fields) {
} }
async function adoptManifestFromPass(pass, topicSeed, manifestPath, options = {}) { async function adoptManifestFromPass(pass, topicSeed, manifestPath, options = {}) {
const { genesisPublicKey, isGenesisRun = false } = options; const { genesisPublicKey, isGenesisRun = false, consensusAutobaseKey = null } = options;
const identity = extractNetworkIdentity(pass, genesisPublicKey); const identity = extractNetworkIdentity(pass, genesisPublicKey);
const doc = await writeManifest(manifestPath, { const doc = await writeManifest(manifestPath, {
...identity, ...identity,
topicSeed: topicSeed || process.env.TOPIC_SEED || 'p2ns-dns', topicSeed: topicSeed || process.env.TOPIC_SEED || 'p2ns-dns',
isGenesis: isGenesisRun isGenesis: isGenesisRun,
...(consensusAutobaseKey ? { consensusAutobaseKey } : {})
}); });
return doc; return doc;
} }
async function updateManifestConsensusKey(manifestPath, consensusAutobaseKey) {
const filePath = getManifestPath(manifestPath);
const existing = await readManifest(manifestPath);
if (!existing) return null;
const doc = {
...existing,
consensusAutobaseKey
};
await fs.writeFile(filePath, `${JSON.stringify(doc, null, 2)}\n`, 'utf8');
logInfo('NetworkManifest', `Updated manifest with consensusAutobaseKey: ${consensusAutobaseKey.slice(0, 16)}...`);
return doc;
}
function recordPeerNetworkStatus(peerId, status) { function recordPeerNetworkStatus(peerId, status) {
if (!peerId || !status) return; if (!peerId || !status) return;
const state = require('./state'); const state = require('./state');
@@ -163,6 +177,7 @@ module.exports = {
readManifest, readManifest,
writeManifest, writeManifest,
adoptManifestFromPass, adoptManifestFromPass,
updateManifestConsensusKey,
recordPeerNetworkStatus, recordPeerNetworkStatus,
checkSplitBrain, checkSplitBrain,
getLocalNetworkSummary getLocalNetworkSummary
+1
View File
@@ -42,6 +42,7 @@ const currentSubnetIndex = 0; // Start with first subnet for round-robin
module.exports = { module.exports = {
dnsPass: null, dnsPass: null,
consensusAutobase: null,
corestore: null, corestore: null,
hypercoreStats: null, hypercoreStats: null,
/** Incremented while dns-pass-queue runs a serialized Autopass op (defer update handlers) */ /** Incremented while dns-pass-queue runs a serialized Autopass op (defer update handlers) */
+30
View File
@@ -449,6 +449,12 @@ async function main() {
function scheduleConnectionReplication(conn) { function scheduleConnectionReplication(conn) {
if (!conn || conn.destroyed) return; if (!conn || conn.destroyed) return;
logDebug('Swarm', 'Skipping p2ns-topic Autopass replication; Autopass swarm handles it'); logDebug('Swarm', 'Skipping p2ns-topic Autopass replication; Autopass swarm handles it');
try {
const { replicateConsensus } = require('./includes/core/consensus-autobase');
replicateConsensus(conn);
} catch (err) {
logDebug('Swarm', `Consensus sidecar replication: ${err.message}`);
}
} }
// Helper function to safely set dnsPass and ensure state consistency // Helper function to safely set dnsPass and ensure state consistency
@@ -1556,6 +1562,21 @@ async function main() {
logInfo('Main', 'Skipping domains.json watcher (secondary master; MASTER_LOAD_DOMAINS=false)'); logInfo('Main', 'Skipping domains.json watcher (secondary master; MASTER_LOAD_DOMAINS=false)');
} }
setupListeners(); setupListeners();
try {
const { initConsensusForNetwork } = require('./includes/core/consensus-autobase');
const base = await initConsensusForNetwork({
store,
dnsPass: newPass,
networkManifest: state.networkManifest,
manifestPath
});
if (base) {
logInfo('Main', 'Consensus sidecar initialized');
}
} catch (err) {
logError('Main', `Failed to initialize consensus sidecar: ${err.message}`);
}
} }
if (isMaster) { if (isMaster) {
@@ -2686,6 +2707,15 @@ async function main() {
} }
} }
// Close consensus sidecar before dnsPass
try {
const { closeConsensusAutobase } = require('./includes/core/consensus-autobase');
await closeConsensusAutobase();
logDebug('Main', 'Consensus sidecar closed');
} catch (err) {
logWarn('Main', `Error closing consensus sidecar: ${err.message}`);
}
// Close DNSPass after pairing operations (main component) // Close DNSPass after pairing operations (main component)
if (state.dnsPass && typeof state.dnsPass.close === 'function') { if (state.dnsPass && typeof state.dnsPass.close === 'function') {
logDebug('Main', 'Closing dnsPass (main component)...'); logDebug('Main', 'Closing dnsPass (main component)...');
+2 -1
View File
@@ -9,7 +9,8 @@
"test:admin": "node test-scripts/admin-smoke.js", "test:admin": "node test-scripts/admin-smoke.js",
"test:peer-paste": "node test-scripts/peer-paste-smoke.js", "test:peer-paste": "node test-scripts/peer-paste-smoke.js",
"test:plugins": "node test-scripts/plugins-smoke.js", "test:plugins": "node test-scripts/plugins-smoke.js",
"test:core-rpc": "node test-scripts/core-rpc-smoke.js && node test-scripts/network-manifest.test.js && node test-scripts/core-swarm-handlers.test.js && node test-scripts/multi-master.test.js && node test-scripts/core-invite-rpc-integration.js", "test:core-rpc": "node test-scripts/core-rpc-smoke.js && node test-scripts/network-manifest.test.js && node test-scripts/core-swarm-handlers.test.js && node test-scripts/multi-master.test.js && node test-scripts/core-invite-rpc-integration.js && npm run test:consensus",
"test:consensus": "node test-scripts/consensus-resolver.test.js && node test-scripts/consensus-apply.test.js",
"test:multi-master": "node test-scripts/multi-master.test.js", "test:multi-master": "node test-scripts/multi-master.test.js",
"test:plugin-rpc": "node test-scripts/plugin-channel-rpc.test.js", "test:plugin-rpc": "node test-scripts/plugin-channel-rpc.test.js",
"audit": "npm audit --audit-level=high", "audit": "npm audit --audit-level=high",
+4
View File
@@ -146,6 +146,10 @@ async function handler(req, res) {
totalVotes: metrics.totalVotes || 0, totalVotes: metrics.totalVotes || 0,
avgVotesPerDomain: metrics.avgVotesPerDomain || 0 avgVotesPerDomain: metrics.avgVotesPerDomain || 0
}, },
sidecar: metrics.sidecar || {
open: false,
bootstrapComplete: false
},
activePeers: connectedPeers, activePeers: connectedPeers,
timestamp: Date.now() timestamp: Date.now()
}); });
+4 -74
View File
@@ -495,14 +495,12 @@ async function getAllEntries(pass = state.dnsPass, useCache = true) {
} }
async function getHashForDomain(domain) { async function getHashForDomain(domain) {
// Check cache first
const cached = getCachedHash(domain); const cached = getCachedHash(domain);
if (cached) { if (cached) {
logDebug('Core', `Cache hit for domain: ${domain}`, { domain }); logDebug('Core', `Cache hit for domain: ${domain}`, { domain });
return cached; return cached;
} }
// Try to use core consensus logic if available
try { try {
const { getHashForDomain: coreGetHashForDomain } = require('../includes/core/core'); const { getHashForDomain: coreGetHashForDomain } = require('../includes/core/core');
const hash = await coreGetHashForDomain(domain); const hash = await coreGetHashForDomain(domain);
@@ -511,12 +509,13 @@ async function getHashForDomain(domain) {
return hash; return hash;
} }
} catch (err) { } catch (err) {
logDebug('Core', `Core consensus not available, using fallback: ${err.message}`, { domain }); logDebug('Core', `Core consensus not available: ${err.message}`, { domain });
} }
return null;
} }
async function getDomainSSLStatus(domain) { async function getDomainSSLStatus(domain) {
// Try to use core consensus logic if available
try { try {
const { getDomainSSLStatus: coreGetDomainSSLStatus } = require('../includes/core/core'); const { getDomainSSLStatus: coreGetDomainSSLStatus } = require('../includes/core/core');
return await coreGetDomainSSLStatus(domain); return await coreGetDomainSSLStatus(domain);
@@ -524,75 +523,6 @@ async function getDomainSSLStatus(domain) {
logDebug('Core', `Core SSL status not available, defaulting to false: ${err.message}`, { domain }); logDebug('Core', `Core SSL status not available, defaulting to false: ${err.message}`, { domain });
return false; return false;
} }
// Fallback to simplified logic if core not available
const pass = state.dnsPass;
if (!pass) {
return null;
}
await pass.ready();
logDebug('Core', `Querying hash for domain: ${domain}`, { domain });
const localWriter = getPersistentPublicKey();
if (!localWriter) {
logWarn('Core', 'Cannot query hash: persistent public key not available', { domain });
return null;
}
const allEntries = await getAllEntries(pass, true);
const claims = {};
const voteCounts = {};
for (const entry of allEntries) {
if (entry.key.startsWith(`claim:${domain}:`)) {
const claimant = entry.key.slice(`claim:${domain}:`.length);
// Parse claim value (may be JSON with timestamp or legacy format)
try {
const parsed = JSON.parse(entry.value);
claims[claimant] = parsed.hash || entry.value;
} catch (e) {
claims[claimant] = entry.value; // Legacy format
}
voteCounts[claimant] = 0;
}
if (entry.key.startsWith(`vote:${domain}:`)) {
const parts = entry.key.split(':');
if (parts.length === 4) {
const claimant = parts[2];
// Validate vote references existing claim
if (voteCounts[claimant] !== undefined) {
voteCounts[claimant]++;
}
}
}
}
if (Object.keys(claims).length === 0) {
return null;
}
const maxVotes = Math.max(...Object.values(voteCounts));
if (maxVotes === 0) {
if (Object.keys(claims).length === 1 && claims.hasOwnProperty(localWriter)) {
const hash = claims[localWriter];
setCachedHash(domain, hash);
return hash;
}
return null;
}
const candidates = Object.keys(voteCounts).filter(claimant => voteCounts[claimant] === maxVotes);
let resolvedClaimant;
if (candidates.length === 1) {
resolvedClaimant = candidates[0];
} else {
resolvedClaimant = candidates.includes(localWriter) ? localWriter : candidates.sort((a, b) => a.localeCompare(b))[0];
}
const hash = claims[resolvedClaimant];
logInfo('Core', `Resolved claimant for ${domain}`, { domain, claimant: resolvedClaimant });
setCachedHash(domain, hash);
return hash;
} }
// ============================================================================ // ============================================================================
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env node
/**
* Unit tests for consensus apply + bootstrap event ordering.
*/
const assert = require('assert');
const { createEmptyViewState, applyEventToView } = require('../includes/core/consensus-apply');
const { EVENT_TYPES, entriesToBootstrapEvents } = require('../includes/core/consensus-events');
const {
resolveDomainConsensus,
buildDomainSnapshotFromView
} = require('../includes/core/consensus-resolver');
const CONFIG = {
CONSENSUS_QUORUM_THRESHOLD: 0.5,
CONSENSUS_MIN_VOTES: 2,
CONSENSUS_TIE_BREAKER: 'timestamp',
CONSENSUS_VOTE_VALIDATION: true
};
function replayEvents(events) {
const view = createEmptyViewState();
for (const event of events) {
applyEventToView(view, event);
}
return view;
}
function resolveFromView(view, domain, localWriter = 'local', activePeers = 4) {
const snapshot = buildDomainSnapshotFromView(view, domain);
return resolveDomainConsensus({
domain,
claims: snapshot.claims,
voterVotes: snapshot.voterVotes,
config: CONFIG,
localWriter,
activePeers
});
}
function testBootstrapOrdering() {
const entries = [
{ key: 'vote:demo.test:peer-b:voter-1', value: '1' },
{ key: 'claim:demo.test:peer-b', value: JSON.stringify({ hash: 'hs://b', timestamp: 2000 }) },
{ key: 'claim:demo.test:peer-a', value: JSON.stringify({ hash: 'hs://a', timestamp: 1000 }) },
{ key: 'vote:demo.test:peer-a:voter-1', value: '1' },
{ key: 'vote:demo.test:peer-a:voter-2', value: '1' }
];
const events = entriesToBootstrapEvents(entries);
assert.strictEqual(events[0].type, EVENT_TYPES.CLAIM_UPSERT);
assert.strictEqual(events[0].claimant, 'peer-a');
assert.strictEqual(events[1].claimant, 'peer-b');
assert.ok(events[2].type.startsWith('vote_'));
}
function testClaimRemoveDropsVotes() {
const view = replayEvents([
{
type: EVENT_TYPES.CLAIM_UPSERT,
domain: 'x.test',
claimant: 'a',
hash: 'hs://a',
timestamp: 1,
ssl: false,
clients: []
},
{ type: EVENT_TYPES.VOTE_UPSERT, domain: 'x.test', claimant: 'a', voter: 'v1' },
{ type: EVENT_TYPES.CLAIM_REMOVE, domain: 'x.test', claimant: 'a' }
]);
assert.strictEqual(view.domains.has('x.test'), false);
}
function testApplyMatchesResolver() {
const domain = 'apply.test';
const events = [
{
type: EVENT_TYPES.CLAIM_UPSERT,
domain,
claimant: 'peer-a',
hash: 'hs://a',
timestamp: 100,
ssl: false,
clients: []
},
{
type: EVENT_TYPES.CLAIM_UPSERT,
domain,
claimant: 'peer-b',
hash: 'hs://b',
timestamp: 200,
ssl: false,
clients: []
},
{ type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: 'peer-a', voter: 'v1' },
{ type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: 'peer-a', voter: 'v2' },
{ type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: 'peer-a', voter: 'v3' }
];
const view = replayEvents(events);
const state = resolveFromView(view, domain, 'peer-z', 4);
assert.strictEqual(state.status, 'resolved');
assert.strictEqual(state.resolvedClaimant, 'peer-a');
assert.strictEqual(state.hash, 'hs://a');
}
function testKvBootstrapFixture() {
const entries = [
{
key: 'claim:fixture.test:alpha',
value: JSON.stringify({ hash: 'hs://alpha', timestamp: 10, ssl: false })
},
{
key: 'claim:fixture.test:beta',
value: JSON.stringify({ hash: 'hs://beta', timestamp: 20, ssl: false })
},
{ key: 'vote:fixture.test:alpha:voter-1', value: 'hs://alpha' },
{ key: 'vote:fixture.test:alpha:voter-2', value: 'hs://alpha' },
{ key: 'vote:fixture.test:alpha:voter-3', value: 'hs://alpha' }
];
const view = replayEvents(entriesToBootstrapEvents(entries));
const state = resolveFromView(view, 'fixture.test', 'gamma', 4);
assert.strictEqual(state.status, 'resolved');
assert.strictEqual(state.resolvedClaimant, 'alpha');
}
function run() {
testBootstrapOrdering();
testClaimRemoveDropsVotes();
testApplyMatchesResolver();
testKvBootstrapFixture();
console.log('consensus-apply.test.js: all passed');
}
run();
+183
View File
@@ -0,0 +1,183 @@
#!/usr/bin/env node
/**
* Unit tests for consensus-resolver.js
*/
const assert = require('assert');
const {
parseClaimValue,
resolveDomainConsensus,
applyTieBreaker,
LEGACY_CLAIM_TIMESTAMP
} = require('../includes/core/consensus-resolver');
const DEFAULT_CONFIG = {
CONSENSUS_QUORUM_THRESHOLD: 0.5,
CONSENSUS_MIN_VOTES: 2,
CONSENSUS_TIE_BREAKER: 'timestamp',
CONSENSUS_VOTE_VALIDATION: true
};
const LOCAL = 'local-peer-key';
const REMOTE_A = 'remote-a-key';
const REMOTE_B = 'remote-b-key';
function resolve(domain, claims, voterVotes, overrides = {}) {
return resolveDomainConsensus({
domain,
claims,
voterVotes,
config: { ...DEFAULT_CONFIG, ...overrides.config },
localWriter: overrides.localWriter ?? LOCAL,
activePeers: overrides.activePeers ?? 4
});
}
function testLegacyClaimTimestamp() {
const parsed = parseClaimValue('hs://legacy-hash');
assert.strictEqual(parsed.timestamp, LEGACY_CLAIM_TIMESTAMP);
assert.strictEqual(parsed.legacy, true);
}
function testNoClaims() {
const state = resolve('example.test', {}, {});
assert.strictEqual(state.status, 'no_claims');
assert.strictEqual(state.hash, null);
}
function testSingleLocalClaim() {
const state = resolve(
'solo.test',
{ [LOCAL]: { hash: 'hs://solo', timestamp: 1000 } },
{}
);
assert.strictEqual(state.status, 'resolved');
assert.strictEqual(state.resolvedClaimant, LOCAL);
assert.strictEqual(state.hash, 'hs://solo');
}
function testInsufficientQuorum() {
const state = resolve(
'quorum.test',
{
[LOCAL]: { hash: 'hs://local', timestamp: 1000 },
[REMOTE_A]: { hash: 'hs://remote', timestamp: 2000 }
},
{ [LOCAL]: LOCAL },
{ activePeers: 10 }
);
assert.strictEqual(state.status, 'insufficient_quorum');
assert.strictEqual(state.quorumMet, false);
}
function testQuorumResolved() {
const claims = {
[LOCAL]: { hash: 'hs://local', timestamp: 1000 },
[REMOTE_A]: { hash: 'hs://remote', timestamp: 2000 }
};
const votes = {
voter1: LOCAL,
voter2: LOCAL,
voter3: LOCAL
};
const state = resolve('winner.test', claims, votes, { activePeers: 4 });
assert.strictEqual(state.status, 'resolved');
assert.strictEqual(state.resolvedClaimant, LOCAL);
}
function testVoteValidationDisabled() {
const state = resolve(
'noval.test',
{ [LOCAL]: { hash: 'hs://local', timestamp: 1 } },
{ voter1: REMOTE_A, voter2: REMOTE_A, voter3: REMOTE_A },
{ config: { CONSENSUS_VOTE_VALIDATION: false }, activePeers: 4 }
);
assert.strictEqual(state.status, 'resolved');
assert.strictEqual(state.resolvedClaimant, REMOTE_A);
}
function testVoteValidationEnabled() {
const state = resolve(
'val.test',
{
[LOCAL]: { hash: 'hs://local', timestamp: 1 },
[REMOTE_A]: { hash: 'hs://remote', timestamp: 2 }
},
{ voter1: REMOTE_B, voter2: REMOTE_B, voter3: REMOTE_B },
{ activePeers: 4 }
);
assert.strictEqual(state.status, 'insufficient_quorum');
}
function testTieTimestamp() {
const claims = {
[REMOTE_A]: { hash: 'hs://a', timestamp: 5000 },
[REMOTE_B]: { hash: 'hs://b', timestamp: 1000 }
};
const votes = { v1: REMOTE_A, v2: REMOTE_B, v3: REMOTE_A, v4: REMOTE_B };
const state = resolve('tie.test', claims, votes, {
activePeers: 4,
localWriter: 'other-peer'
});
assert.strictEqual(state.status, 'tie');
assert.strictEqual(state.resolvedClaimant, REMOTE_B);
}
function testTieBreakerLexicographicPrefersLocal() {
const winner = applyTieBreaker([REMOTE_B, LOCAL], {}, LOCAL, 'lexicographic');
assert.strictEqual(winner, LOCAL);
}
function testDuplicateVoteLastWinsViaOrderedReplay() {
const { createEmptyViewState, applyEventToView } = require('../includes/core/consensus-apply');
const { EVENT_TYPES } = require('../includes/core/consensus-events');
const view = createEmptyViewState();
const domain = 'dup.test';
applyEventToView(view, {
type: EVENT_TYPES.CLAIM_UPSERT,
domain,
claimant: REMOTE_A,
hash: 'hs://a',
timestamp: 1,
ssl: false,
clients: []
});
applyEventToView(view, {
type: EVENT_TYPES.CLAIM_UPSERT,
domain,
claimant: REMOTE_B,
hash: 'hs://b',
timestamp: 2,
ssl: false,
clients: []
});
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_A, voter: 'voter1' });
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_B, voter: 'voter1' });
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_A, voter: 'voter2' });
applyEventToView(view, { type: EVENT_TYPES.VOTE_UPSERT, domain, claimant: REMOTE_A, voter: 'voter3' });
const domainState = view.domains.get(domain);
assert.strictEqual(domainState.voterVotes.get('voter1'), REMOTE_B);
const { buildDomainSnapshotFromView } = require('../includes/core/consensus-resolver');
const snapshot = buildDomainSnapshotFromView(view, domain);
const state = resolve(domain, snapshot.claims, snapshot.voterVotes, { activePeers: 4, localWriter: 'x' });
assert.strictEqual(state.resolvedClaimant, REMOTE_A);
}
function run() {
testLegacyClaimTimestamp();
testNoClaims();
testSingleLocalClaim();
testInsufficientQuorum();
testQuorumResolved();
testVoteValidationDisabled();
testVoteValidationEnabled();
testTieTimestamp();
testTieBreakerLexicographicPrefersLocal();
testDuplicateVoteLastWinsViaOrderedReplay();
console.log('consensus-resolver.test.js: all passed');
}
run();