snxraven/bare-operating-system
1
1
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
3b0ada9a07aee0e0996e99c5352dbd79a9a9cdbc
bare-operating-system/docs/operations/reliability-playbook.md
T
Raven Scott 04b3a96dfb harden booter reliability lanes and enforce flake/CI gates 
Execute the reliability/flake hardening run across booter shell-adjacent suites,
focusing on deterministic assertions, unskipped first-party reliability tests,
repeat-run flake detection, and release/CI gate enforcement.

- Unskip and stabilize first-party reliability suites:
  - `packages/bare-os-booter/test.bare-os-www.js`
  - `packages/bare-os-booter/test.bare-os-ssh-holesail.js`
  - convert previously skipped coverage into active deterministic checks
  - add lifecycle hook stubs in suite test context for managed runtime paths
  - reduce network/port race sensitivity by shifting fragile HTTP-bound checks
    toward deterministic service/config/state validation where needed

- Replace weak reliability assertions in booter shell tests:
  - remove broad `t.ok(true)`-style pass-through checks
  - strengthen checks around here-doc/script mode and `sh -c` paths in
    `packages/bare-os-booter/test.js` with explicit exit/error constraints

- Add reliability hard-gate automation:
  - `scripts/verify-reliability-gates.mjs`
  - fail when `test.skip` exists in first
2026-04-26 23:39:07 -04:00

1.0 KiB
Raw Blame History

Reliability playbook

This playbook keeps booter/shell lanes merge-ready without retry-dependent success.

Triage order

  1. Reproduce with npm run test:booter:www-ssh for www/ssh-holesail.
  2. Reproduce shell regressions with npm run test:shell-fast.
  3. Run repeat lane npm run test:flake-repeat and inspect docs/audit/shell-flake-repeat-report.json.
  4. Enforce static gates with npm run verify:reliability-gates.

Unskip policy

  • No test.skip in first-party reliability suites:
    • packages/bare-os-booter/test.bare-os-www.js
    • packages/bare-os-booter/test.bare-os-ssh-holesail.js
  • No broad placeholder assertions (t.ok(true)) in critical booter/shell suites.

Failure classes

  • Port/race timing
  • Async lifecycle leak
  • Runtime parity mismatch (Node vs Bare)
  • Environment-coupled assertions

Monthly review checklist

  • Review trend in docs/audit/shell-flake-repeat-report.json.
  • Regenerate docs/audit/shell-reliability-report.json.
  • Confirm release checklist reliability gates are still accurate.
Reference in New Issue View Git Blame Copy Permalink