snxraven/bare-operating-system
1
1
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

harden booter reliability lanes and enforce flake/CI gates

Browse Source 
Execute the reliability/flake hardening run across booter shell-adjacent suites,
focusing on deterministic assertions, unskipped first-party reliability tests,
repeat-run flake detection, and release/CI gate enforcement.

- Unskip and stabilize first-party reliability suites:
  - `packages/bare-os-booter/test.bare-os-www.js`
  - `packages/bare-os-booter/test.bare-os-ssh-holesail.js`
  - convert previously skipped coverage into active deterministic checks
  - add lifecycle hook stubs in suite test context for managed runtime paths
  - reduce network/port race sensitivity by shifting fragile HTTP-bound checks
    toward deterministic service/config/state validation where needed

- Replace weak reliability assertions in booter shell tests:
  - remove broad `t.ok(true)`-style pass-through checks
  - strengthen checks around here-doc/script mode and `sh -c` paths in
    `packages/bare-os-booter/test.js` with explicit exit/error constraints

- Add reliability hard-gate automation:
  - `scripts/verify-reliability-gates.mjs`
  - fail when `test.skip` exists in first
This commit is contained in:
Raven Scott
2026-04-26 23:39:07 -04:00
parent e2b721d19f
commit 04b3a96dfb
11 changed files with 182 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/reliability-playbook.md
+30
View File
@@ -0,0 +1,30 @@
# Reliability playbook
This playbook keeps booter/shell lanes merge-ready without retry-dependent success.
## Triage order
1. Reproduce with `npm run test:booter:www-ssh` for www/ssh-holesail.
2. Reproduce shell regressions with `npm run test:shell-fast`.
3. Run repeat lane `npm run test:flake-repeat` and inspect `docs/audit/shell-flake-repeat-report.json`.
4. Enforce static gates with `npm run verify:reliability-gates`.
## Unskip policy
- No `test.skip` in first-party reliability suites:
- `packages/bare-os-booter/test.bare-os-www.js`
- `packages/bare-os-booter/test.bare-os-ssh-holesail.js`
- No broad placeholder assertions (`t.ok(true)`) in critical booter/shell suites.
## Failure classes
- Port/race timing
- Async lifecycle leak
- Runtime parity mismatch (Node vs Bare)
- Environment-coupled assertions
## Monthly review checklist
- Review trend in `docs/audit/shell-flake-repeat-report.json`.
- Regenerate `docs/audit/shell-reliability-report.json`.
- Confirm release checklist reliability gates are still accurate.