Raven Scott
aa81fbff5b
Close the BareOS shell + Fish REPL roadmap tracker and ship the remaining
...
shell surfaces in-tree.
Roadmap / CI
- docs/data/shell-roadmap-features.json: all shell-001…shell-100 rows and
P1–P8 phases marked implemented; note documents closure date and pointers.
- scripts/verify-shell-roadmap.mjs: validate JSON (schema 1, 8 phases,
100 implemented items) plus existing source needles; wire npm run
verify:shell-roadmap into root pretest (package.json).
- scripts/README.md: document verifier behavior.
Lexer & expansion (packages/bare-os-booter/lib)
- shell-lex.js: central lexShellLine; ANSI-C $'…' via decodeBareOsDollarQuote;
keep diagnostics/tokenizer aligned with execution tokenizer.
- shell.js: stray reserved words at statement start → syntax error exit 2;
optional [[ … ]] when BARE_OS_SHELL_DOUBLE_BRACKET=1 (==, !=);
alias expansion before function dispatch (ordering tests);
expandWordWithCmdSubst: balanced $(…) vs skipped $((…)); backtick
command substitution when BARE_OS_SHELL_CMDSUBST; default ctx.execLine for
nested cmdsubst when unset; index passthrough for DOUBLE_BRACKET env.
- shell-glob.js: ~login → HOME when USER matches, else /home/login (bounded
login pattern); tests for pathname + execShellLine.
- shell-tokenizer.js: align with shell-lex detailed spans/modes where needed.
Completion / REPL
- completion-engine.js: completion depth / collectors per shell program work.
- packages/bare-os-booter/index.js: small wiring for shell env passthrough.
/bin/sh front-end
- packages/bare-os-coreutils/src/sh.js; kernel/bin/sh; seeder copies: stay in
sync with shell behavior and env flags.
Tests
- packages/bare-os-booter/test.js: coverage for misplaced reserved words,
gated [[ ]], alias vs function, ~user/~~ paths, $'…', cmdsubst $(…) and
backticks, and related regressions.
Documentation
- docs/reference/shell-grammar.md: $'…', $(…) / backticks vs $((…)).
- handbook/09-posix-utilities-shell-and-vfs.md, environment appendix,
shell-troubleshooting / shell-unsupported-behavior, release checklist,
docs/reference/README.md: shell behavior and operator surfaces.
- developer-guide/19-how-to-fish-keybinding-completer.md: Fish keybinding /
completer how-to (new).
Generated / synced artifacts
- kernel/lib/bare/manifest.json, kernel/share/man/man.json,
kernel/lib/bare/shell-completion.json, posix_utilities.json,
docs/audit/bundle-health.json: regenerated or synced with tooling.
- scripts/bench-shell-phases.mjs: bench script touch.
2026-04-27 00:41:06 -04:00
Raven Scott
5647491b08
Implement BareOS Zero-Trust Boot and Runtime Hardening (plan batches A–E).
...
Peer admission and bootstrap controls
- Fail closed when BARE_OS_PEER_ALLOWLIST_HEX is empty unless explicit
break-glass BARE_OS_PEER_ALLOW_ALL=1.
- Treat BARE_OS_ZERO_TRUST_PROFILE=strict|security like strict admission
posture alongside BARE_OS_PEER_ALLOWLIST_STRICT.
- Document BARE_OS_PEER_ALLOW_ALL and profile semantics; update boot trust
model operator guidance.
Peer system seed and provenance
- In strict/security profile, peer system seed defaults off unless
BARE_OS_PEER_SYSTEM_SEED is explicitly enabled (1/true/yes).
- Disable synthetic capability filling in strict profile; keep compat path
when profile is not strict.
- Extend test.peer-system-seed.js for strict default-off and no-synthesis.
Path capability signer trust
- When BARE_OS_PATH_CAPABILITY_ENFORCE_READ is on, require trusted issuer
if BARE_OS_PATH_CAPABILITY_REQUIRE_TRUSTED_SIGNER is set or profile is
strict; wire verifyPathCapabilityEnvelopeTrusted into the primary deny
path.
- Document BARE_OS_PATH_CAPABILITY_REQUIRE_TRUSTED_SIGNER and trusted key
list usage in environment appendix.
Host delegates (least privilege)
- Under strict/security profile, empty BARE_OS_DELEGATE_ALLOW means deny-all
delegates instead of allow-all; document behavior.
- Add delegate strict-profile test coverage.
Audit durability and telemetry hygiene
- Retain audit chain rows in memory and add bareOsAuditPersistRows for
optional NDJSON persistence via VFS.
- Broaden var-log redaction for secret-shaped strings and env-like assignments.
- Emit boot.log security line when unsafe trust combinations are detected.
Release and CI gates
- Add scripts/verify-zero-trust-gates.mjs and npm run verify:zero-trust-gates.
- Document verifier in scripts/README.md and zero-trust steps in
docs/release-checklist.md.
Tests
- Update bare-os-booter admission tests for allow-all and empty-allowlist
messaging.
- Relax brittle man.json page-count equality to a minimal sanity check to
avoid brittle/os.cwd brittle failures on inventory drift.
Verification (local): npm run verify:zero-trust-gates; npm run test -w
bare-os-booter; peer-system-seed brittle lane as applicable.
Plan file (.cursor/plans/zero-trust-boot-runtime-100-plan_*.plan.md) was not
edited per instructions.
2026-04-26 23:49:54 -04:00
Raven Scott
04b3a96dfb
harden booter reliability lanes and enforce flake/CI gates
...
Execute the reliability/flake hardening run across booter shell-adjacent suites,
focusing on deterministic assertions, unskipped first-party reliability tests,
repeat-run flake detection, and release/CI gate enforcement.
- Unskip and stabilize first-party reliability suites:
- `packages/bare-os-booter/test.bare-os-www.js`
- `packages/bare-os-booter/test.bare-os-ssh-holesail.js`
- convert previously skipped coverage into active deterministic checks
- add lifecycle hook stubs in suite test context for managed runtime paths
- reduce network/port race sensitivity by shifting fragile HTTP-bound checks
toward deterministic service/config/state validation where needed
- Replace weak reliability assertions in booter shell tests:
- remove broad `t.ok(true)`-style pass-through checks
- strengthen checks around here-doc/script mode and `sh -c` paths in
`packages/bare-os-booter/test.js` with explicit exit/error constraints
- Add reliability hard-gate automation:
- `scripts/verify-reliability-gates.mjs`
- fail when `test.skip` exists in first
2026-04-26 23:39:07 -04:00
Raven Scott
e2b721d19f
complete shell 100-item plan across parser, execution, security, and docs
...
Implement the full BareOS shell roadmap end-to-end, including grammar/tokenization
diagnostics, expansion/runtime hardening, execution graph tooling, builtins/job-control
stability, policy/sandbox controls, and release/traceability documentation updates.
- Add shell grammar baseline and diagnostics primitives:
- introduce `docs/reference/shell-grammar.md` with lexer modes and EBNF contract
- add rich diagnostic tokenizer output (mode + span metadata) via `tokenizeBareShellLineDetailed`
- export structured parse snapshot helpers (`bareOsShellAstSnapshot`) and shell error kinds
- add determinism coverage for tokenizer and AST snapshot outputs
- Harden expansion semantics and guardrails:
- enforce expansion byte budgets (`BARE_OS_SHELL_EXPANSION_MAX_BYTES`)
- add expansion trace hooks (`BARE_OS_SHELL_EXPANSION_TRACE`) with stage-level rows
- add expansion recursion depth limits (`BARE_OS_SHELL_EXPANSION_MAX_DEPTH`)
- tighten POSIX-mode arithmetic invalid-token diagnostics
- preserve declared expansion ordering and document it in code/docs
- Extend redirection/pipeline execution model:
- add normalized redirection planner (`planShellRedirections`) independent of side effects
- add execution graph builder/debug surface (`buildShellExecutionGraph`)
- support `<<-` operator in tokenizer/parser paths
- add pipeline stage timeout safety (`BARE_OS_SHELL_PIPELINE_STAGE_TIMEOUT_MS`)
- keep pipefail/pipestatus behavior verified with integration tests
- Improve builtins and control-flow reliability:
- expand `read` builtin support:
- `-r` raw mode
- `-d` single-char delimiter
- `-t` timeout semantics
- refine wait/jobs semantics:
- stable `jobs -l` parseable format expectations
- synthetic pid mapping (`wait 410x`) and `wait all` support
- keep trap registration/listing behavior deterministic and test-covered
- add trap signal dispatch helper (`dispatchShellTrapSignal`) with normalization
- Add security and policy enforcement hooks:
- command deny/allow policy gates:
- `BARE_OS_SHELL_DENY_COMMANDS`
- `BARE_OS_SHELL_ALLOW_COMMANDS`
- sandbox mode (`BARE_OS_SHELL_SANDBOX`) to block external command execution
- redirect path safety guard (`BARE_OS_SHELL_REDIRECT_GUARD`) for pseudo-path/traversal risks
- emit structured shell audit event rows (`ctx.shellAuditEvents`) for start/error/finish
- Improve interactive UX resilience:
- add prompt-hook timeout protection in fish readline:
- `resolveShellPromptHookSegment`
- env control `BARE_OS_SHELL_PROMPT_HOOK_TIMEOUT_MS`
- ensure prompt segment resolution is non-blocking and safe on timeout/error
- Add reliability/performance artifacts and shell fast lane:
- add `scripts/bench-shell-phases.mjs` for shell microbench sanity checks
- add `scripts/gen-shell-reliability-report.mjs` and generate reliability JSON artifact
- add root scripts:
- `test:shell-fast`
- `bench:shell`
- `report:shell-reliability`
- Expand shell-focused docs and traceability:
- add:
- `docs/reference/shell-unsupported-behavior.md`
- `docs/reference/shell-troubleshooting.md`
- add contributor guides:
- `developer-guide/17-how-to-add-shell-builtin.md`
- `developer-guide/18-how-to-add-shell-grammar-feature.md`
- update indexes/traceability/release gate docs:
- `docs/reference/README.md`
- `docs/reference/posix-issue7-traceability.md`
- `docs/reference/environment-and-posix-appendix.md`
- `docs/release-checklist.md`
- `developer-guide/README.md`
- `scripts/README.md`
- Add and update shell regression tests in `packages/bare-os-booter/test.js` for:
- tokenizer spans/modes and deterministic output
- AST snapshot schema/shape
- redirection planner and execution graph behavior
- expansion trace and strict arithmetic paths
- `<<-` support
- pipeline stage timeout handling
- `read` delimiter/raw/timeout semantics
- jobs/wait parseability and selection semantics
- trap dispatch and normalization behavior
- policy/sandbox/redirect-guard/audit-event pathways
Validation:
- `npm run test -w bare-os-booter`
- `npm run test:shell-bracket -w bare-os-booter`
- `npm run test:shell-fast`
- `npm run report:shell-reliability`
2026-04-26 23:17:20 -04:00
Raven Scott
d286ce19b5
chore(plan): cancel end-to-end seeder-to-booter smoke harness task
...
test(protocol): add deterministic MBR failover-key coverage
docs(protocol): align package-bare-os-protocol version to 0.9.1
test(booter): add MBR corruption and wrong-topic smoke fixtures
test(peer-seed): add strict pre-MBR bare_os.capabilities negotiation check
feat(seeder): validate BARE_OS_SEED_REQUIRE_MBR_LABELS
feat(seeder): validate BARE_OS_SEED_CAPABILITY_ATTESTATION_JSON schema
docs(boot-policy): add requireProtocolPackageMin 0.9.1 example
test(kernel): cover boot.policy denySeedRpcMethods behavior
test(protocol): add app/cap/chat/meshdrop channel compatibility fixture
test(swarm-disk): cover duplicate Protomux channel null-return path
test(protocol): add 11-word kernelCapabilityWords round-trip fixture
docs(schema): add mbr-layout schema and validate seeder examples
test(protocol): add topicKey() golden hash fixture
docs(trust): document block-0 trust assumptions in boot docs
feat(seeder): add discovery.flushed readiness logging
feat(booter): record peer discovery timings in boot-perf.json
feat(integration): add local testnet mode to integration lab smoke
test(booter): add Hyperswarm connection-budget env regression coverage
test(booter): add swarm plus Corestore suspend/resume integration coverage
feat(booter): mirror swarm ban events into host audit logs
feat(booter): add direct-peer boot via BARE_OS_BOOT_JOIN_PEER_HEX
feat(seeder): pass BARE_OS_SEED_MAX_PEERS to Hyperswarm
feat(seeder): log drive.version and discoveryKey at startup
test(booter): add Hyperdrive.checkout read-only boot probe coverage
feat(booter): prefetch /boot/init.js before kernel handoff
feat(booter): add optional /bin warm replication via downloadDiff
feat(seeder): add manifestPaths SHA-256 generation in stage-kernel-tree
test(peer-seed): cover helper-served block-0 after seeder exit
feat(protocol): add Protomux cork batching for initial channel sends
test(boot-graph): compare kernel/init labels with booter graph proc
docs(boot-policy): add v9-v11 schema examples
feat(release): add requireInitJsSha256 fixture generation step
test(vfs): add BARE_OS_VFS_SYSTEM_RO_ALIAS coverage
test(vfs): strengthen system-drive write-deny path coverage
feat(identity): add personal-drive namespace export/import docs and tests
test(booter): add guest-to-login warm cache invalidation regression
test(vfs): add guest deny coverage for /.bare sensitive paths
test(coreutils): add cross-drive mv failure injection coverage
test(vfs): add .bareos_empty round-trip coverage across mkdir/rmdir/cp/git-fs
test(vfs): add /dev/shm quota enforcement coverage
test(proc): add /proc/bare_os/index.json sortedness and schema checks
test(vfs): add warm read cache invalidation on replication growth
docs(ctx): document bareOsInvalidateWarmReadCaches(reason)
test(kernel): add BARE_OS_BOOT_DRY_RUN behavior coverage
docs(posix): add dashboard rows for all COREUTILS_COMMANDS
feat(curl): expand -w variables beyond http_code/url_effective/size_download
feat(wget): mark -N timestamping as explicit unsupported error
feat(curl): plumb mutual TLS cert/key intent to ctx.httpFetch metadata
feat(shuf): add deterministic seed mode via BARE_OS_SHUF_SEED
docs(sort): document -M month-sort as unsupported
feat(grep): add explicit -E and -G mode handling
test(sed): add Open Group Issue 7 golden fixtures
test(awk): add getline VFS regressions for missing/repeat/boundary cases
test(shell): add non-interactive here-doc coverage
test(shell): add trap delivery coverage for synthetic PIDs/job IDs
test(shell): add set -e compound-body behavior coverage
docs(shell): strengthen read builtin opt-in guidance
test(env): add Bare-runtime coverage for -S and --env-file
docs(man): add examples for pathcap-verify pkg-swarm-index corestorectl
test(identity): add account/vault backup-restore smoke coverage
feat(audit): add tamper detection verification for audit chain rows
test(peer-admission): cover strict empty allowlist deny behavior
test(peer-admission): add denylist precedence over allowlist coverage
test(peer-admission): add BARE_OS_PEER_REQUIRE_CAPS_JSON metadata checks
docs(identity): add trusted-key rotation example for path capabilities
feat(schema): tighten extensionSignerPinsV2-V4 hash validation
test(delegate): add allowlist negative cases for curl/wget/git/hrpc/systemctl
test(proc): extend /proc/self/environ redaction key coverage
docs(security): add peer-assisted block-0 mirroring threat-model notes
feat(bench): add boot budget trend output from real booter phases
test(baretop): align fixture coverage with /proc snapshot key set
test(metrics): validate /proc/bare_os/metrics.prom OpenMetrics shape
docs(ops): add structured seeder NDJSON examples
test(replication): add live stall-hint coverage for no_peers/length_unavailable/ok
docs(release): add corestore-snapshot workflow to checklist
docs(ops): add mirror-drive experiment utility to maintainer workflow
test(booter): add monitor progress coverage for replication live sketch
feat(seeder): validate DHT bootstrap address class JSON inputs
docs(network): add HYPERSWARM_BOOTSTRAP testnet operator guidance
chore(root): add deterministic test:integration script
docs(ci): add local CI runbook for no-.github environments
docs(release): add npm run test:bare after npm test
feat(verify): add protocol docs/package version parity checker
feat(verify): enforce feature-roadmap canonical path consistency
feat(lockfile-drift): add tier-1 strict fail option for mismatches
docs(lockfile-drift): add udx-native and blind-peering upgrade workflow notes
docs(cli-parity): add bare-fetch upstream issue tracking row
feat(bundle-health): generate per-tier bundle size regression thresholds
feat(doc-contracts): verify handbook references to current proc schema versions
feat(pretest): add validate-mermaid-syntax gate
feat(probe): add bare-runtime top-25 critical command lane
docs(protocol): update capability-word prose from bits..bits5 to current words
docs(two-drive): document /tmp /var/log and account-prefix routing
docs(security): add concise boot trust model page and links
docs(dev-guide): add P2P lab cookbook section
docs(dev-guide): add how-to for adding seed RPCs
docs(dev-guide): add how-to for adding /proc/bare_os nodes
docs(dev-guide): add /bin utility checklist for man/posix/build/parity/tests
docs(user-manual): add short What BareOS is not section
2026-04-26 22:28:21 -04:00
Raven Scott
0e3e5be329
Further Updates to MD
2026-04-25 23:15:49 -04:00
Raven Scott
b9d3a8b17f
Further MD Fixes
2026-04-25 23:04:56 -04:00
Raven Scott
b07bc38708
MD Formatting fixes
2026-04-25 23:02:07 -04:00
Raven Scott
eb8f33f6d8
Update Docs
2026-04-05 23:29:36 -04:00
Raven Scott
8bde745191
feat: complete 20-track kernel roadmap (P2P ops, POSIX, HRPC, CI, docs)
...
- Add optional Holepunch clone lag gate (holepunch-freshness-gate.json,
verify-holepunch-clone-freshness.mjs) and wire into pretest/docs.
- Extend stock ctx.bareOsHrpcRequest with disk.os replication routes;
bump hrpc_allowlist_sketch proc to schema 2 with stockRoutes list.
- Security posture: blindRelayAudit; hyper_multisig_trust_pointer schema 2
+ vault multisig continuity env; login/unlock audit hook.
- Syscalls schema 9 alignment (JSON schema, compatibility matrix, conformance
matrix clock_gettime); boot budget telemetry schema 2 in metrics_live.
- Coreutils hostname -s/--short man/options; rebuild kernel bins/man.
- POSIX + P2P dashboard section in docs/README; handbook/DOCUMENTATION/
release-checklist/OTA/KERNEL_CONTRACT/PEAR-RUN and related reference updates.
- verify-boot-policy-extension-signer-pins: scan kernel init fragments.
Note: vendor drift section removed from kernel/lib/bare/README.md (intentional).
2026-04-05 14:17:20 -04:00
Raven Scott
1faf9715b7
feat: POSIX/P2P roadmap — proc surfaces, syscalls, boot perf, docs, CI
...
- Align posix-conformance-matrix bareOsSyscallOps with getconf BARE_OS_SYSCALL_OPS
- Add verify-boot-policy-extension-signer-pins.mjs to pretest; document in scripts/README
- Document extensionSignerPinsV2–V5 env wiring in OTA_AND_BUNDLES; vault multisig sketch
- Note telemetry NDJSON redaction limits in environment appendix
- Expand booter CHANGELOG 1.40.0 (boot-perf, shell POSIX mode, subprocess errors, etc.)
- Holepunch sync README: NDJSON summary path
Seeder/kernel parity and pretest already green for bundled changes.
2026-04-05 00:02:23 -04:00
Raven Scott
c29ec13cf9
Docs Update
2026-04-04 19:51:05 -04:00
Raven Scott
f8ddef7950
docs(ci): rename ctx client script, extend bare-import scan, POSIX tests, and doc sync
...
- Rename scripts/gen-ctx-client-stub.mjs to gen-ctx-client-helper.mjs; update
pretest, scripts README, and PLACEHOLDER_BASELINE.
- Scan packages/bare-os-protocol/lib in verify-bare-imports.mjs; clarify scope
in script header.
- Document bundle health / marker allowlist workflow in bare-os-bare-libs README
and quarterly upstream notes in docs/release-checklist.md.
- Add ctx API release checklist to docs/reference/ctx-api-versioning.md.
- Add shell param expansion V3 tests in bare-os-booter/test.js; add coreutils
posix-test-int-compare.test.mjs and wire into bare-os-coreutils test script.
- Sync conformance matrix, environment appendix, handbook ch.9, and add
pathconf to docs/reference/posix-conformance-matrix.json for verify-compat-matrix.
2026-04-04 19:41:15 -04:00
Raven Scott
44355f6924
feat(shell): pathname globbing, noglob/set -f, and syscall pathconf
...
- Add shell-glob.js (fnmatch, brace expansion gate, GLOB_* / DOTGLOB / STRICT_POSIX)
- Tokenizer tracks u/s/d quote segments; expand argv, redirects, and `for … in`
- Builtin `set -f` / `set +f`; richer `case` patterns via bareOsFnmatch
- CI: verify-bundle-throws.mjs + bundle-throw-allowlist.json in pretest
- VFS: symlink cycle detection (ELOOP); stat follow depth 64
- ctx.bareOsPathconf + bareOsSyscall pathconf; syscalls.json schemaVersion 2 + errnoHints
- getconf: pathconf in BARE_OS_SYSCALL_OPS; _PC_*, _POSIX_THREAD_ATTR_STACKSIZE
- Docs: handbook ch.9, environment appendix, user manual ch.4, release-checklist, scripts README
- Tests: tokenize/glob cases in bare-os-booter test.js
Keep posix-conformance-matrix.json bareOsSyscallOps in sync with getconf (add pathconf).
2026-04-04 19:36:32 -04:00
Raven Scott