Commit Graph
348 Commits
Author SHA1 Message Date
Raven Scott e06dd0fb92 Updates to packages 2026-05-26 18:08:59 -04:00
Raven Scott 59593e43f6 Pear updates 2026-05-26 18:02:37 -04:00
Raven Scott 1e3d89e810 Fix pear 2026-05-26 17:58:46 -04:00
Raven Scott 014c70ad09 feat: introduce ctx.pear surface and /bin/pear for in-OS Pear development
Complete the full planned effort for the detailed ctx.bare code audit
and the new ctx.pear surface, delivering the ability to create, stage,
and integrate real Pear applications from within a booted Bare OS.

### Audit (ctx.bare)
- Performed exhaustive code audit of bare-os-ctx-bare.js (host import
  path, drive bundle eval + require.addon wrappers, referrer workarounds).
- Inventoried all manifest/bundle verifiers and related scripts.
- Researched manifest format, implicit tiering model, and dual loading
  strategy (JSON + .data.mjs).
- Deep analysis of the local Holepunch clone (bare-* and pear-* packages)
  to identify realistic guest vs host-delegate boundaries.
- Full cross-reference of call sites, greps, and historical pain points
  (pear:// referrer resolution, nativeHint handling, addon stubs).

### Implementation (ctx.pear)
- Added `pearEntries` tier to bare-module-manifest.json with initial
  high-value packages (pear-build, pear-bundle, pear-ref, etc.).
- Implemented `loadPearModuleManifest()` and `buildPearCtxObjectFromHost()`.
- Wired ctx.pear exposure through the booter into the guest context.
- Updated TypeScript definitions (`bare-os-ctx.d.ts`).

### User-Facing Surface
- Created full `/bin/pear` command with `help`, `info`, `list`, `init`
  (functional skeleton creation), and improved `stage` subcommands.
- Registered as Tier-1 command (now 183 total commands).
- Added man page and rebuilt coreutils (kernel + seeder).

### Agent Autonomy
- Created production-quality `pear-dev` agent skill.
- Added to skill seed list with cross-references to the appstore skill.

### P2P App Store Integration
- Updated appstore skill with explicit Pear development synergy section.
- Updated p2p-app-store design doc to document the new closed loop.
- Added cross-references in both skills and design documents.

### Verification & Hygiene
- Created `scripts/verify-pear-module-manifest-data.mjs`.
- Enhanced `verify-pear-no-static-node-import.mjs` with explicit pear
  command coverage.
- Integrated new verifier into release-checklist and agent hints.
- Performed comprehensive zero-TODO/scaffolding sweep across all new
  Pear artifacts (clean).
- Multiple full verification harness runs (all green).

### Documentation & Governance
- Added complete "Pear Development Environment" thread to feature-roadmap.md.
- Updated developer guide (Chapter 12).
- Maintained living plan document and detailed audit notes with full
  Implementation Log throughout.
- Updated command counts across READMEs and supporting docs.

All changes follow project governance:
- Bare-only guest constraints strictly observed
- Verifier-first discipline maintained
- Living plan + audit documents kept as single source of truth
- Production quality bar matching the completed P2P App Store feature

Plan items 04–21 completed.

See:
- docs/design/ctx-pear-surface-and-bare-audit-plan.md
- docs/audit/ctx-bare-audit-notes.md (full audit + implementation log)
2026-05-26 17:54:06 -04:00
Raven Scott afb591f219 Update key 2026-05-17 12:15:52 -04:00
snxraven 7de85968cc Raven Scott 2026-05-17 16:01:52 +00:00
Raven Scott 0f056c65e2 FIxes for SSH 2026-05-03 01:58:10 -04:00
Raven Scott b99da7bebc patch kex 2026-05-03 01:46:14 -04:00
Raven Scott 38e632e4b2 Updates 2026-04-27 21:56:22 -04:00
Raven Scott ff9ab499a2 New tooling
Entry: scripts/run-bin.mjs
Preload shim (must load first): scripts/run-bin-prelude.mjs
Harness: scripts/lib/run-bin-harness.mjs

Usage:

node scripts/run-bin.mjs ls -la /bin
node scripts/run-bin.mjs cat /etc/passwd
node scripts/run-bin.mjs --cmd sed -n '1,5p' /home/user/note.txt
node scripts/run-bin.mjs --json echo "hello"
printf 'a\nb\n' | node scripts/run-bin.mjs cat
npm run test:coreutils -- --suite basic
npm run test:coreutils -- --build --watch -- echo test   # rebuild + watch src/<cmd>.js (use `--buil
2026-04-27 21:32:50 -04:00
Raven Scott 93742db1cc Abort Controller for whois 2026-04-27 21:04:34 -04:00
Raven Scott 60e2efa64a Try to fix whois 2026-04-27 21:00:49 -04:00
Raven Scott 9074978c83 Chat updates 2026-04-27 20:14:01 -04:00
Raven Scott 12b9c1e200 Updates to Swarm Disk 2026-04-27 19:38:07 -04:00
Raven Scott 97ba72f639 Updates to Chat Protomux and Boot Splash 2026-04-27 19:26:44 -04:00
Raven Scott b133e8afc5 Late peers — Pending block reads were only sent to peers that existed when read() ran; your plan’s rebroadcast on join fixes that.
Protomux timing — Even with a connected socket, messages[0].send() must run after the channel is fully open. A bare send() right after chan.open() often does nothing useful until pairing finishes. Waiting on peer.chan.fullyOpened() (from Protomux) before each send fixed the standalone TCP check and matches real Hyperswarm behavior.
Code changes (packages/bare-os-booter/lib/swarm-disk.js)
read(index) — For each peer, schedule the read request with
peer.chan.fullyOpened().then(() => … send(index)) instead of sending immediately.
addPeer — After this.peers.add(peer), for every pending index in pendingReads, schedule the same fullyOpened() → send path so joiners get block requests once their channel is ready.
2026-04-27 19:02:24 -04:00
Raven Scott 193464bd2e Added a full ctx-level TCP connector path for telnet:
packages/bare-os-booter/index.js
Added ctx.bareOsTelnetConnect(host, port, opts) that prefers ctx.bare.bareTcp.createConnection, then new bareTcp.Socket(), with explicit diagnostics if unavailable.
packages/bare-os-coreutils/src/telnet.js already preferred ctx.bareOsTelnetConnect, so it now uses this richer path first automatically.
Added missing ssh command:

New file: packages/bare-os-coreutils/src/ssh.js
Supports -h/--help
Delegates to ctx.bareOsRunSshCli when present
Emits explicit nonzero unavailable/runtime diagnostics when absent
Registered command:
packages/bare-os-coreutils/lib/commands.mjs
Hardened HDMS create mount visibility:

packages/bare-os-booter/lib/hdms-manager.js
hdms create now verifies mount visibility via getMountMap().has(label) and errors non-silently if missing.
Success message now includes mount path (mounted=/mnt/<label>).
Fixed parser/semantics and explicit failures:

packages/bare-os-coreutils/src/kill.js
-1 is now treated as a target (not misparsed as signal shorthand), so kill -TERM -1 is explicit/non-silent.
packages/bare-os-coreutils/src/test.js
Numeric test comparisons now emit diagnostic + exit 2 for empty integer operand cases (keeps existing non-decimal behavior stable).
packages/bare-os-coreutils/src/crontab.js
Explicitly rejects stdin/fd-style installs (-, /dev/fd/*, /proc/self/fd/*) with nonzero error.
Updated ctx typings:

packages/bare-os-booter/lib/bare-os-ctx.d.ts
Added bareOsRunSshCli?
Added bareOsTelnetConnect?
2026-04-27 09:08:34 -04:00
Raven Scott 27292f2ba6 Network/auth/delegate hardening
hdms
Added ls alias to list.
Made delegate failures explicit and nonzero in packages/bare-os-coreutils/src/hdms.js.
Added nonzero error exit in packages/bare-os-booter/lib/hdms-manager.js.
git-pear
Implemented clone subcommand routing to git clone in packages/bare-os-coreutils/src/git-pear.js.
trustctl
Added ls alias to status/policy output in packages/bare-os-coreutils/src/trustctl.js.
oidc-publish
Added explicit unknown-subcommand handling and publish subcommand compatibility in packages/bare-os-coreutils/src/oidc-publish.js.
ssh-keygen
Wrapped delegate invocation with explicit error propagation in packages/bare-os-coreutils/src/ssh-keygen.js.
Added success output on generated keypair in packages/bare-os-booter/lib/ssh-keygen-cli.js.
sshd
Added -t config test mode and explicit exit semantics in packages/bare-os-booter/lib/bare-openssh.js.
Ensured wrapper sets exit code consistently in packages/bare-os-openssh/src/sshd.js.
telnet
Changed connector preference to use net.createConnection first when available, then syscall bridge fallback, in packages/bare-os-coreutils/src/telnet.js.
crontab -e flow

Implemented edit flow with VISUAL/EDITOR fallback, unlocked-state checks, temp file handling, install, and cleanup in packages/bare-os-coreutils/src/crontab.js.
Shell/runtime hardcore semantics

Added numeric brace range expansion {1..5} in packages/bare-os-booter/lib/shell-glob.js.
Enabled brace expansion by default unless explicitly disabled.
Added normalization for inline brace-expression tokens in packages/bare-os-booter/lib/shell.js.
Added arithmetic command-form handling for (( ... )) in packages/bare-os-booter/lib/shell.js.
Extended shell signal trap dispatch support for USR1/USR2 (in addition to INT/TERM) in packages/bare-os-booter/index.js.
Hardened kill command delivery validation in packages/bare-os-coreutils/src/kill.js.
Regression tests

Added new: packages/bare-os-coreutils/test/hardcore-bugs.test.mjs.
Extended shell tests in packages/bare-os-booter/test.js for:
default cmdsub behavior,
brace range expansion,
arithmetic command form.
Existing regression files still pass after updates.
2026-04-27 08:54:50 -04:00
Raven Scott d5535cdb65 Fixed command substitution behavior in shell expansion:
packages/bare-os-booter/lib/shell.js
$(...)/backticks are now enabled by default unless explicitly disabled via BARE_OS_SHELL_CMDSUBST=0|false|off.
Fixed sh -c exit/status propagation:

packages/bare-os-booter/lib/shell.js
Syncs BARE_OS_EXIT_STATUS after each executed statement so later commands in the same line (like echo $?) see the immediately previous status.
packages/bare-os-coreutils/src/sh.js
In -c mode, reads final BARE_OS_EXIT_STATUS back into ctx.exitCode for consistent result propagation.
Implemented trap behavior for requested scope (EXIT, INT, TERM):

packages/bare-os-booter/lib/shell.js
exit builtin now runs EXIT trap handler before requesting booter exit.
packages/bare-os-booter/index.js
Signal delivery for shell PID now triggers trap dispatch for INT/TERM.
Hardened ulimit -f invalid diagnostics:

packages/bare-os-coreutils/src/ulimit.js
Explicit invalid-value error for malformed -f setter input, with nonzero exit.
Supported-but-unimplemented setter values still return explicit unsupported-setter error.
2026-04-27 08:41:43 -04:00
Raven Scott dd5890b98c Core error propagation hardening:
Updated kernel-runner delegate deny path to emit on stderr in packages/bare-os-booter/lib/kernel-runner.js.
Added structured errno metadata (err.code) via vfsErr(...) and applied it to key VFS traversal/permission throws in packages/bare-os-booter/lib/vfs.js (ENOENT/EACCES paths).
Critical command exit-code fixes:

packages/bare-os-coreutils/src/ls.js
Tracks access/stat failures and sets nonzero exit when any operand fails.
packages/bare-os-coreutils/src/grep.js
Recursive walker now reports traversal/stat errors back to main flow so fatal status becomes 2.
packages/bare-os-coreutils/src/rm.js
-f only suppresses not-found style errors; permission/deny failures stay nonzero.
packages/bare-os-coreutils/src/chmod.js
Treats falsy/no-op backend chmod result as failure (nonzero).
packages/bare-os-coreutils/src/find.js
Added root-path preflight so missing root now reports explicit error + nonzero.
Minor quirks:

packages/bare-os-coreutils/src/xargs.js
-P0 now maps to bounded parallel cap (env/default cap), not forced serial.
packages/bare-os-coreutils/src/ulimit.js
-f with value now returns explicit unsupported-setter diagnostic + nonzero.
-f alone reports unlimited.
Regression tests:

Added packages/bare-os-coreutils/test/error-propagation.test.mjs covering:
grep missing file => exit 2
rm -f permission error => nonzero
find missing root => nonzero + diagnostic
ulimit -f 1M => explicit unsupported + nonzero
xargs -P0 bounded behavior
2026-04-27 08:31:31 -04:00
Raven Scott 237e0ef63a Added missing locale command support:
packages/bare-os-coreutils/src/locale.js
registered in packages/bare-os-coreutils/lib/commands.mjs
Fixed trustctl generated runtime mismatch:

added preamble wiring in packages/bare-os-coreutils/build.mjs (trustctl: ['p2p-suite.js'])
Improved help behavior:

packages/bare-os-coreutils/src/ssh-keygen.js (help, -h, --help, -?)
packages/bare-os-coreutils/src/oidc-publish.js (help detection across args)
Improved procstat empty-state UX while preserving JSON output:

packages/bare-os-coreutils/src/procstat.js
adds sourcePath and additive hint when process table is empty
Enforced stricter network behavior:

packages/bare-os-coreutils/src/whois.js
prefers ctx.httpFetch, adds bounded timeout via AbortController
clearer timeout/policy/offline error mapping
packages/bare-os-coreutils/src/telnet.js
strict-close behavior by default
--strict-close / --no-strict-close
BARE_OS_TELNET_STRICT_CLOSE env override
Implemented expected-item UX/compat updates:

packages/bare-os-coreutils/src/iconv.js: added -l/--list, improved encoding-pair error messaging
packages/bare-os-coreutils/src/crontab.js: clearer -l empty-file message
packages/bare-os-coreutils/src/getconf.js: added PAGE_SIZE / PAGESIZE aliases
packages/bare-os-coreutils/src/nproc.js: clarified help text
packages/bare-os-coreutils/src/time.js: clarified output comments/behavior
2026-04-27 08:22:57 -04:00
Raven Scott beaff551cf Fixed dd /dev/zero truncation in packages/bare-os-coreutils/src/dd.js
if=/dev/zero with finite count*bs now produces requested size directly (not limited by VFS 64KiB pseudo-file).
Raw stdout now uses bareOsEmitRaw, improving pipeline correctness.
Fixed pipeline capture path for FIFO-relevant commands

echo and cat now emit via bareOsEmitRaw (packages/bare-os-coreutils/src/echo.js, packages/bare-os-coreutils/src/cat.js).
Shell pipeline now injects bareOsBinWrite into child contexts when output is captured, so raw writes are captured (packages/bare-os-booter/lib/shell.js).
Improved xattr behavior in packages/bare-os-coreutils/src/xattr.js

Added -p/--print NAME.
-w now logs written attribute name for visible success feedback.
Roundtrip behaviors preserved with sidecar metadata format.
Improved ACL UX in:

packages/bare-os-coreutils/src/getfacl.js
Explicit source header for sidecar and mode fallback (non-compact mode).
packages/bare-os-coreutils/src/setfacl.js
Added -m/--modify ENTRY in addition to stdin blob and -b.
Implemented dynamic ulimit output in packages/bare-os-coreutils/src/ulimit.js

Reads /proc/bare_os/rlimits.json when available.
Supports -a, -n, -Sn, -Hn.
Added shuf -e support in packages/bare-os-coreutils/src/shuf.js.

Added split byte-suffix parsing (k/K/m/M/g/G) and attached -bSIZE support in packages/bare-os-coreutils/src/split.js.
2026-04-27 08:15:50 -04:00
Raven Scott b795397261 Added consistent non-empty version output for:
curl --version in packages/bare-os-coreutils/src/curl.js
wget --version in packages/bare-os-coreutils/src/wget.js
openssl version and openssl --version in packages/bare-os-coreutils/src/openssl.js
nano --version (via edit) in packages/bare-os-coreutils/src/edit.js
Updated tar compatibility in packages/bare-os-booter/lib/tar-cli.js:

Added tar --version
Added grouped short-flag parsing (-czf, etc.)
Added first-arg legacy cluster parsing (czf)
Added - archive target support for create/list/extract stream paths
Kept implementation as ustar subset (accepts z for compatibility parsing)
Fixed pipeline/timeout behavior:

Optimized head line mode to stop scanning early (no full split) in packages/bare-os-coreutils/src/head.js
Added bounded safety for yes cap in packages/bare-os-coreutils/src/yes.js
Hardened timeout so timed-out commands reliably produce exit code 124 in packages/bare-os-coreutils/src/timeout.js
Added scoped expr compatibility in shell:

Prevented pathname expansion of bare * only for expr arithmetic-token case in packages/bare-os-booter/lib/shell.js
Added glob option plumbing in packages/bare-os-booter/lib/shell-glob.js
2026-04-27 08:07:18 -04:00
Raven Scott db3aece34d Implemented all plan to-dos end-to-end, including regeneration of staged bins, and verified with build + booter tests.
What changed
Fixed /bin generation for AsyncFunction execution in build pipeline:
Added appctl preamble wiring (p2p-suite.js) in packages/bare-os-coreutils/build.mjs.
Added a stripRunExport transform to remove trailing export { run } from generated /bin scripts.
Updated cut delimiter parsing in packages/bare-os-coreutils/src/cut.js:
Supports both -d X and attached -dX forms (including -d' ').
Validates -d argument presence and enforces single-character delimiter.
Improved uname -a output in packages/bare-os-coreutils/src/uname.js:
Better release/version derivation from env and /etc/os-release.
Uses env-backed machine architecture fallback instead of hardcoded unknown where available.
Added help aliases in packages/bare-os-coreutils/src/git-pear.js:
git-pear --help and git-pear -h now map to help output.
Reduced persistent invalid timer spam in packages/bare-os-booter/lib/bare-cron.js:
Added per-file parse-error dedupe/rate-limiting (first-seen, content-change, then hourly).
Added actionable invalid timer format hint in log message.
2026-04-27 07:59:00 -04:00
Raven Scott 21ad2f415b fix(shell): propagate identity across pipeline child contexts for savevault 2026-04-27 07:35:48 -04:00
Raven Scott 337b616a74 ix(kernel-runner): execute direct .sh scripts via /bin/sh
Route direct .sh invocations (e.g. ./test.sh) through the shell frontend instead of JS evaluation, and add a regression test to cover direct .sh execution.
2026-04-27 02:54:04 -04:00
Raven Scott 099641a77f Remove No Longer needed data 2026-04-27 01:02:23 -04:00
Raven Scott 6bc5accb66 Agent Updates 2026-04-27 01:00:06 -04:00
Raven Scott aa81fbff5b Close the BareOS shell + Fish REPL roadmap tracker and ship the remaining
shell surfaces in-tree.
Roadmap / CI
- docs/data/shell-roadmap-features.json: all shell-001…shell-100 rows and
  P1–P8 phases marked implemented; note documents closure date and pointers.
- scripts/verify-shell-roadmap.mjs: validate JSON (schema 1, 8 phases,
  100 implemented items) plus existing source needles; wire npm run
  verify:shell-roadmap into root pretest (package.json).
- scripts/README.md: document verifier behavior.
Lexer & expansion (packages/bare-os-booter/lib)
- shell-lex.js: central lexShellLine; ANSI-C $'…' via decodeBareOsDollarQuote;
  keep diagnostics/tokenizer aligned with execution tokenizer.
- shell.js: stray reserved words at statement start → syntax error exit 2;
  optional [[ … ]] when BARE_OS_SHELL_DOUBLE_BRACKET=1 (==, !=);
  alias expansion before function dispatch (ordering tests);
  expandWordWithCmdSubst: balanced $(…) vs skipped $((…)); backtick
  command substitution when BARE_OS_SHELL_CMDSUBST; default ctx.execLine for
  nested cmdsubst when unset; index passthrough for DOUBLE_BRACKET env.
- shell-glob.js: ~login → HOME when USER matches, else /home/login (bounded
  login pattern); tests for pathname + execShellLine.
- shell-tokenizer.js: align with shell-lex detailed spans/modes where needed.
Completion / REPL
- completion-engine.js: completion depth / collectors per shell program work.
- packages/bare-os-booter/index.js: small wiring for shell env passthrough.
/bin/sh front-end
- packages/bare-os-coreutils/src/sh.js; kernel/bin/sh; seeder copies: stay in
  sync with shell behavior and env flags.
Tests
- packages/bare-os-booter/test.js: coverage for misplaced reserved words,
  gated [[ ]], alias vs function, ~user/~~ paths, $'…', cmdsubst $(…) and
  backticks, and related regressions.
Documentation
- docs/reference/shell-grammar.md: $'…', $(…) / backticks vs $((…)).
- handbook/09-posix-utilities-shell-and-vfs.md, environment appendix,
  shell-troubleshooting / shell-unsupported-behavior, release checklist,
  docs/reference/README.md: shell behavior and operator surfaces.
- developer-guide/19-how-to-fish-keybinding-completer.md: Fish keybinding /
  completer how-to (new).
Generated / synced artifacts
- kernel/lib/bare/manifest.json, kernel/share/man/man.json,
  kernel/lib/bare/shell-completion.json, posix_utilities.json,
  docs/audit/bundle-health.json: regenerated or synced with tooling.
- scripts/bench-shell-phases.mjs: bench script touch.
2026-04-27 00:41:06 -04:00
Raven Scott 5647491b08 Implement BareOS Zero-Trust Boot and Runtime Hardening (plan batches A–E).
Peer admission and bootstrap controls
- Fail closed when BARE_OS_PEER_ALLOWLIST_HEX is empty unless explicit
  break-glass BARE_OS_PEER_ALLOW_ALL=1.
- Treat BARE_OS_ZERO_TRUST_PROFILE=strict|security like strict admission
  posture alongside BARE_OS_PEER_ALLOWLIST_STRICT.
- Document BARE_OS_PEER_ALLOW_ALL and profile semantics; update boot trust
  model operator guidance.

Peer system seed and provenance
- In strict/security profile, peer system seed defaults off unless
  BARE_OS_PEER_SYSTEM_SEED is explicitly enabled (1/true/yes).
- Disable synthetic capability filling in strict profile; keep compat path
  when profile is not strict.
- Extend test.peer-system-seed.js for strict default-off and no-synthesis.

Path capability signer trust
- When BARE_OS_PATH_CAPABILITY_ENFORCE_READ is on, require trusted issuer
  if BARE_OS_PATH_CAPABILITY_REQUIRE_TRUSTED_SIGNER is set or profile is
  strict; wire verifyPathCapabilityEnvelopeTrusted into the primary deny
  path.
- Document BARE_OS_PATH_CAPABILITY_REQUIRE_TRUSTED_SIGNER and trusted key
  list usage in environment appendix.

Host delegates (least privilege)
- Under strict/security profile, empty BARE_OS_DELEGATE_ALLOW means deny-all
  delegates instead of allow-all; document behavior.
- Add delegate strict-profile test coverage.

Audit durability and telemetry hygiene
- Retain audit chain rows in memory and add bareOsAuditPersistRows for
  optional NDJSON persistence via VFS.
- Broaden var-log redaction for secret-shaped strings and env-like assignments.
- Emit boot.log security line when unsafe trust combinations are detected.

Release and CI gates
- Add scripts/verify-zero-trust-gates.mjs and npm run verify:zero-trust-gates.
- Document verifier in scripts/README.md and zero-trust steps in
  docs/release-checklist.md.

Tests
- Update bare-os-booter admission tests for allow-all and empty-allowlist
  messaging.
- Relax brittle man.json page-count equality to a minimal sanity check to
  avoid brittle/os.cwd brittle failures on inventory drift.

Verification (local): npm run verify:zero-trust-gates; npm run test -w
bare-os-booter; peer-system-seed brittle lane as applicable.

Plan file (.cursor/plans/zero-trust-boot-runtime-100-plan_*.plan.md) was not
edited per instructions.
2026-04-26 23:49:54 -04:00
Raven Scott 04b3a96dfb harden booter reliability lanes and enforce flake/CI gates
Execute the reliability/flake hardening run across booter shell-adjacent suites,
focusing on deterministic assertions, unskipped first-party reliability tests,
repeat-run flake detection, and release/CI gate enforcement.

- Unskip and stabilize first-party reliability suites:
  - `packages/bare-os-booter/test.bare-os-www.js`
  - `packages/bare-os-booter/test.bare-os-ssh-holesail.js`
  - convert previously skipped coverage into active deterministic checks
  - add lifecycle hook stubs in suite test context for managed runtime paths
  - reduce network/port race sensitivity by shifting fragile HTTP-bound checks
    toward deterministic service/config/state validation where needed

- Replace weak reliability assertions in booter shell tests:
  - remove broad `t.ok(true)`-style pass-through checks
  - strengthen checks around here-doc/script mode and `sh -c` paths in
    `packages/bare-os-booter/test.js` with explicit exit/error constraints

- Add reliability hard-gate automation:
  - `scripts/verify-reliability-gates.mjs`
  - fail when `test.skip` exists in first
2026-04-26 23:39:07 -04:00
Raven Scott e2b721d19f complete shell 100-item plan across parser, execution, security, and docs
Implement the full BareOS shell roadmap end-to-end, including grammar/tokenization
diagnostics, expansion/runtime hardening, execution graph tooling, builtins/job-control
stability, policy/sandbox controls, and release/traceability documentation updates.

- Add shell grammar baseline and diagnostics primitives:
  - introduce `docs/reference/shell-grammar.md` with lexer modes and EBNF contract
  - add rich diagnostic tokenizer output (mode + span metadata) via `tokenizeBareShellLineDetailed`
  - export structured parse snapshot helpers (`bareOsShellAstSnapshot`) and shell error kinds
  - add determinism coverage for tokenizer and AST snapshot outputs

- Harden expansion semantics and guardrails:
  - enforce expansion byte budgets (`BARE_OS_SHELL_EXPANSION_MAX_BYTES`)
  - add expansion trace hooks (`BARE_OS_SHELL_EXPANSION_TRACE`) with stage-level rows
  - add expansion recursion depth limits (`BARE_OS_SHELL_EXPANSION_MAX_DEPTH`)
  - tighten POSIX-mode arithmetic invalid-token diagnostics
  - preserve declared expansion ordering and document it in code/docs

- Extend redirection/pipeline execution model:
  - add normalized redirection planner (`planShellRedirections`) independent of side effects
  - add execution graph builder/debug surface (`buildShellExecutionGraph`)
  - support `<<-` operator in tokenizer/parser paths
  - add pipeline stage timeout safety (`BARE_OS_SHELL_PIPELINE_STAGE_TIMEOUT_MS`)
  - keep pipefail/pipestatus behavior verified with integration tests

- Improve builtins and control-flow reliability:
  - expand `read` builtin support:
    - `-r` raw mode
    - `-d` single-char delimiter
    - `-t` timeout semantics
  - refine wait/jobs semantics:
    - stable `jobs -l` parseable format expectations
    - synthetic pid mapping (`wait 410x`) and `wait all` support
  - keep trap registration/listing behavior deterministic and test-covered
  - add trap signal dispatch helper (`dispatchShellTrapSignal`) with normalization

- Add security and policy enforcement hooks:
  - command deny/allow policy gates:
    - `BARE_OS_SHELL_DENY_COMMANDS`
    - `BARE_OS_SHELL_ALLOW_COMMANDS`
  - sandbox mode (`BARE_OS_SHELL_SANDBOX`) to block external command execution
  - redirect path safety guard (`BARE_OS_SHELL_REDIRECT_GUARD`) for pseudo-path/traversal risks
  - emit structured shell audit event rows (`ctx.shellAuditEvents`) for start/error/finish

- Improve interactive UX resilience:
  - add prompt-hook timeout protection in fish readline:
    - `resolveShellPromptHookSegment`
    - env control `BARE_OS_SHELL_PROMPT_HOOK_TIMEOUT_MS`
  - ensure prompt segment resolution is non-blocking and safe on timeout/error

- Add reliability/performance artifacts and shell fast lane:
  - add `scripts/bench-shell-phases.mjs` for shell microbench sanity checks
  - add `scripts/gen-shell-reliability-report.mjs` and generate reliability JSON artifact
  - add root scripts:
    - `test:shell-fast`
    - `bench:shell`
    - `report:shell-reliability`

- Expand shell-focused docs and traceability:
  - add:
    - `docs/reference/shell-unsupported-behavior.md`
    - `docs/reference/shell-troubleshooting.md`
  - add contributor guides:
    - `developer-guide/17-how-to-add-shell-builtin.md`
    - `developer-guide/18-how-to-add-shell-grammar-feature.md`
  - update indexes/traceability/release gate docs:
    - `docs/reference/README.md`
    - `docs/reference/posix-issue7-traceability.md`
    - `docs/reference/environment-and-posix-appendix.md`
    - `docs/release-checklist.md`
    - `developer-guide/README.md`
    - `scripts/README.md`

- Add and update shell regression tests in `packages/bare-os-booter/test.js` for:
  - tokenizer spans/modes and deterministic output
  - AST snapshot schema/shape
  - redirection planner and execution graph behavior
  - expansion trace and strict arithmetic paths
  - `<<-` support
  - pipeline stage timeout handling
  - `read` delimiter/raw/timeout semantics
  - jobs/wait parseability and selection semantics
  - trap dispatch and normalization behavior
  - policy/sandbox/redirect-guard/audit-event pathways

Validation:
- `npm run test -w bare-os-booter`
- `npm run test:shell-bracket -w bare-os-booter`
- `npm run test:shell-fast`
- `npm run report:shell-reliability`
2026-04-26 23:17:20 -04:00
Raven Scott ef5a30fbef TextEncoder Fix 2026-04-26 22:34:04 -04:00
Raven Scott d286ce19b5 chore(plan): cancel end-to-end seeder-to-booter smoke harness task
test(protocol): add deterministic MBR failover-key coverage
docs(protocol): align package-bare-os-protocol version to 0.9.1
test(booter): add MBR corruption and wrong-topic smoke fixtures
test(peer-seed): add strict pre-MBR bare_os.capabilities negotiation check
feat(seeder): validate BARE_OS_SEED_REQUIRE_MBR_LABELS
feat(seeder): validate BARE_OS_SEED_CAPABILITY_ATTESTATION_JSON schema
docs(boot-policy): add requireProtocolPackageMin 0.9.1 example
test(kernel): cover boot.policy denySeedRpcMethods behavior
test(protocol): add app/cap/chat/meshdrop channel compatibility fixture
test(swarm-disk): cover duplicate Protomux channel null-return path
test(protocol): add 11-word kernelCapabilityWords round-trip fixture
docs(schema): add mbr-layout schema and validate seeder examples
test(protocol): add topicKey() golden hash fixture
docs(trust): document block-0 trust assumptions in boot docs
feat(seeder): add discovery.flushed readiness logging
feat(booter): record peer discovery timings in boot-perf.json
feat(integration): add local testnet mode to integration lab smoke
test(booter): add Hyperswarm connection-budget env regression coverage
test(booter): add swarm plus Corestore suspend/resume integration coverage
feat(booter): mirror swarm ban events into host audit logs
feat(booter): add direct-peer boot via BARE_OS_BOOT_JOIN_PEER_HEX
feat(seeder): pass BARE_OS_SEED_MAX_PEERS to Hyperswarm
feat(seeder): log drive.version and discoveryKey at startup
test(booter): add Hyperdrive.checkout read-only boot probe coverage
feat(booter): prefetch /boot/init.js before kernel handoff
feat(booter): add optional /bin warm replication via downloadDiff
feat(seeder): add manifestPaths SHA-256 generation in stage-kernel-tree
test(peer-seed): cover helper-served block-0 after seeder exit
feat(protocol): add Protomux cork batching for initial channel sends
test(boot-graph): compare kernel/init labels with booter graph proc
docs(boot-policy): add v9-v11 schema examples
feat(release): add requireInitJsSha256 fixture generation step
test(vfs): add BARE_OS_VFS_SYSTEM_RO_ALIAS coverage
test(vfs): strengthen system-drive write-deny path coverage
feat(identity): add personal-drive namespace export/import docs and tests
test(booter): add guest-to-login warm cache invalidation regression
test(vfs): add guest deny coverage for /.bare sensitive paths
test(coreutils): add cross-drive mv failure injection coverage
test(vfs): add .bareos_empty round-trip coverage across mkdir/rmdir/cp/git-fs
test(vfs): add /dev/shm quota enforcement coverage
test(proc): add /proc/bare_os/index.json sortedness and schema checks
test(vfs): add warm read cache invalidation on replication growth
docs(ctx): document bareOsInvalidateWarmReadCaches(reason)
test(kernel): add BARE_OS_BOOT_DRY_RUN behavior coverage
docs(posix): add dashboard rows for all COREUTILS_COMMANDS
feat(curl): expand -w variables beyond http_code/url_effective/size_download
feat(wget): mark -N timestamping as explicit unsupported error
feat(curl): plumb mutual TLS cert/key intent to ctx.httpFetch metadata
feat(shuf): add deterministic seed mode via BARE_OS_SHUF_SEED
docs(sort): document -M month-sort as unsupported
feat(grep): add explicit -E and -G mode handling
test(sed): add Open Group Issue 7 golden fixtures
test(awk): add getline VFS regressions for missing/repeat/boundary cases
test(shell): add non-interactive here-doc coverage
test(shell): add trap delivery coverage for synthetic PIDs/job IDs
test(shell): add set -e compound-body behavior coverage
docs(shell): strengthen read builtin opt-in guidance
test(env): add Bare-runtime coverage for -S and --env-file
docs(man): add examples for pathcap-verify pkg-swarm-index corestorectl
test(identity): add account/vault backup-restore smoke coverage
feat(audit): add tamper detection verification for audit chain rows
test(peer-admission): cover strict empty allowlist deny behavior
test(peer-admission): add denylist precedence over allowlist coverage
test(peer-admission): add BARE_OS_PEER_REQUIRE_CAPS_JSON metadata checks
docs(identity): add trusted-key rotation example for path capabilities
feat(schema): tighten extensionSignerPinsV2-V4 hash validation
test(delegate): add allowlist negative cases for curl/wget/git/hrpc/systemctl
test(proc): extend /proc/self/environ redaction key coverage
docs(security): add peer-assisted block-0 mirroring threat-model notes
feat(bench): add boot budget trend output from real booter phases
test(baretop): align fixture coverage with /proc snapshot key set
test(metrics): validate /proc/bare_os/metrics.prom OpenMetrics shape
docs(ops): add structured seeder NDJSON examples
test(replication): add live stall-hint coverage for no_peers/length_unavailable/ok
docs(release): add corestore-snapshot workflow to checklist
docs(ops): add mirror-drive experiment utility to maintainer workflow
test(booter): add monitor progress coverage for replication live sketch
feat(seeder): validate DHT bootstrap address class JSON inputs
docs(network): add HYPERSWARM_BOOTSTRAP testnet operator guidance
chore(root): add deterministic test:integration script
docs(ci): add local CI runbook for no-.github environments
docs(release): add npm run test:bare after npm test
feat(verify): add protocol docs/package version parity checker
feat(verify): enforce feature-roadmap canonical path consistency
feat(lockfile-drift): add tier-1 strict fail option for mismatches
docs(lockfile-drift): add udx-native and blind-peering upgrade workflow notes
docs(cli-parity): add bare-fetch upstream issue tracking row
feat(bundle-health): generate per-tier bundle size regression thresholds
feat(doc-contracts): verify handbook references to current proc schema versions
feat(pretest): add validate-mermaid-syntax gate
feat(probe): add bare-runtime top-25 critical command lane
docs(protocol): update capability-word prose from bits..bits5 to current words
docs(two-drive): document /tmp /var/log and account-prefix routing
docs(security): add concise boot trust model page and links
docs(dev-guide): add P2P lab cookbook section
docs(dev-guide): add how-to for adding seed RPCs
docs(dev-guide): add how-to for adding /proc/bare_os nodes
docs(dev-guide): add /bin utility checklist for man/posix/build/parity/tests
docs(user-manual): add short What BareOS is not section
2026-04-26 22:28:21 -04:00
Raven Scott a553a4e4a7 further shell updates 2026-04-26 16:24:57 -04:00
Raven Scott d3001aba9c Updates 2026-04-26 16:12:58 -04:00
Raven Scott 20907247cc Enable BARE_OS_SH_EXTENDED_PROFILE default 2026-04-26 15:56:01 -04:00
Raven Scott 2dc388ffd0 Updates 2026-04-26 15:49:55 -04:00
Raven Scott 4cdd1569af sh: add POSIX-like -c support and complete shell subset gaps
Implement sh -c COMMAND [NAME [ARG...]] in /bin/sh, add shell function
declarations/invocation support, expand export semantics (NAME, NAME=value, -p),
and make until/loop-control behavior available by default. Add focused shell
tests and update sh man-page option docs.
2026-04-26 15:28:29 -04:00
Raven Scott 9471788f6b Core Utils Breakdown Tests 2026-04-26 15:05:19 -04:00
Raven Scott e59c57e538 Align Bare OS with Holepunch stack across runtime, P2P, storage, trust, and ops surfaces
Implement the 50-point Holepunch alignment roadmap with a first-pass delivery across coreutils commands, policy examples, audit tooling, and docs. This adds new operator CLIs (appctl/corestorectl/ctxbaredoctor/dhtctl/trustctl), tiered catalog and runtime-compat reports, release-checklist integration, contributor guidance, and kernel/seeder mirrored artifacts for app registry, trust, network services, corestore namespaces, and update manifest workflows.
2026-04-26 08:57:34 -04:00
Raven Scott 6f90952611 Updates to tools 2026-04-26 08:40:53 -04:00
Raven Scott 67743621a5 Updates for btop 2026-04-26 08:28:12 -04:00
Raven Scott 194fbd2c67 fix(swarmtop): expose and render full peer identity/details from proc surfaces 2026-04-26 08:02:37 -04:00
Raven Scott 8c3d5f8795 feat(p2p): introduce shared p2p transport/service stack and launch swarmtop, meshdrop, taskmesh, peernote, and hypershell-board 2026-04-26 07:48:55 -04:00
Raven Scott b1736b7a7a booter: apply 20-point Holepunch resilience and observability upgrades
add grouped retry orchestration, duplicate arbitration hardening, and transport-aware peer scoring
expand IPC with readiness gates, soft backpressure, request deadlines, batching, and method circuit-breakers
strengthen worker/control-plane lifecycle via idempotent termination, orphan detection, and liveness checkpoint helpers
improve lifecycle correctness with state-machine + transactional hook infrastructure and adaptive quiesce utilities
enrich operator surfaces/tests for degraded readiness, crash-safe telemetry patterns, and contract-level regressions
2026-04-26 07:22:31 -04:00
Raven Scott dcf07fbaa0 booter: harden lifecycle and IPC; add richer swarm/process observability
add deterministic connection arbitration, reconnect gating, and localhost multi-address fallback
introduce IPC request/response correlation, soft backpressure thresholds, readiness waits, and batch sends
add lifecycle state machine + transactional suspend/resume hooks + replication quiesce helper
enrich process/job lifecycle phases and add close/crash journaling hooks
extend metrics live surface with degraded readiness and aggregate pressure views
2026-04-26 07:15:34 -04:00
Raven Scott d95c135dd9 booter updates 2026-04-26 07:05:24 -04:00
Raven Scott 47da0b8531 normalize initd state modeling and stabilize synthetic initd PID mapping
add IPC backpressure/drop telemetry and jittered swarm retry tiers
harden socket bridge timeout/lifecycle handling and replace recv polling with readiness signaling
improve shell background child isolation and worker offload fallback/output diagnostics
add var-log fast append path to reduce read-concat-write amplification
2026-04-26 06:57:07 -04:00
Raven Scott dfcd36dc58 /proc/bare_os/initd_readiness.json schema 2 2026-04-26 06:44:02 -04:00