Close the BareOS shell + Fish REPL roadmap tracker and ship the remaining

shell surfaces in-tree.
Roadmap / CI
- docs/data/shell-roadmap-features.json: all shell-001…shell-100 rows and
  P1–P8 phases marked implemented; note documents closure date and pointers.
- scripts/verify-shell-roadmap.mjs: validate JSON (schema 1, 8 phases,
  100 implemented items) plus existing source needles; wire npm run
  verify:shell-roadmap into root pretest (package.json).
- scripts/README.md: document verifier behavior.
Lexer & expansion (packages/bare-os-booter/lib)
- shell-lex.js: central lexShellLine; ANSI-C $'…' via decodeBareOsDollarQuote;
  keep diagnostics/tokenizer aligned with execution tokenizer.
- shell.js: stray reserved words at statement start → syntax error exit 2;
  optional [[ … ]] when BARE_OS_SHELL_DOUBLE_BRACKET=1 (==, !=);
  alias expansion before function dispatch (ordering tests);
  expandWordWithCmdSubst: balanced $(…) vs skipped $((…)); backtick
  command substitution when BARE_OS_SHELL_CMDSUBST; default ctx.execLine for
  nested cmdsubst when unset; index passthrough for DOUBLE_BRACKET env.
- shell-glob.js: ~login → HOME when USER matches, else /home/login (bounded
  login pattern); tests for pathname + execShellLine.
- shell-tokenizer.js: align with shell-lex detailed spans/modes where needed.
Completion / REPL
- completion-engine.js: completion depth / collectors per shell program work.
- packages/bare-os-booter/index.js: small wiring for shell env passthrough.
/bin/sh front-end
- packages/bare-os-coreutils/src/sh.js; kernel/bin/sh; seeder copies: stay in
  sync with shell behavior and env flags.
Tests
- packages/bare-os-booter/test.js: coverage for misplaced reserved words,
  gated [[ ]], alias vs function, ~user/~~ paths, $'…', cmdsubst $(…) and
  backticks, and related regressions.
Documentation
- docs/reference/shell-grammar.md: $'…', $(…) / backticks vs $((…)).
- handbook/09-posix-utilities-shell-and-vfs.md, environment appendix,
  shell-troubleshooting / shell-unsupported-behavior, release checklist,
  docs/reference/README.md: shell behavior and operator surfaces.
- developer-guide/19-how-to-fish-keybinding-completer.md: Fish keybinding /
  completer how-to (new).
Generated / synced artifacts
- kernel/lib/bare/manifest.json, kernel/share/man/man.json,
  kernel/lib/bare/shell-completion.json, posix_utilities.json,
  docs/audit/bundle-health.json: regenerated or synced with tooling.
- scripts/bench-shell-phases.mjs: bench script touch.
This commit is contained in:
Raven Scott
2026-04-27 00:41:06 -04:00
parent 5647491b08
commit aa81fbff5b
31 changed files with 11675 additions and 9943 deletions
+26 -1
View File
@@ -8,6 +8,7 @@ promise of full POSIX `sh` parity.
- `normal`: unquoted words and operators
- `single`: `'...` literal text (no interpolation)
- **`$'…'`**: ANSI-C style escapes in the lexer (`\n`, `\t`, `\\`, `\'`, `\xHH`, …); emitted as a single-quoted segment (no further `$` expansion inside).
- `double`: `\"...\"` with escapes and expansions
- `arith`: `$(( ... ))`
- `heredoc`: `<<WORD` / `<<-WORD` delimiter capture
@@ -15,7 +16,7 @@ promise of full POSIX `sh` parity.
## Token Classes
- `word`
- `op` (`;`, `&&`, `||`, `|`, `<`, `>`, `>>`, `2>`, `2>>`, `2>&1`, `<<`, `<<<`, `(`, `)`, `{`, `}`)
- `op` (`;`, `;;`, `;&`, `&&`, `||`, `|`, `|&`, `<`, `>`, `>>`, `>&`, `2>`, `2>>`, `2>&1`, `<<`, `<<-`, `<<<`, `(`, `)`, `{`, `}`)
All diagnostic tokens should include:
@@ -78,5 +79,29 @@ ws = { " " | "\t" | "\n" } ;
## Notes
- Parsing/expansion/runtime error phases should remain distinguishable.
- **`( … )` grouped list**: in-tree execution is a **parenthesized compound list** in the **current** guest environment (**no host `fork`**, no separate subshell process). Enable with **`BARE_OS_SHELL_POSIX_MODE`** or **`BARE_OS_SHELL_GROUPING`** (see environment appendix). This is **not** POSIX subshell isolation for variables or `cd`.
- **`;;`** terminates each `case` arm (lexer may emit one `;;` token or two `;` operators; execution accepts both).
- **`select`** is parsed as a single top-level statement through **`select … in …; do …; done`** (semicolons before **`do`** do not split the list); execution still returns an explicit **unsupported** error. **`[[ … ]]`** is rejected unless **`BARE_OS_SHELL_DOUBLE_BRACKET=1`**, in which case only **`[[ WORD == WORD ]]`** / **`[[ WORD != WORD ]]`** is accepted.
- Stray closing / middle reserved words (**`then`**, **`else`**, **`fi`**, **`do`**, **`done`**, **`esac`**, **`in`**, **`elif`**) at the start of a statement are rejected with **`shell: syntax error: reserved word '…' cannot start a statement`** (exit **2**).
## `/bin/sh` frontend
- **`sh [-euvxC] [-c COMMAND | SCRIPT | -s]`** maps **`-e`** → **`BARE_OS_SHELL_ERREXIT`**, **`-u`** → **`BARE_OS_SHELL_NOUNSET`**, **`-x`/`-v`** → **`BARE_OS_SHELL_XTRACE`** (same hook), **`-C`** → **`BARE_OS_SHELL_NOCLOBBER`** (honored when noclobber checks exist).
- Line continuation **`\\\n`** is merged before **`execLine`** evaluates a line.
## Expansion (selected)
| Form | Requires |
|---|---|
| **`$'…'`** | — (lexer decode; bounded escape set) |
| **`$(…)`** / **`` `` ``** | **`BARE_OS_SHELL_CMDSUBST=1`** (bounded; **`$((…))`** is arithmetic, not command substitution) |
| **`${#name}`** | Length of **`$name`** |
| **`${!name}`** | **`BARE_OS_SHELL_INDIRECT_EXPANSION`** |
| **`${name:off:len}`** | **`BARE_OS_SHELL_PARAM_EXPANSION_V3`** |
| **`${name//pat/repl}`** | **`BARE_OS_SHELL_PARAM_EXPANSION_V3`** |
## Diagnostics token stream
Structured diagnostics use **`packages/bare-os-booter/lib/shell-lex.js`** (**`lexShellLine`**) and **`tokenizeBareShellLineDetailed`** for stable spans aligned with **`tokenize`**.
- This grammar intentionally omits unsupported POSIX constructs until adopted.
- Deterministic AST snapshots should use this document as the canonical shape reference.