Updates to CURL and WGET

This commit is contained in:
Raven Scott
2026-04-03 17:36:57 -04:00
parent 77638c4138
commit 9d836e8072
16 changed files with 561 additions and 140 deletions
+1
View File
@@ -1,4 +1,5 @@
node_modules/ node_modules/
# Legacy repo-root Corestore; default host data is ~/.bare-os/corestore/
data/ data/
coverage/ coverage/
.DS_Store .DS_Store
+9 -8
View File
@@ -1,6 +1,6 @@
# bare-operating-system — full codebase reference # bare-operating-system — full codebase reference
This document describes **every first-party file** in the repository: purpose, contents, dependencies, and how pieces connect. It does **not** enumerate third-party code under `node_modules/` or VCS metadata under `.git/`. Runtime directories `data/` and `packages/*/.test-data/` are **gitignored** and hold Corestore/RocksDB state when you run the apps or tests; they are not source files. This document describes **every first-party file** in the repository: purpose, contents, dependencies, and how pieces connect. It does **not** enumerate third-party code under `node_modules/` or VCS metadata under `.git/`. **Host** Corestore defaults live under **`~/.bare-os/corestore/`** (seeder and booter). Legacy **`data/`** at the repo root and **`packages/*/.test-data/`** remain **gitignored** if present; brittle tests use `.test-data/` under the booter package. None of these are source files.
For a **narrative how-to** on writing `async function run(ctx, argv)` and `start(ctx)` scripts, extending `/bin`, and separating host Pear code from in-image `AsyncFunction` execution, see [developer-guide/README.md](developer-guide/README.md). For a **narrative how-to** on writing `async function run(ctx, argv)` and `start(ctx)` scripts, extending `/bin`, and separating host Pear code from in-image `AsyncFunction` execution, see [developer-guide/README.md](developer-guide/README.md).
@@ -104,7 +104,7 @@ No runtime dependencies at the root; all stack deps live in workspace packages.
## 4. Root: [README.md](README.md) ## 4. Root: [README.md](README.md)
User-oriented documentation: project goal, layout table, prerequisites (Node 20+, Pear/Bare), `npm ci`, `npm test`, how to run seeder and booter with `node index.js`, environment variables, Corestore path semantics (repo-root `data/`), placeholder `pear://` table, protocol summary (`bare-os-v1`, MBR layout), license pointer. User-oriented documentation: project goal, layout table, prerequisites (Node 20+, Pear/Bare), `npm ci`, `npm test`, how to run seeder and booter with `node index.js`, environment variables, Corestore path semantics (`~/.bare-os` defaults), placeholder `pear://` table, protocol summary (`bare-os-v1`, MBR layout), license pointer.
Narrative handbook (architecture diagrams, chapter walkthrough): [handbook/README.md](handbook/README.md). Per-workspace overviews: [packages/bare-os-protocol/README.md](packages/bare-os-protocol/README.md), [packages/bare-os-coreutils/README.md](packages/bare-os-coreutils/README.md), [packages/bare-os-seeder/README.md](packages/bare-os-seeder/README.md), [packages/bare-os-booter/README.md](packages/bare-os-booter/README.md), [kernel/README.md](kernel/README.md), [scripts/README.md](scripts/README.md). Narrative handbook (architecture diagrams, chapter walkthrough): [handbook/README.md](handbook/README.md). Per-workspace overviews: [packages/bare-os-protocol/README.md](packages/bare-os-protocol/README.md), [packages/bare-os-coreutils/README.md](packages/bare-os-coreutils/README.md), [packages/bare-os-seeder/README.md](packages/bare-os-seeder/README.md), [packages/bare-os-booter/README.md](packages/bare-os-booter/README.md), [kernel/README.md](kernel/README.md), [scripts/README.md](scripts/README.md).
@@ -121,7 +121,7 @@ Short Apache License, Version 2.0 header: copyright year 2026, standard AS-IS di
Ignores: Ignores:
- `node_modules/` - `node_modules/`
- `data/` (Corestore for seeder/booter at repo root) - `data/` (legacy; optional local Corestore if you still keep trees here — defaults now use `~/.bare-os`)
- `coverage/` - `coverage/`
- `.DS_Store` - `.DS_Store`
- `*.log` - `*.log`
@@ -275,7 +275,7 @@ Pear-safe path resolution (same idea as Holepunch [pear-rti](https://github.com/
- **`packageRootDir(metaUrl)`** — If `import.meta.url` is `file:`, `path.dirname(fileURLToPath(...))` via **`node:url`** (so `file:` is only passed to `fileURLToPath`). If `pear:` / other, `global.Pear.constructor.RTI.mount`, else `Pear.config.swapDir`, else `process.cwd()`. - **`packageRootDir(metaUrl)`** — If `import.meta.url` is `file:`, `path.dirname(fileURLToPath(...))` via **`node:url`** (so `file:` is only passed to `fileURLToPath`). If `pear:` / other, `global.Pear.constructor.RTI.mount`, else `Pear.config.swapDir`, else `process.cwd()`.
- **`defaultKernelRoot(pkgRoot, metaUrl)`** — `BARE_OS_KERNEL_ROOT` or `path.join(pkgRoot, 'kernel')` (vendored copy under the seeder package for Pear bundles). - **`defaultKernelRoot(pkgRoot, metaUrl)`** — `BARE_OS_KERNEL_ROOT` or `path.join(pkgRoot, 'kernel')` (vendored copy under the seeder package for Pear bundles).
- **`defaultSeedCorestorePath(pkgRoot, metaUrl)`** — `BARE_OS_SEED_STORE` or `../../data` from `pkgRoot` when `file:`, else `pkgRoot/data` for Pear checkout. - **`defaultSeedCorestorePath(pkgRoot, metaUrl)`** — `BARE_OS_SEED_STORE` or `path.join(hostDataRoot(), 'corestore', 'seeder')` where **`hostDataRoot()`** is `BARE_OS_HOST_DATA` (resolved) or `~/.bare-os`. Signature keeps `pkgRoot` / `metaUrl` for callers; defaults do not use them.
### 11.3 [packages/bare-os-seeder/index.js](packages/bare-os-seeder/index.js) ### 11.3 [packages/bare-os-seeder/index.js](packages/bare-os-seeder/index.js)
@@ -325,7 +325,7 @@ Pear-safe path resolution (same idea as Holepunch [pear-rti](https://github.com/
### 12.2 [packages/bare-os-booter/lib/paths.js](packages/bare-os-booter/lib/paths.js) ### 12.2 [packages/bare-os-booter/lib/paths.js](packages/bare-os-booter/lib/paths.js)
- **`packageRootDir(metaUrl)`** — Same as seeder (Pear RTI / `swapDir` / `cwd`). - **`packageRootDir(metaUrl)`** — Same as seeder (Pear RTI / `swapDir` / `cwd`).
- **`defaultBootCorestorePath`** / **`defaultLocalSeedCorestorePath`** — `../../data` when `file:`, else `pkgRoot/data`. - **`defaultBootCorestorePath`** / **`defaultLocalSeedCorestorePath`** — `BARE_OS_BOOT_STORE` / `BARE_OS_LOCAL_SEED`, or under **`hostDataRoot()`** (`BARE_OS_HOST_DATA` or `~/.bare-os`): `corestore/booter` and `corestore/seeder` respectively. Same signature stability as the seeder helper.
### 12.3 [packages/bare-os-booter/index.js](packages/bare-os-booter/index.js) ### 12.3 [packages/bare-os-booter/index.js](packages/bare-os-booter/index.js)
@@ -492,8 +492,9 @@ sequenceDiagram
| Variable | Used by | Meaning | | Variable | Used by | Meaning |
| ------------------------- | ---------- | ---------------------------------------------------------------------------------------------- | | ------------------------- | ---------- | ---------------------------------------------------------------------------------------------- |
| `BARE_OS_KERNEL_ROOT` | Seeder | Absolute path to kernel tree (default: `repo/kernel`) | | `BARE_OS_KERNEL_ROOT` | Seeder | Absolute path to kernel tree (default: `repo/kernel`) |
| `BARE_OS_SEED_STORE` | Seeder | Corestore directory (default: `repo/data/corestore-seeder`) | | `BARE_OS_HOST_DATA` | paths | Base directory for host state (default `~/.bare-os`; Corestore dirs live under `corestore/`) |
| `BARE_OS_BOOT_STORE` | Booter | Corestore for boot side (default: `repo/data/corestore-booter`) | | `BARE_OS_SEED_STORE` | Seeder | Corestore directory (default: `~/.bare-os/corestore/seeder`) |
| `BARE_OS_BOOT_STORE` | Booter | Corestore for boot side (default: `~/.bare-os/corestore/booter`) |
| `BARE_OS_BOOT_TIMEOUT_MS` | Booter | Wall-clock budget for peer wait + network boot (default `60000`) | | `BARE_OS_BOOT_TIMEOUT_MS` | Booter | Wall-clock budget for peer wait + network boot (default `60000`) |
| `BARE_OS_NO_SPLASH` | Booter | If `1`, skip TTY splash (plain logs / non-TTY behavior unchanged) | | `BARE_OS_NO_SPLASH` | Booter | If `1`, skip TTY splash (plain logs / non-TTY behavior unchanged) |
| `BARE_OS_LOCAL_SEED` | paths | Overrides local seed path helper (`defaultLocalSeedCorestorePath`); booter does not local-boot | | `BARE_OS_LOCAL_SEED` | paths | Overrides local seed path helper (`defaultLocalSeedCorestorePath`); booter does not local-boot |
@@ -524,7 +525,7 @@ Reference: Open Group POSIX.1-2017 utilities index; GNU coreutils (external) for
- **No** `node_modules` documentation (upstream packages). - **No** `node_modules` documentation (upstream packages).
- **No** line-by-line `package-lock.json` (machine-generated). - **No** line-by-line `package-lock.json` (machine-generated).
- **No** committed `data/` or `.test-data/` binary stores (runtime artifacts). - **No** committed host Corestore trees, legacy `data/`, or `.test-data/` binary stores (runtime artifacts).
--- ---
+7 -2
View File
@@ -54,7 +54,8 @@ Runs workspace tests (`brittle-node` / `brittle-bare` where configured).
Optional: Optional:
- `BARE_OS_KERNEL_ROOT` — absolute path to a kernel tree (defaults to repo `kernel/`). - `BARE_OS_KERNEL_ROOT` — absolute path to a kernel tree (defaults to repo `kernel/`).
- `BARE_OS_SEED_STORE` — Corestore directory (default `./data/corestore-seeder`). - `BARE_OS_HOST_DATA` — base directory for host state (default `~/.bare-os`).
- `BARE_OS_SEED_STORE` — Corestore directory (default `~/.bare-os/corestore/seeder`).
2. **Booter** (another terminal): 2. **Booter** (another terminal):
@@ -64,11 +65,15 @@ Runs workspace tests (`brittle-node` / `brittle-bare` where configured).
``` ```
Optional: Optional:
- `BARE_OS_HOST_DATA` — base directory for host state (default `~/.bare-os`).
- `BARE_OS_BOOT_STORE` — Corestore directory (default `~/.bare-os/corestore/booter`).
- `BARE_OS_BOOT_TIMEOUT_MS` — max ms for peer discovery + loading the kernel from the network (default `60000`). The booter does not fall back to a local seed; run the seeder on the same topic. - `BARE_OS_BOOT_TIMEOUT_MS` — max ms for peer discovery + loading the kernel from the network (default `60000`). The booter does not fall back to a local seed; run the seeder on the same topic.
- `BARE_OS_NO_SPLASH=1` — disable the TTY boot splash. - `BARE_OS_NO_SPLASH=1` — disable the TTY boot splash.
- `BARE_OS_SKIP_REPL=1` — non-interactive kernel (CI / automation). - `BARE_OS_SKIP_REPL=1` — non-interactive kernel (CI / automation).
Corestore defaults (repo root, not temp dirs): `data/corestore-seeder` and `data/corestore-booter` (gitignored). Paths resolve from the monorepo root so you can run `node index.js` from either app package. Host Corestore defaults live under **`~/.bare-os/corestore/`** (`seeder` and `booter`), not in the repo. Override the base with `BARE_OS_HOST_DATA`, or set `BARE_OS_SEED_STORE` / `BARE_OS_BOOT_STORE` for individual stores.
If you previously used gitignored **`data/`** at the repo root, move or copy `data/corestore-seeder` → `~/.bare-os/corestore/seeder` and `data/corestore-booter` → `~/.bare-os/corestore/booter`, or point the env vars at the old paths.
## Guest session and identity ## Guest session and identity
+4 -2
View File
@@ -15,7 +15,7 @@ This chapter is the **operators desk**: how to install, test, run Pear apps,
| `packages/bare-os-seeder` | Publish OS drive | | `packages/bare-os-seeder` | Publish OS drive |
| `packages/bare-os-booter` | Network boot + runtime | | `packages/bare-os-booter` | Network boot + runtime |
| `scripts/` | Pear `node_modules` fixer, etc. | | `scripts/` | Pear `node_modules` fixer, etc. |
| `data/` | Gitignored Corestore dirs (default for dev) | | `data/` | Gitignored legacy Corestore dir (optional; defaults use `~/.bare-os`) |
Each workspace has its own **`README.md`** with package-specific commands. Each workspace has its own **`README.md`** with package-specific commands.
@@ -49,7 +49,7 @@ From **`packages/bare-os-booter`**:
node index.js node index.js
``` ```
Corestore paths resolve under repo **`data/`** when using `file:` URLs (see each packages `lib/paths.js`). Default Corestore paths are **`~/.bare-os/corestore/seeder`** and **`~/.bare-os/corestore/booter`** (override base with **`BARE_OS_HOST_DATA`**, or set **`BARE_OS_SEED_STORE`** / **`BARE_OS_BOOT_STORE`**). See each packages `lib/paths.js`.
--- ---
@@ -71,7 +71,9 @@ These run **`scripts/ensure-pear-node-modules.mjs`** first. **Do not** run `pear
| Variable | Component | Meaning | | Variable | Component | Meaning |
| ------------------------- | ------------- | -------------------------------- | | ------------------------- | ------------- | -------------------------------- |
| `BARE_OS_KERNEL_ROOT` | Seeder | Override kernel tree path | | `BARE_OS_KERNEL_ROOT` | Seeder | Override kernel tree path |
| `BARE_OS_HOST_DATA` | paths | Host state base (default `~/.bare-os`) |
| `BARE_OS_SEED_STORE` | Seeder | Corestore directory | | `BARE_OS_SEED_STORE` | Seeder | Corestore directory |
| `BARE_OS_BOOT_STORE` | Booter | Booter Corestore directory |
| `BARE_OS_BOOT_TIMEOUT_MS` | Booter | Boot deadline (default 60000) | | `BARE_OS_BOOT_TIMEOUT_MS` | Booter | Boot deadline (default 60000) |
| `BARE_OS_NO_SPLASH` | Booter | Disable TTY splash | | `BARE_OS_NO_SPLASH` | Booter | Disable TTY splash |
| `BARE_OS_SKIP_REPL` | Booter | Non-interactive kernel | | `BARE_OS_SKIP_REPL` | Booter | Non-interactive kernel |
File diff suppressed because one or more lines are too long
+3
View File
@@ -41,6 +41,9 @@ node index.js
| Variable | Meaning | | Variable | Meaning |
| ------------------------- | ------------------------------------------------------------- | | ------------------------- | ------------------------------------------------------------- |
| `BARE_OS_HOST_DATA` | Base directory for host state (default `~/.bare-os`) |
| `BARE_OS_BOOT_STORE` | Corestore directory (default `~/.bare-os/corestore/booter`) |
| `BARE_OS_LOCAL_SEED` | Overrides `defaultLocalSeedCorestorePath` (booter does not local-boot) |
| `BARE_OS_BOOT_TIMEOUT_MS` | Total time to find peers + load kernel (default `60000`) | | `BARE_OS_BOOT_TIMEOUT_MS` | Total time to find peers + load kernel (default `60000`) |
| `BARE_OS_NO_SPLASH` | Disable TTY splash | | `BARE_OS_NO_SPLASH` | Disable TTY splash |
| `BARE_OS_SKIP_REPL` | Non-interactive kernel (`readLine` → null) | | `BARE_OS_SKIP_REPL` | Non-interactive kernel (`readLine` → null) |
+134 -25
View File
@@ -3,6 +3,13 @@
* @see https://curl.se/docs/manpage.html for flag inspiration; behavior is a subset. * @see https://curl.se/docs/manpage.html for flag inspiration; behavior is a subset.
*/ */
import b4a from 'b4a'
import {
DEFAULT_CURL_USER_AGENT,
isSupportedFetchUrl,
normalizeFetchUrl
} from './http-fetch-url.js'
/** @type {boolean} */ /** @type {boolean} */
let bareFetchTried = false let bareFetchTried = false
@@ -37,14 +44,6 @@ function basicAuthHeader(user, pass) {
throw new Error('curl: cannot encode Basic auth (no btoa/Buffer)') throw new Error('curl: cannot encode Basic auth (no btoa/Buffer)')
} }
function looksLikeUrl(s) {
return (
/^https?:\/\//i.test(s) ||
s.startsWith('data:') ||
s.startsWith('file://')
)
}
function usage() { function usage() {
return ( return (
'usage: curl [options] URL...\n' + 'usage: curl [options] URL...\n' +
@@ -52,8 +51,26 @@ function usage() {
) )
} }
/**
* Split -H line into headers[] or record User-Agent for argv-order precedence with -A.
* @param {string} line
* @param {string[]} headers
* @param {string[]} userAgentSequence
* @returns {boolean} false if malformed
*/
function pushCurlHeaderLine(line, headers, userAgentSequence) {
const colon = line.indexOf(':')
if (colon < 1) return false
const name = line.slice(0, colon).trim()
const value = line.slice(colon + 1).trim()
if (name.toLowerCase() === 'user-agent') userAgentSequence.push(value)
else headers.push(line)
return true
}
function utf8Encode(str) { function utf8Encode(str) {
return new TextEncoder().encode(str) // Bare may not define TextEncoder; b4a works on Node and Bare.
return b4a.from(String(str), 'utf8')
} }
/** @param {(string | Uint8Array)[]} parts */ /** @param {(string | Uint8Array)[]} parts */
@@ -80,6 +97,47 @@ function expandWriteOut(fmt, vars) {
}) })
} }
const CURL_MAX_REDIRECTS = 50
/**
* curl -I -L: send HEAD first; after any 3xx, follow with GET (curl man: "GET is used after redirect").
* Using fetch(HEAD, redirect:'follow') keeps HEAD on each hop and breaks many CDNs; manual hops match curl.
*/
async function fetchHeadWithLocationFollow(fetchFn, startUrl, hdr, signal) {
let url = startUrl
let method = 'HEAD'
for (let hop = 0; hop < CURL_MAX_REDIRECTS; hop++) {
const res = await fetchFn(url, {
method,
headers: hdr,
redirect: 'manual',
signal,
body: undefined
})
if (res.status >= 300 && res.status < 400) {
const loc = res.headers.get('Location')
if (!loc) return res
let nextUrl
try {
nextUrl = new URL(loc, url).href
} catch {
return res
}
method = 'GET'
try {
if (res.body && typeof res.body.cancel === 'function')
await res.body.cancel()
} catch {
/* ignore */
}
url = nextUrl
continue
}
return res
}
return new Response('', { status: 310, statusText: 'Too many redirects' })
}
/** /**
* @param {Record<string, unknown>} ctx * @param {Record<string, unknown>} ctx
* @param {string[]} argv * @param {string[]} argv
@@ -90,6 +148,8 @@ export async function runCurlCli(ctx, argv) {
let method = '' let method = ''
const headers = [] const headers = []
/** @type {string[]} -H User-Agent and -A/--user-agent in argv order; last wins */
const userAgentSequence = []
/** @type {string[]} */ /** @type {string[]} */
const dataChunks = [] const dataChunks = []
let dataBinary = false let dataBinary = false
@@ -112,15 +172,23 @@ export async function runCurlCli(ctx, argv) {
/** @type {string | null} */ /** @type {string | null} */
let uploadPath = null let uploadPath = null
/** @type {string[]} */
const urls = []
let i = 0 let i = 0
while (i < args.length) { while (i < args.length) {
const a = args[i] const a = args[i]
if (a === '--') { if (a === '--') {
i++ i++
while (i < args.length) {
urls.push(normalizeFetchUrl(String(args[i++])))
}
break break
} }
if (!a.startsWith('-') || a === '-') { if (!a.startsWith('-') || a === '-') {
break urls.push(normalizeFetchUrl(String(a)))
i++
continue
} }
if (a === '-h' || a === '--help') { if (a === '-h' || a === '--help') {
@@ -154,12 +222,43 @@ export async function runCurlCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
headers.push(String(args[++i])) const line = String(args[++i])
if (!pushCurlHeaderLine(line, headers, userAgentSequence)) {
ctx.console.error('curl: malformed header: ' + line)
ctx.exitCode = 2
return
}
i++ i++
continue continue
} }
if (a.startsWith('-H') && a.length > 2) { if (a.startsWith('-H') && a.length > 2) {
headers.push(a.slice(2)) const line = a.slice(2)
if (!pushCurlHeaderLine(line, headers, userAgentSequence)) {
ctx.console.error('curl: malformed header: ' + line)
ctx.exitCode = 2
return
}
i++
continue
}
if (a === '-A' || a === '--user-agent') {
if (i + 1 >= args.length) {
ctx.console.error('curl: option requires an argument: ' + a)
ctx.exitCode = 2
return
}
userAgentSequence.push(String(args[++i]))
i++
continue
}
if (a.startsWith('--user-agent=')) {
userAgentSequence.push(a.slice('--user-agent='.length))
i++
continue
}
if (a.startsWith('-A') && a.length > 2) {
userAgentSequence.push(a.slice(2))
i++ i++
continue continue
} }
@@ -340,11 +439,6 @@ export async function runCurlCli(ctx, argv) {
i++ i++
} }
const urls = []
while (i < args.length) {
urls.push(String(args[i++]))
}
if (urls.length === 0) { if (urls.length === 0) {
ctx.console.error(usage()) ctx.console.error(usage())
ctx.exitCode = 2 ctx.exitCode = 2
@@ -352,7 +446,7 @@ export async function runCurlCli(ctx, argv) {
} }
for (const u of urls) { for (const u of urls) {
if (!looksLikeUrl(u)) { if (!isSupportedFetchUrl(u)) {
ctx.console.error( ctx.console.error(
'curl: URL rejected (need http(s)://, data:, or file://): ' + u 'curl: URL rejected (need http(s)://, data:, or file://): ' + u
) )
@@ -401,12 +495,17 @@ export async function runCurlCli(ctx, argv) {
const pass = colon >= 0 ? userColonPass.slice(colon + 1) : '' const pass = colon >= 0 ? userColonPass.slice(colon + 1) : ''
hdr.set('Authorization', basicAuthHeader(user, pass)) hdr.set('Authorization', basicAuthHeader(user, pass))
} }
{
const ua = userAgentSequence.length
? userAgentSequence[userAgentSequence.length - 1]
: DEFAULT_CURL_USER_AGENT
hdr.set('User-Agent', ua)
}
/** @type {string | Uint8Array | undefined} */ /** @type {string | Uint8Array | undefined} */
let body = undefined let body = undefined
if (jsonBody) { if (jsonBody) {
if (!hdr.has('Content-Type')) if (!hdr.has('Content-Type')) hdr.set('Content-Type', 'application/json')
hdr.set('Content-Type', 'application/json')
body = jsonBody body = jsonBody
} else if (dataChunks.length > 0) { } else if (dataChunks.length > 0) {
const joined = dataChunks.join('&') const joined = dataChunks.join('&')
@@ -422,7 +521,8 @@ export async function runCurlCli(ctx, argv) {
return return
} }
body = new Uint8Array(buf) body = new Uint8Array(buf)
if (!hdr.has('Content-Type')) hdr.set('Content-Type', 'application/octet-stream') if (!hdr.has('Content-Type'))
hdr.set('Content-Type', 'application/octet-stream')
} }
let lastSize = 0 let lastSize = 0
@@ -430,7 +530,8 @@ export async function runCurlCli(ctx, argv) {
for (let ui = 0; ui < urls.length; ui++) { for (let ui = 0; ui < urls.length; ui++) {
const url = urls[ui] const url = urls[ui]
const outForUrl = urls.length > 1 && outputPath ? `${outputPath}.${ui}` : outputPath const outForUrl =
urls.length > 1 && outputPath ? `${outputPath}.${ui}` : outputPath
const ac = maxTimeMs > 0 ? new AbortController() : null const ac = maxTimeMs > 0 ? new AbortController() : null
const t = const t =
@@ -446,6 +547,14 @@ export async function runCurlCli(ctx, argv) {
let res let res
try { try {
if (headOnly && location) {
res = await fetchHeadWithLocationFollow(
fetchFn,
url,
hdr,
ac ? ac.signal : undefined
)
} else {
res = await fetchFn(url, { res = await fetchFn(url, {
method: m, method: m,
headers: hdr, headers: hdr,
@@ -453,6 +562,7 @@ export async function runCurlCli(ctx, argv) {
redirect: location ? 'follow' : 'manual', redirect: location ? 'follow' : 'manual',
signal: ac ? ac.signal : undefined signal: ac ? ac.signal : undefined
}) })
}
} catch (e) { } catch (e) {
if (t) clearTimeout(t) if (t) clearTimeout(t)
const msg = e && e.message ? e.message : String(e) const msg = e && e.message ? e.message : String(e)
@@ -466,7 +576,7 @@ export async function runCurlCli(ctx, argv) {
if (!location && res.status >= 300 && res.status < 400) { if (!location && res.status >= 300 && res.status < 400) {
const loc = res.headers.get('Location') const loc = res.headers.get('Location')
if (loc && m === 'GET') { if (loc && (m === 'GET' || m === 'HEAD')) {
if (!silent || showError) if (!silent || showError)
ctx.console.error( ctx.console.error(
'curl: redirect not followed (use -L): ' + res.status + ' -> ' + loc 'curl: redirect not followed (use -L): ' + res.status + ' -> ' + loc
@@ -494,8 +604,7 @@ export async function runCurlCli(ctx, argv) {
const chunks = [] const chunks = []
if (m === 'HEAD' || (includeHeaders && m !== 'HEAD')) { if (m === 'HEAD' || (includeHeaders && m !== 'HEAD')) {
const statusLine = const statusLine = 'HTTP/1.1 ' + res.status + ' ' + (res.statusText || '')
'HTTP/1.1 ' + res.status + ' ' + (res.statusText || '')
chunks.push(statusLine + '\r\n') chunks.push(statusLine + '\r\n')
res.headers.forEach((v, k) => { res.headers.forEach((v, k) => {
chunks.push(k + ': ' + v + '\r\n') chunks.push(k + ': ' + v + '\r\n')
@@ -0,0 +1,31 @@
/**
* URL normalization for fetch-based curl/wget subsets (scheme guessing like CLI tools).
*/
/** curl-style default (see https://everything.curl.dev/http/modify/user-agent.html); not a real libcurl build version. */
export const DEFAULT_CURL_USER_AGENT = 'curl/8.14.1'
/** GNU wget-style default (many builds use Wget/VERSION (linux-gnu)). */
export const DEFAULT_WGET_USER_AGENT = 'Wget/1.25.0 (linux-gnu)'
/** True if the string is usable by our fetch stack without guessing a scheme. */
export function isSupportedFetchUrl(s) {
return (
/^https?:\/\//i.test(s) || s.startsWith('data:') || s.startsWith('file://')
)
}
/**
* If there is no http(s)/data/file scheme, prepend http:// for host-like URLs (wget default).
* Protocol-relative //host → https://host. Leaves other schemes (ftp:, mailto:, …) unchanged.
* Does not rewrite `-`, ./ ../, or absolute /paths (not treated as remote URLs).
*/
export function normalizeFetchUrl(s) {
const t = String(s)
if (isSupportedFetchUrl(t)) return t
if (t.startsWith('//')) return 'https:' + t
if (/^[a-zA-Z][a-zA-Z0-9+.-]*:/.test(t)) return t
if (t === '-') return t
if (t.startsWith('/') || t.startsWith('./') || t.startsWith('../')) return t
return 'http://' + t
}
+9 -10
View File
@@ -13,6 +13,13 @@ function env(name) {
return globalThis.process?.env?.[name] return globalThis.process?.env?.[name]
} }
/** Host-side state root (Corestore, etc.): `BARE_OS_HOST_DATA` or `~/.bare-os`. */
function hostDataRoot() {
const override = env('BARE_OS_HOST_DATA')
if (override) return path.resolve(override)
return path.join(os.homedir(), '.bare-os')
}
/** Booter has no `kernel/`; use `package.json` to detect the staged app root. */ /** Booter has no `kernel/`; use `package.json` to detect the staged app root. */
function stagedAppRootHeuristic(root) { function stagedAppRootHeuristic(root) {
try { try {
@@ -47,19 +54,11 @@ export function packageRootDir(metaUrl) {
export function defaultBootCorestorePath(pkgRoot, metaUrl) { export function defaultBootCorestorePath(pkgRoot, metaUrl) {
const override = env('BARE_OS_BOOT_STORE') const override = env('BARE_OS_BOOT_STORE')
if (override) return override if (override) return override
const href = String(metaUrl) return path.join(hostDataRoot(), 'corestore', 'booter')
const dataRoot = href.startsWith('file:')
? path.join(pkgRoot, '..', '..', 'data')
: path.join(pkgRoot, 'data')
return path.join(dataRoot, 'corestore-booter')
} }
export function defaultLocalSeedCorestorePath(pkgRoot, metaUrl) { export function defaultLocalSeedCorestorePath(pkgRoot, metaUrl) {
const override = env('BARE_OS_LOCAL_SEED') const override = env('BARE_OS_LOCAL_SEED')
if (override) return override if (override) return override
const href = String(metaUrl) return path.join(hostDataRoot(), 'corestore', 'seeder')
const dataRoot = href.startsWith('file:')
? path.join(pkgRoot, '..', '..', 'data')
: path.join(pkgRoot, 'data')
return path.join(dataRoot, 'corestore-seeder')
} }
+157 -45
View File
@@ -4,6 +4,11 @@
*/ */
import path from 'path' import path from 'path'
import {
DEFAULT_WGET_USER_AGENT,
isSupportedFetchUrl,
normalizeFetchUrl
} from './http-fetch-url.js'
/** @type {boolean} */ /** @type {boolean} */
let bareFetchTried = false let bareFetchTried = false
@@ -31,14 +36,6 @@ async function ensureBareFetchForWget() {
} }
} }
function looksLikeUrl(s) {
return (
/^https?:\/\//i.test(s) ||
s.startsWith('data:') ||
s.startsWith('file://')
)
}
function defaultLocalName(urlString) { function defaultLocalName(urlString) {
try { try {
const u = new URL(urlString) const u = new URL(urlString)
@@ -59,6 +56,104 @@ function usage() {
) )
} }
/**
* One argv token of short options: -qO-, -T30, -O file (via -O then next arg).
* @returns {number} next index into args, or -1 on error, -2 on help/version done
*/
function wgetConsumeShortCluster(a, args, i, st, ctx) {
let j = 1
while (j < a.length) {
const c = a[j]
if (c === 'q') {
st.quiet = true
j++
continue
}
if (c === 'h') {
ctx.console.log(usage())
ctx.exitCode = 0
return -2
}
if (c === 'V') {
ctx.console.log('wget (Bare OS fetch subset) 0.1 — not GNU wget2')
ctx.exitCode = 0
return -2
}
if (c === 'O') {
const tail = a.slice(j + 1)
if (tail !== '') {
st.outputDocument = tail
return i + 1
}
if (i + 1 >= args.length) {
ctx.console.error('wget: option requires an argument: -O')
ctx.exitCode = 2
return -1
}
st.outputDocument = String(args[i + 1])
return i + 2
}
if (c === 'P') {
const tail = a.slice(j + 1)
if (tail !== '') {
st.directoryPrefix = tail
return i + 1
}
if (i + 1 >= args.length) {
ctx.console.error('wget: option requires an argument: -P')
ctx.exitCode = 2
return -1
}
st.directoryPrefix = String(args[i + 1])
return i + 2
}
if (c === 'U') {
const tail = a.slice(j + 1)
if (tail !== '') {
st.userAgent = tail
return i + 1
}
if (i + 1 >= args.length) {
ctx.console.error('wget: option requires an argument: -U')
ctx.exitCode = 2
return -1
}
st.userAgent = String(args[i + 1])
return i + 2
}
if (c === 'T') {
const tail = a.slice(j + 1)
if (tail !== '') {
const sec = Number(tail)
if (!Number.isFinite(sec) || sec < 0) {
ctx.console.error('wget: invalid timeout')
ctx.exitCode = 2
return -1
}
st.timeoutMs = Math.round(sec * 1000)
return i + 1
}
if (i + 1 >= args.length) {
ctx.console.error('wget: option requires an argument: -T')
ctx.exitCode = 2
return -1
}
const sec = Number(args[i + 1])
if (!Number.isFinite(sec) || sec < 0) {
ctx.console.error('wget: invalid timeout')
ctx.exitCode = 2
return -1
}
st.timeoutMs = Math.round(sec * 1000)
return i + 2
}
ctx.console.error('wget: invalid option: -' + c)
ctx.exitCode = 2
return -1
}
return i + 1
}
/** /**
* @param {Record<string, unknown>} ctx * @param {Record<string, unknown>} ctx
* @param {string[]} argv * @param {string[]} argv
@@ -67,28 +162,35 @@ export async function runWgetCli(ctx, argv) {
const vfs = ctx.vfs const vfs = ctx.vfs
const args = argv.slice(1) const args = argv.slice(1)
let quiet = false const st = {
/** @type {string | null} */ quiet: false,
let outputDocument = null outputDocument: /** @type {string | null} */ (null),
/** @type {string | null} */ directoryPrefix: /** @type {string | null} */ (null),
let directoryPrefix = null userAgent: /** @type {string | null} */ (null),
/** @type {string | null} */ timeoutMs: 0,
let userAgent = null extraHeaders: /** @type {string[]} */ ([]),
let timeoutMs = 0 postData: /** @type {string | null} */ (null),
const extraHeaders = [] postFile: /** @type {string | null} */ (null)
/** @type {string | null} */ }
let postData = null
/** @type {string | null} */ /** @type {string[]} */
let postFile = null const urls = []
let i = 0 let i = 0
while (i < args.length) { while (i < args.length) {
const a = args[i] const a = args[i]
if (a === '--') { if (a === '--') {
i++ i++
while (i < args.length) {
urls.push(normalizeFetchUrl(String(args[i++])))
}
break break
} }
if (!a.startsWith('-') || a === '-') break if (!a.startsWith('-') || a === '-') {
urls.push(normalizeFetchUrl(String(a)))
i++
continue
}
if (a === '-h' || a === '--help') { if (a === '-h' || a === '--help') {
ctx.console.log(usage()) ctx.console.log(usage())
@@ -100,7 +202,7 @@ export async function runWgetCli(ctx, argv) {
} }
if (a === '-q' || a === '--quiet') { if (a === '-q' || a === '--quiet') {
quiet = true st.quiet = true
i++ i++
continue continue
} }
@@ -111,12 +213,12 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
outputDocument = String(args[++i]) st.outputDocument = String(args[++i])
i++ i++
continue continue
} }
if (a.startsWith('--output-document=')) { if (a.startsWith('--output-document=')) {
outputDocument = a.slice('--output-document='.length) st.outputDocument = a.slice('--output-document='.length)
i++ i++
continue continue
} }
@@ -127,12 +229,12 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
directoryPrefix = String(args[++i]) st.directoryPrefix = String(args[++i])
i++ i++
continue continue
} }
if (a.startsWith('--directory-prefix=')) { if (a.startsWith('--directory-prefix=')) {
directoryPrefix = a.slice('--directory-prefix='.length) st.directoryPrefix = a.slice('--directory-prefix='.length)
i++ i++
continue continue
} }
@@ -143,12 +245,12 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
userAgent = String(args[++i]) st.userAgent = String(args[++i])
i++ i++
continue continue
} }
if (a.startsWith('--user-agent=')) { if (a.startsWith('--user-agent=')) {
userAgent = a.slice('--user-agent='.length) st.userAgent = a.slice('--user-agent='.length)
i++ i++
continue continue
} }
@@ -165,7 +267,7 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
timeoutMs = Math.round(sec * 1000) st.timeoutMs = Math.round(sec * 1000)
i++ i++
continue continue
} }
@@ -176,7 +278,7 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
timeoutMs = Math.round(sec * 1000) st.timeoutMs = Math.round(sec * 1000)
i++ i++
continue continue
} }
@@ -187,12 +289,12 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
extraHeaders.push(String(args[++i])) st.extraHeaders.push(String(args[++i]))
i++ i++
continue continue
} }
if (a.startsWith('--header=')) { if (a.startsWith('--header=')) {
extraHeaders.push(a.slice('--header='.length)) st.extraHeaders.push(a.slice('--header='.length))
i++ i++
continue continue
} }
@@ -203,12 +305,12 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
postData = String(args[++i]) st.postData = String(args[++i])
i++ i++
continue continue
} }
if (a.startsWith('--post-data=')) { if (a.startsWith('--post-data=')) {
postData = a.slice('--post-data='.length) st.postData = a.slice('--post-data='.length)
i++ i++
continue continue
} }
@@ -219,7 +321,7 @@ export async function runWgetCli(ctx, argv) {
ctx.exitCode = 2 ctx.exitCode = 2
return return
} }
postFile = String(args[++i]) st.postFile = String(args[++i])
i++ i++
continue continue
} }
@@ -230,15 +332,22 @@ export async function runWgetCli(ctx, argv) {
return return
} }
ctx.console.error('wget: invalid option: ' + a) const ni = wgetConsumeShortCluster(a, args, i, st, ctx)
ctx.exitCode = 2 if (ni === -1) return
return if (ni === -2) return
i = ni
} }
const urls = [] const {
while (i < args.length) { quiet,
urls.push(String(args[i++])) outputDocument,
} directoryPrefix,
userAgent,
timeoutMs,
extraHeaders,
postData,
postFile
} = st
if (urls.length === 0) { if (urls.length === 0) {
ctx.console.error(usage()) ctx.console.error(usage())
@@ -247,7 +356,7 @@ export async function runWgetCli(ctx, argv) {
} }
for (const u of urls) { for (const u of urls) {
if (!looksLikeUrl(u)) { if (!isSupportedFetchUrl(u)) {
ctx.console.error( ctx.console.error(
'wget: URL rejected (need http(s)://, data:, or file://): ' + u 'wget: URL rejected (need http(s)://, data:, or file://): ' + u
) )
@@ -290,7 +399,8 @@ export async function runWgetCli(ctx, argv) {
if (postFile) { if (postFile) {
const buf = await vfs.readFile(postFile) const buf = await vfs.readFile(postFile)
if (!buf) { if (!buf) {
if (!quiet) ctx.console.error('wget: cannot read --post-file: ' + postFile) if (!quiet)
ctx.console.error('wget: cannot read --post-file: ' + postFile)
ctx.exitCode = 3 ctx.exitCode = 3
return return
} }
@@ -312,6 +422,8 @@ export async function runWgetCli(ctx, argv) {
hdr.set(line.slice(0, colon).trim(), line.slice(colon + 1).trim()) hdr.set(line.slice(0, colon).trim(), line.slice(colon + 1).trim())
} }
if (userAgent) hdr.set('User-Agent', userAgent) if (userAgent) hdr.set('User-Agent', userAgent)
else if (!hdr.has('User-Agent'))
hdr.set('User-Agent', DEFAULT_WGET_USER_AGENT)
if (body !== undefined && !hdr.has('Content-Type')) if (body !== undefined && !hdr.has('Content-Type'))
hdr.set('Content-Type', 'application/x-www-form-urlencoded') hdr.set('Content-Type', 'application/x-www-form-urlencoded')
+172 -9
View File
@@ -36,8 +36,19 @@ import {
jobMatchesDate jobMatchesDate
} from './lib/bare-cron.js' } from './lib/bare-cron.js'
import { registerBareInitdDisposer, stopBareInitd } from './lib/bare-initd.js' import { registerBareInitdDisposer, stopBareInitd } from './lib/bare-initd.js'
import {
DEFAULT_CURL_USER_AGENT,
DEFAULT_WGET_USER_AGENT
} from './lib/http-fetch-url.js'
const __dirname = path.dirname(fileURLToPath(import.meta.url)) const __dirname = path.dirname(fileURLToPath(import.meta.url))
function headerFromInit(init, name) {
if (!init?.headers) return undefined
const h = init.headers
if (typeof h.get === 'function') return h.get(name)
return undefined
}
function testCorestoreDir(name) { function testCorestoreDir(name) {
const dir = path.join( const dir = path.join(
__dirname, __dirname,
@@ -446,11 +457,10 @@ test('expandArgvAliases expands first word and keeps trailing argv', async (t) =
}) })
test('expandArgvAliases leaves sed unchanged', async (t) => { test('expandArgvAliases leaves sed unchanged', async (t) => {
t.alike(expandArgvAliases(['sed', 's/a/b/', 'x.txt'], defaultShellAliases()), [ t.alike(
'sed', expandArgvAliases(['sed', 's/a/b/', 'x.txt'], defaultShellAliases()),
's/a/b/', ['sed', 's/a/b/', 'x.txt']
'x.txt' )
])
}) })
test('defaultShellAliases does not remap sed', async (t) => { test('defaultShellAliases does not remap sed', async (t) => {
@@ -758,14 +768,19 @@ test('curl delegated from booter with stub fetch', async (t) => {
lines.push('e:' + String(s)) lines.push('e:' + String(s))
} }
} }
ctx.httpFetch = async () => /** @type {RequestInit | undefined} */
new Response('hello', { let curlInitHello
ctx.httpFetch = async (_url, init) => {
curlInitHello = init
return new Response('hello', {
status: 200, status: 200,
headers: { 'Content-Type': 'text/plain' } headers: { 'Content-Type': 'text/plain' }
}) })
}
await runBinCommand(ctx, ['curl', 'https://stub.example/x']) await runBinCommand(ctx, ['curl', 'https://stub.example/x'])
t.is(ctx.exitCode, 0) t.is(ctx.exitCode, 0)
t.is(lines[0], 'hello') t.is(lines[0], 'hello')
t.is(headerFromInit(curlInitHello, 'User-Agent'), DEFAULT_CURL_USER_AGENT)
lines.length = 0 lines.length = 0
ctx.exitCode = 0 ctx.exitCode = 0
@@ -773,6 +788,112 @@ test('curl delegated from booter with stub fetch', async (t) => {
await runBinCommand(ctx, ['curl', '-f', '-s', 'https://stub.example/missing']) await runBinCommand(ctx, ['curl', '-f', '-s', 'https://stub.example/missing'])
t.is(ctx.exitCode, 22) t.is(ctx.exitCode, 22)
lines.length = 0
ctx.exitCode = 0
let curlSeenUrl = ''
/** @type {RequestInit | undefined} */
let curlSeenInit
ctx.httpFetch = async (url, init) => {
curlSeenUrl = String(url)
curlSeenInit = init
return new Response('', { status: 200 })
}
await runBinCommand(ctx, ['curl', 'example.com', '-IL'])
t.is(ctx.exitCode, 0)
t.is(curlSeenUrl, 'http://example.com')
t.is(curlSeenInit && curlSeenInit.method, 'HEAD')
lines.length = 0
ctx.exitCode = 0
/** @type {{ url: string, method: string }[]} */
const curlHopCalls = []
ctx.httpFetch = async (url, init) => {
const u = String(url)
const method = String(init?.method || 'GET')
curlHopCalls.push({ url: u, method })
if (method === 'HEAD' && u === 'http://hop.example/start') {
return new Response('', {
status: 302,
headers: { Location: 'https://hop.example/done' }
})
}
if (method === 'GET' && u === 'https://hop.example/done') {
return new Response('secret-body', {
status: 200,
headers: { 'Content-Type': 'text/plain' }
})
}
return new Response('unexpected', { status: 500 })
}
await runBinCommand(ctx, ['curl', '-IL', 'http://hop.example/start'])
t.is(ctx.exitCode, 0)
t.is(curlHopCalls.length, 2)
t.is(curlHopCalls[0].method, 'HEAD')
t.is(curlHopCalls[1].method, 'GET')
t.ok(lines[0].includes('200'))
t.ok(!lines[0].includes('secret-body'))
lines.length = 0
ctx.exitCode = 0
ctx.httpFetch = async (url) => {
curlSeenUrl = String(url)
return new Response('', { status: 200 })
}
await runBinCommand(ctx, ['curl', '-I', '//stub.example/p'])
t.is(ctx.exitCode, 0)
t.is(curlSeenUrl, 'https://stub.example/p')
lines.length = 0
ctx.exitCode = 0
/** @type {RequestInit | undefined} */
let curlInitUa
ctx.httpFetch = async (_url, init) => {
curlInitUa = init
return new Response('x', { status: 200 })
}
await runBinCommand(ctx, [
'curl',
'-A',
'CustomCurlUA/9',
'https://stub.example/ua'
])
t.is(ctx.exitCode, 0)
t.is(headerFromInit(curlInitUa, 'User-Agent'), 'CustomCurlUA/9')
lines.length = 0
ctx.exitCode = 0
ctx.httpFetch = async (_url, init) => {
curlInitUa = init
return new Response('x', { status: 200 })
}
await runBinCommand(ctx, [
'curl',
'-H',
'User-Agent: from-header',
'-A',
'from-flag',
'https://stub.example/ua2'
])
t.is(ctx.exitCode, 0)
t.is(headerFromInit(curlInitUa, 'User-Agent'), 'from-flag')
lines.length = 0
ctx.exitCode = 0
ctx.httpFetch = async (_url, init) => {
curlInitUa = init
return new Response('x', { status: 200 })
}
await runBinCommand(ctx, [
'curl',
'-A',
'from-flag',
'-H',
'User-Agent: from-header',
'https://stub.example/ua3'
])
t.is(ctx.exitCode, 0)
t.is(headerFromInit(curlInitUa, 'User-Agent'), 'from-header')
await store.close() await store.close()
rmSync(dir, { recursive: true, force: true }) rmSync(dir, { recursive: true, force: true })
}) })
@@ -795,11 +916,15 @@ test('wget delegated from booter with stub fetch', async (t) => {
lines.push('e:' + String(s)) lines.push('e:' + String(s))
} }
} }
ctx.httpFetch = async () => /** @type {RequestInit | undefined} */
new Response('payload', { let wgetInitPayload
ctx.httpFetch = async (_url, init) => {
wgetInitPayload = init
return new Response('payload', {
status: 200, status: 200,
headers: { 'Content-Type': 'text/plain' } headers: { 'Content-Type': 'text/plain' }
}) })
}
await runBinCommand(ctx, [ await runBinCommand(ctx, [
'wget', 'wget',
'-O', '-O',
@@ -808,6 +933,7 @@ test('wget delegated from booter with stub fetch', async (t) => {
'https://stub.example/a' 'https://stub.example/a'
]) ])
t.is(ctx.exitCode, 0) t.is(ctx.exitCode, 0)
t.is(headerFromInit(wgetInitPayload, 'User-Agent'), DEFAULT_WGET_USER_AGENT)
const out = await ctx.vfs.readFile('saved.txt') const out = await ctx.vfs.readFile('saved.txt')
t.ok(out) t.ok(out)
t.is(ctx.b4a.toString(out), 'payload') t.is(ctx.b4a.toString(out), 'payload')
@@ -825,6 +951,43 @@ test('wget delegated from booter with stub fetch', async (t) => {
t.is(ctx.exitCode, 0) t.is(ctx.exitCode, 0)
t.is(lines[0], 'stdout-body') t.is(lines[0], 'stdout-body')
lines.length = 0
ctx.exitCode = 0
let wgetSeenUrl = ''
ctx.httpFetch = async (url) => {
wgetSeenUrl = String(url)
return new Response('p2', {
status: 200,
headers: { 'Content-Type': 'text/plain' }
})
}
await runBinCommand(ctx, ['wget', 'stub.example/z', '-qO', 'w2.txt'])
t.is(ctx.exitCode, 0)
t.is(wgetSeenUrl, 'http://stub.example/z')
const w2 = await ctx.vfs.readFile('w2.txt')
t.ok(w2)
t.is(ctx.b4a.toString(w2), 'p2')
lines.length = 0
ctx.exitCode = 0
/** @type {RequestInit | undefined} */
let wgetInitU
ctx.httpFetch = async (_url, init) => {
wgetInitU = init
return new Response('u', { status: 200 })
}
await runBinCommand(ctx, [
'wget',
'-q',
'-U',
'WgetCustom/1',
'-O',
'u.txt',
'https://stub.example/u'
])
t.is(ctx.exitCode, 0)
t.is(headerFromInit(wgetInitU, 'User-Agent'), 'WgetCustom/1')
await store.close() await store.close()
rmSync(dir, { recursive: true, force: true }) rmSync(dir, { recursive: true, force: true })
}) })
@@ -16,6 +16,10 @@
"flag": "-H, --header LINE", "flag": "-H, --header LINE",
"meaning": "Request header (repeatable)" "meaning": "Request header (repeatable)"
}, },
{
"flag": "-A, --user-agent STRING",
"meaning": "Set User-Agent (default curl/VERSION-style string; last -A or -H User-Agent on the command line wins)"
},
{ {
"flag": "-d, --data / --json", "flag": "-d, --data / --json",
"meaning": "Request body" "meaning": "Request body"
@@ -33,14 +37,8 @@
"meaning": "See full man curl.json" "meaning": "See full man curl.json"
} }
], ],
"keywords": [ "keywords": ["curl", "http", "https", "fetch", "download"],
"curl", "bareOsNotes": "Not https://curl.se libcurl; booter curl-cli.js. Default User-Agent matches curl/VERSION form (see everything.curl.dev user-agent). Options and URLs may be interleaved (POSIX-style). Host-style URLs without a scheme get http://; protocol-relative //host gets https://. Fetch uses http(s), data:, or file:// after normalization.",
"http",
"https",
"fetch",
"download"
],
"bareOsNotes": "Not https://curl.se libcurl; booter curl-cli.js. URLs: http(s), data:, file://.",
"seeAlso": [ "seeAlso": [
{ {
"name": "git", "name": "git",
+9 -16
View File
@@ -22,7 +22,7 @@
}, },
{ {
"flag": "-U, --user-agent STRING", "flag": "-U, --user-agent STRING",
"meaning": "Set User-Agent request header" "meaning": "Set User-Agent (default Wget/VERSION (linux-gnu)-style when -U and --header User-Agent are absent)"
}, },
{ {
"flag": "-T, --timeout SECONDS", "flag": "-T, --timeout SECONDS",
@@ -50,15 +50,8 @@
} }
], ],
"environment": [], "environment": [],
"keywords": [ "keywords": ["wget", "download", "http", "https", "fetch", "mirror"],
"wget", "bareOsNotes": "Not GNU wget2 (https://gitlab.com/gnuwget/wget2); booter wget-cli.js. Default User-Agent matches GNU wget-style Wget/VERSION (linux-gnu). -U overrides; --header User-Agent is used if present unless -U is set. Options and URLs may be interleaved; short options can be clustered (e.g. -qO-, -T30). Host-style URLs without a scheme get http://; //host gets https://. No recursive retrieval, FTP, or WARC. Cannot combine -O and -P. Tests may set ctx.httpFetch.",
"download",
"http",
"https",
"fetch",
"mirror"
],
"bareOsNotes": "Not GNU wget2 (https://gitlab.com/gnuwget/wget2). URLs must start with http://, https://, data:, or file://. No recursive retrieval, FTP, or WARC. Cannot combine -O and -P. On Pear/Bare, use bare-fetch for global fetch. Tests may set ctx.httpFetch.",
"seeAlso": [ "seeAlso": [
{ {
"name": "curl", "name": "curl",
@@ -88,11 +81,11 @@
} }
], ],
"exitStatus": [ "exitStatus": [
"0 success", "0 \u2014 success",
"1 generic error (reserved)", "1 \u2014 generic error (reserved)",
"2 bad usage or options", "2 \u2014 bad usage or options",
"3 file I/O error (e.g. --post-file unreadable)", "3 \u2014 file I/O error (e.g. --post-file unreadable)",
"4 network failure or no fetch implementation", "4 \u2014 network failure or no fetch implementation",
"8 HTTP 4xx/5xx response" "8 \u2014 HTTP 4xx/5xx response"
] ]
} }
+2 -1
View File
@@ -27,7 +27,8 @@ npm run os:seeder
| Variable | Meaning | | Variable | Meaning |
| --------------------- | -------------------------------------------------------------------------------------------------------------------------- | | --------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `BARE_OS_KERNEL_ROOT` | Absolute path to kernel tree to stage (default: package `kernel/` vendored copy, or repo `kernel/` under Node `file:` URL) | | `BARE_OS_KERNEL_ROOT` | Absolute path to kernel tree to stage (default: package `kernel/` vendored copy, or repo `kernel/` under Node `file:` URL) |
| `BARE_OS_SEED_STORE` | Corestore directory (default under repo `data/` when resolving from `file:`) | | `BARE_OS_HOST_DATA` | Base directory for host state (default `~/.bare-os`) |
| `BARE_OS_SEED_STORE` | Corestore directory (default `~/.bare-os/corestore/seeder`) |
## Pear vs Node ## Pear vs Node
File diff suppressed because one or more lines are too long
+8 -5
View File
@@ -13,6 +13,13 @@ function env(name) {
return globalThis.process?.env?.[name] return globalThis.process?.env?.[name]
} }
/** Host-side state root (Corestore, etc.): `BARE_OS_HOST_DATA` or `~/.bare-os`. */
function hostDataRoot() {
const override = env('BARE_OS_HOST_DATA')
if (override) return path.resolve(override)
return path.join(os.homedir(), '.bare-os')
}
function kernelDirPresent(root) { function kernelDirPresent(root) {
try { try {
return statSync(path.join(root, 'kernel')).isDirectory() return statSync(path.join(root, 'kernel')).isDirectory()
@@ -59,9 +66,5 @@ export function defaultKernelRoot(pkgRoot, metaUrl) {
export function defaultSeedCorestorePath(pkgRoot, metaUrl) { export function defaultSeedCorestorePath(pkgRoot, metaUrl) {
const override = env('BARE_OS_SEED_STORE') const override = env('BARE_OS_SEED_STORE')
if (override) return override if (override) return override
const href = String(metaUrl) return path.join(hostDataRoot(), 'corestore', 'seeder')
const dataRoot = href.startsWith('file:')
? path.join(pkgRoot, '..', '..', 'data')
: path.join(pkgRoot, 'data')
return path.join(dataRoot, 'corestore-seeder')
} }