bare-ffmpeg encode support
CI / Build & Test (push) Successful in 8m23s

This commit is contained in:
Raven Scott
2026-07-26 23:34:02 -04:00
parent 906422d0a5
commit bb3b46a987
17 changed files with 1438 additions and 32 deletions
+5 -4
View File
@@ -28,7 +28,7 @@ These are the main APIs your page uses. For host request types and event payload
- **`BridgeSwarm.capabilities`** — Bare capability packs on the host (`media` is default). `list()`, `has(pack)`, `call(pack, cmd, payload)`, `on(event, fn)`. See [CAPABILITIES.md](CAPABILITIES.md) / [DEFAULT-MODULES.md](DEFAULT-MODULES.md).
- **`BridgeSwarm.media.*`** — Media pack wrappers (`info`, `imageTransform`, `extractFrame`, `transcode`, `cancel`, `writeInput`, `readOutput`).
- **`BridgeSwarm.media.*`** — Media pack: batch (`info`, `imageTransform`, `extractFrame`, `transcode`, …) and live encode (`encodeStart`, `encodePushFrame`, `encodeStop`, `liveSession`, `attachLiveReceiver`). See [CAPABILITIES.md](CAPABILITIES.md).
---
## Host request types and events
@@ -145,14 +145,15 @@ Full details and examples: [DATA-API.md](DATA-API.md).
| `capabilities.has` | `{ pack }` | `{ ok, pack, has }` |
| `capability` | `{ pack, cmd, payload }` | pack-specific |
| `media.info` / `media.imageTransform` / `media.extractFrame` / `media.transcode` / `media.cancel` / … | command payload | see [CAPABILITIES.md](CAPABILITIES.md) |
| `media.encodeStart` / `encodePushFrame` / `encodePush` / `encodeSubscribe` / `encodeStop` | live session payload | `{ ok, sessionId, … }` + `cap-chunk` segments |
### Capability events
| Event | Payload |
|-------|---------|
| `cap-chunk` | `{ pack, jobId, index?, data?, progress?, bytes? }` |
| `cap-end` | `{ pack, jobId, … }` |
| `cap-error` | `{ pack, jobId, message }` |
| `cap-chunk` | `{ pack, jobId/sessionId, kind?, seq?, index?, data?, progress?, bytes? }` |
| `cap-end` | `{ pack, jobId/sessionId, … }` |
| `cap-error` | `{ pack, jobId/sessionId, message }` |
These events are broadcast to all subscribed tabs (no `swarmId` filter).
+28 -8
View File
@@ -11,23 +11,30 @@ await BridgeSwarm.capabilities.has('media')
// Generic dispatch
await BridgeSwarm.capabilities.call('media', 'info', { path: '…' })
// Thin wrappers
// Batch wrappers
await BridgeSwarm.media.info({ dataBase64, filename })
await BridgeSwarm.media.imageTransform({ dataBase64, maxWidth: 640, mimetype: 'image/webp' })
await BridgeSwarm.media.extractFrame({ dataBase64, frameIndex: 0 })
await BridgeSwarm.media.transcode({ dataBase64, format: 'webm' }, { onProgress })
await BridgeSwarm.media.cancel(jobId)
// Live encode (host bare-ffmpeg → MSE)
const live = BridgeSwarm.media.liveSession(videoEl)
await live.start({ width: 640, height: 360, fps: 10, ingest: 'frames' })
await live.pushFrame(canvas)
await live.subscribe({ swarm: { connIds: [conn.connId] } })
await live.stop()
```
Streaming results use host events (under the ~1 MB native-messaging limit):
| Event | Payload |
|-------|---------|
| `cap-chunk` | `{ pack, jobId, index?, data? (base64), progress?, bytes? }` |
| `cap-end` | `{ pack, jobId, chunks?, path?, mimetype?, … }` |
| `cap-error` | `{ pack, jobId, message }` |
| `cap-chunk` | `{ pack, jobId/sessionId, kind?, index?, data? (base64), progress?, bytes? }` |
| `cap-end` | `{ pack, jobId/sessionId, chunks?, path?, mimetype?, … }` |
| `cap-error` | `{ pack, jobId/sessionId, message }` |
`BridgeSwarm.media.*` helpers assign a `jobId`, listen for these events, and resolve with assembled `dataBase64` (or a host-relative `path` for large transcodes).
Live sessions emit `cap-chunk` with `kind: 'segment'` (WebM fragments for MSE) or `kind: 'drop'` under backpressure.
```js
const off = BridgeSwarm.capabilities.on('cap-chunk', (p) => console.log(p))
@@ -53,16 +60,29 @@ Shipped in the standard `bridge-swarm-host` artifact (`bare-media` + `bare-ffmpe
| `media.imageTransform` | Decode → optional resize/crop → encode (webp/jpeg/png); stream base64 |
| `media.extractFrame` | Extract one video frame → encode; stream base64 |
| `media.transcode` | Async job to webm/mp4/mkv (VP9+Opus where supported); progress events |
| `media.cancel` | Cancel in-flight job |
| `media.encodeStart` | Start live VP9/WebM session (`ingest: frames\|segments`, egress page/swarm/file) |
| `media.encodePushFrame` | Push JPEG/WebP frame (base64) into live session |
| `media.encodePush` | Push MediaRecorder timeslice or frame alias |
| `media.encodeSubscribe` | Update fan-out `connIds` / page egress mid-session |
| `media.encodeStop` | Flush encoder, finalize archive, `cap-end` |
| `media.cancel` | Cancel batch job or live session |
| `media.writeInput` / `media.readOutput` | Chunked upload / download under `cap-jobs/` |
After installing or updating the host on macOS, run `npm run repair:macos` (or `npm run install:capability:media`, which now aliases repair) so new `.bare` addons are extracted and codesigned. Fully quit the browser, then:
### Live encode notes
- Chrome NMH ~1 MB/message → ingest **compressed JPEG frames** (or WebM slices), not raw RGBA.
- Defaults for demos: **640×360 @ 10fps**; soft max ~720p @ 15fps; session max duration 10 minutes.
- Egress: `page` (MSE segments), `swarm` (BSML-framed binary on Hyperswarm sockets), `file` (`cap-jobs/<sessionId>/live.webm`).
- Peer fan-out writes on the **host socket** (good for bitrates); the receiving page still reads via NMH `conn.on('data')` unless you stay host-side.
- Helpers: `BridgeSwarm.media.liveSession(videoEl)` and `BridgeSwarm.media.attachLiveReceiver(conn, videoEl)`.
After installing or updating the host on macOS, run `npm run repair:macos` so `.bare` addons (including ffmpeg) are extracted and codesigned. Fully quit the browser, then:
```js
await BridgeSwarm.capabilities.has('media') // true
```
Demo: [`examples/media-demo/`](../examples/media-demo/) (`npm run examples``/media-demo/`).
Demos: [`examples/live-encode/`](../examples/live-encode/), [`examples/media-demo/`](../examples/media-demo/), [`examples/clip-studio/`](../examples/clip-studio/).
## Bundled for upcoming packs
+1 -1
View File
@@ -30,7 +30,7 @@ Curated modules shipped in the **default** BridgeSwarm native host. Not every `b
| Module | Why |
|--------|-----|
| `bare-media` | High-level image/video API for `BridgeSwarm.media.*` |
| `bare-ffmpeg` | Transcode / frame extract (pulled by bare-media; listed explicitly) |
| `bare-ffmpeg` | Batch transcode **and** live VP9/WebM encode sessions (`encodeStart` / `encodePushFrame`) |
| Image codecs | Via bare-media: `bare-jpeg`, `bare-png`, `bare-webp`, `bare-gif`, `bare-heif`, `bare-bmp`, `bare-ico`, `bare-tiff`, `bare-svg`, `bare-image-resample`, `bare-exif` |
### Local power (bundled for capability packs)