This commit is contained in:
@@ -28,7 +28,7 @@ These are the main APIs your page uses. For host request types and event payload
|
||||
|
||||
- **`BridgeSwarm.capabilities`** — Bare capability packs on the host (`media` is default). `list()`, `has(pack)`, `call(pack, cmd, payload)`, `on(event, fn)`. See [CAPABILITIES.md](CAPABILITIES.md) / [DEFAULT-MODULES.md](DEFAULT-MODULES.md).
|
||||
|
||||
- **`BridgeSwarm.media.*`** — Media pack wrappers (`info`, `imageTransform`, `extractFrame`, `transcode`, `cancel`, `writeInput`, `readOutput`).
|
||||
- **`BridgeSwarm.media.*`** — Media pack: batch (`info`, `imageTransform`, `extractFrame`, `transcode`, …) and live encode (`encodeStart`, `encodePushFrame`, `encodeStop`, `liveSession`, `attachLiveReceiver`). See [CAPABILITIES.md](CAPABILITIES.md).
|
||||
---
|
||||
|
||||
## Host request types and events
|
||||
@@ -145,14 +145,15 @@ Full details and examples: [DATA-API.md](DATA-API.md).
|
||||
| `capabilities.has` | `{ pack }` | `{ ok, pack, has }` |
|
||||
| `capability` | `{ pack, cmd, payload }` | pack-specific |
|
||||
| `media.info` / `media.imageTransform` / `media.extractFrame` / `media.transcode` / `media.cancel` / … | command payload | see [CAPABILITIES.md](CAPABILITIES.md) |
|
||||
| `media.encodeStart` / `encodePushFrame` / `encodePush` / `encodeSubscribe` / `encodeStop` | live session payload | `{ ok, sessionId, … }` + `cap-chunk` segments |
|
||||
|
||||
### Capability events
|
||||
|
||||
| Event | Payload |
|
||||
|-------|---------|
|
||||
| `cap-chunk` | `{ pack, jobId, index?, data?, progress?, bytes? }` |
|
||||
| `cap-end` | `{ pack, jobId, … }` |
|
||||
| `cap-error` | `{ pack, jobId, message }` |
|
||||
| `cap-chunk` | `{ pack, jobId/sessionId, kind?, seq?, index?, data?, progress?, bytes? }` |
|
||||
| `cap-end` | `{ pack, jobId/sessionId, … }` |
|
||||
| `cap-error` | `{ pack, jobId/sessionId, message }` |
|
||||
|
||||
These events are broadcast to all subscribed tabs (no `swarmId` filter).
|
||||
|
||||
|
||||
+28
-8
@@ -11,23 +11,30 @@ await BridgeSwarm.capabilities.has('media')
|
||||
// Generic dispatch
|
||||
await BridgeSwarm.capabilities.call('media', 'info', { path: '…' })
|
||||
|
||||
// Thin wrappers
|
||||
// Batch wrappers
|
||||
await BridgeSwarm.media.info({ dataBase64, filename })
|
||||
await BridgeSwarm.media.imageTransform({ dataBase64, maxWidth: 640, mimetype: 'image/webp' })
|
||||
await BridgeSwarm.media.extractFrame({ dataBase64, frameIndex: 0 })
|
||||
await BridgeSwarm.media.transcode({ dataBase64, format: 'webm' }, { onProgress })
|
||||
await BridgeSwarm.media.cancel(jobId)
|
||||
|
||||
// Live encode (host bare-ffmpeg → MSE)
|
||||
const live = BridgeSwarm.media.liveSession(videoEl)
|
||||
await live.start({ width: 640, height: 360, fps: 10, ingest: 'frames' })
|
||||
await live.pushFrame(canvas)
|
||||
await live.subscribe({ swarm: { connIds: [conn.connId] } })
|
||||
await live.stop()
|
||||
```
|
||||
|
||||
Streaming results use host events (under the ~1 MB native-messaging limit):
|
||||
|
||||
| Event | Payload |
|
||||
|-------|---------|
|
||||
| `cap-chunk` | `{ pack, jobId, index?, data? (base64), progress?, bytes? }` |
|
||||
| `cap-end` | `{ pack, jobId, chunks?, path?, mimetype?, … }` |
|
||||
| `cap-error` | `{ pack, jobId, message }` |
|
||||
| `cap-chunk` | `{ pack, jobId/sessionId, kind?, index?, data? (base64), progress?, bytes? }` |
|
||||
| `cap-end` | `{ pack, jobId/sessionId, chunks?, path?, mimetype?, … }` |
|
||||
| `cap-error` | `{ pack, jobId/sessionId, message }` |
|
||||
|
||||
`BridgeSwarm.media.*` helpers assign a `jobId`, listen for these events, and resolve with assembled `dataBase64` (or a host-relative `path` for large transcodes).
|
||||
Live sessions emit `cap-chunk` with `kind: 'segment'` (WebM fragments for MSE) or `kind: 'drop'` under backpressure.
|
||||
|
||||
```js
|
||||
const off = BridgeSwarm.capabilities.on('cap-chunk', (p) => console.log(p))
|
||||
@@ -53,16 +60,29 @@ Shipped in the standard `bridge-swarm-host` artifact (`bare-media` + `bare-ffmpe
|
||||
| `media.imageTransform` | Decode → optional resize/crop → encode (webp/jpeg/png); stream base64 |
|
||||
| `media.extractFrame` | Extract one video frame → encode; stream base64 |
|
||||
| `media.transcode` | Async job to webm/mp4/mkv (VP9+Opus where supported); progress events |
|
||||
| `media.cancel` | Cancel in-flight job |
|
||||
| `media.encodeStart` | Start live VP9/WebM session (`ingest: frames\|segments`, egress page/swarm/file) |
|
||||
| `media.encodePushFrame` | Push JPEG/WebP frame (base64) into live session |
|
||||
| `media.encodePush` | Push MediaRecorder timeslice or frame alias |
|
||||
| `media.encodeSubscribe` | Update fan-out `connIds` / page egress mid-session |
|
||||
| `media.encodeStop` | Flush encoder, finalize archive, `cap-end` |
|
||||
| `media.cancel` | Cancel batch job or live session |
|
||||
| `media.writeInput` / `media.readOutput` | Chunked upload / download under `cap-jobs/` |
|
||||
|
||||
After installing or updating the host on macOS, run `npm run repair:macos` (or `npm run install:capability:media`, which now aliases repair) so new `.bare` addons are extracted and codesigned. Fully quit the browser, then:
|
||||
### Live encode notes
|
||||
|
||||
- Chrome NMH ~1 MB/message → ingest **compressed JPEG frames** (or WebM slices), not raw RGBA.
|
||||
- Defaults for demos: **640×360 @ 10fps**; soft max ~720p @ 15fps; session max duration 10 minutes.
|
||||
- Egress: `page` (MSE segments), `swarm` (BSML-framed binary on Hyperswarm sockets), `file` (`cap-jobs/<sessionId>/live.webm`).
|
||||
- Peer fan-out writes on the **host socket** (good for bitrates); the receiving page still reads via NMH `conn.on('data')` unless you stay host-side.
|
||||
- Helpers: `BridgeSwarm.media.liveSession(videoEl)` and `BridgeSwarm.media.attachLiveReceiver(conn, videoEl)`.
|
||||
|
||||
After installing or updating the host on macOS, run `npm run repair:macos` so `.bare` addons (including ffmpeg) are extracted and codesigned. Fully quit the browser, then:
|
||||
|
||||
```js
|
||||
await BridgeSwarm.capabilities.has('media') // true
|
||||
```
|
||||
|
||||
Demo: [`examples/media-demo/`](../examples/media-demo/) (`npm run examples` → `/media-demo/`).
|
||||
Demos: [`examples/live-encode/`](../examples/live-encode/), [`examples/media-demo/`](../examples/media-demo/), [`examples/clip-studio/`](../examples/clip-studio/).
|
||||
|
||||
## Bundled for upcoming packs
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ Curated modules shipped in the **default** BridgeSwarm native host. Not every `b
|
||||
| Module | Why |
|
||||
|--------|-----|
|
||||
| `bare-media` | High-level image/video API for `BridgeSwarm.media.*` |
|
||||
| `bare-ffmpeg` | Transcode / frame extract (pulled by bare-media; listed explicitly) |
|
||||
| `bare-ffmpeg` | Batch transcode **and** live VP9/WebM encode sessions (`encodeStart` / `encodePushFrame`) |
|
||||
| Image codecs | Via bare-media: `bare-jpeg`, `bare-png`, `bare-webp`, `bare-gif`, `bare-heif`, `bare-bmp`, `bare-ico`, `bare-tiff`, `bare-svg`, `bare-image-resample`, `bare-exif` |
|
||||
|
||||
### Local power (bundled for capability packs)
|
||||
|
||||
Reference in New Issue
Block a user