pearcord/pearcord-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: v0.7.8 live voice audio (Phase 32, P3-3 complete)

Browse Source 
Update VOICE_MEDIA.md for Opus/pcm frames and udx transport. Roadmap Phase 32,
parity P3-3 marked done, module map + Gitea org list for pearcord-voice-codec.

Co-authored-by: Cursor <[email protected]>
This commit is contained in:
Raven Scott
2026-05-21 21:59:49 -04:00
co-authored by Cursor
parent 621968eb57
commit cffd5d0de4
5 changed files with 45 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
GITEA_DEVELOPER.md
+1
View File
@@ -116,6 +116,7 @@ git remote -v
| `pearcord-forum` | Forum post tags + palette helpers | | `pearcord-forum` | Forum post tags + palette helpers |
| `pearcord-stage` | Stage channel speaker roles + `STAGE_SPEAKER_UPDATE` gossip | | `pearcord-stage` | Stage channel speaker roles + `STAGE_SPEAKER_UPDATE` gossip |
| `pearcord-automod` | Local automod rules + `AUTOMOD_CONFIG_UPDATE` gossip | | `pearcord-automod` | Local automod rules + `AUTOMOD_CONFIG_UPDATE` gossip |
| `pearcord-voice-codec` | Opus/pcm voice frames for live P2P audio |
| `pearcord-device-sync` | `pcdv_` multi-device pairing | | `pearcord-device-sync` | `pcdv_` multi-device pairing |
| `pearcord-emoji` | Guild emoji registry + `EMOJI_UPSERT` gossip | | `pearcord-emoji` | Guild emoji registry + `EMOJI_UPSERT` gossip |
| `pearcord-sync` | Planned Autobase bridge | | `pearcord-sync` | Planned Autobase bridge |
MODULES.md
+2 -1
View File
@@ -10,7 +10,8 @@
| `pearcord-message` | `modules/pearcord-message` | Send/history | | `pearcord-message` | `modules/pearcord-message` | Send/history |
| `pearcord-presence` | `modules/pearcord-presence` | Status gossip | | `pearcord-presence` | `modules/pearcord-presence` | Status gossip |
| `pearcord-voice` | `modules/pearcord-voice` | Voice occupancy + gossip | | `pearcord-voice` | `modules/pearcord-voice` | Voice occupancy + gossip |
| `pearcord-voice-media` | `modules/pearcord-voice-media` | P2P voice media (protomux) | | `pearcord-voice-media` | `modules/pearcord-voice-media` | P2P voice media (protomux + audio) |
| `pearcord-voice-codec` | `modules/pearcord-voice-codec` | Opus/pcm voice frame codec |
| `pearcord-discovery` | `modules/pearcord-discovery` | Local directory + P2P public listings mesh | | `pearcord-discovery` | `modules/pearcord-discovery` | Local directory + P2P public listings mesh |
| `pearcord-embed` | `modules/pearcord-embed` | OpenGraph fetch + cache | | `pearcord-embed` | `modules/pearcord-embed` | OpenGraph fetch + cache |
| `pearcord-crypto` | `modules/pearcord-crypto` | DM payload sealing | | `pearcord-crypto` | `modules/pearcord-crypto` | DM payload sealing |
PEARCORD_PARITY.md
+5 -4
View File
@@ -4,7 +4,7 @@ Gap analysis between **Pearcord** (P2P Pear app) and a typical **centralized cha
> This is the only doc that compares against an external reference product. All other docs use Pearcord naming only. > This is the only doc that compares against an external reference product. All other docs use Pearcord naming only.
## Summary (v0.7.7) ## Summary (v0.7.8)
| Area | Reference product | Pearcord | Gap | | Area | Reference product | Pearcord | Gap |
|------|-------------------|----------|-----| |------|-------------------|----------|-----|
@@ -12,7 +12,7 @@ Gap analysis between **Pearcord** (P2P Pear app) and a typical **centralized cha
| Servers | Hosted guilds | P2P guild topic + local DB + **public discovery mesh** | No curated global index (fully P2P topic) | | Servers | Hosted guilds | P2P guild topic + local DB + **public discovery mesh** | No curated global index (fully P2P topic) |
| Text chat | Full | … **announcement** + **forum** channels, tags, forum slowmode, threads, typing, **guild emoji** | animated stickers | | Text chat | Full | … **announcement** + **forum** channels, tags, forum slowmode, threads, typing, **guild emoji** | animated stickers |
| DMs | E2E optional | … **sidebar previews**, **group DM (3–10)**, read-on-open | Full E2E group vault | | DMs | E2E optional | … **sidebar previews**, **group DM (3–10)**, read-on-open | Full E2E group vault |
| Voice/video | WebRTC SFU | Voice + **stage** (audience/speaker, request to speak) + **P2P protomux media** | Opus/udx live audio (P3-3 follow-up) | | Voice/video | WebRTC SFU | Voice + **stage** + **Opus/pcm live audio** (20 ms P2P frames, optional udx) | screen share, real mic in Pear UI |
| Moderation | Roles, bans, automod | Kick, ban, timeout, roles UI, slowmode, audit+gossip; **local automod** (keywords, mentions, invites); step-up PIN | ML/heuristic cloud automod | | Moderation | Roles, bans, automod | Kick, ban, timeout, roles UI, slowmode, audit+gossip; **local automod** (keywords, mentions, invites); step-up PIN | ML/heuristic cloud automod |
| Account security | 2FA, session revoke | Local step-up PIN (`pearcord-step-up`, 5 min session) | Cloud 2FA, OAuth | | Account security | 2FA, session revoke | Local step-up PIN (`pearcord-step-up`, 5 min session) | Cloud 2FA, OAuth |
| Friends | Friend list + requests | **P2P contacts** + presence + **custom status** on contacts mesh | Activity feed, rich presence games | | Friends | Friend list + requests | **P2P contacts** + presence + **custom status** on contacts mesh | Activity feed, rich presence games |
@@ -100,10 +100,11 @@ Phases **0–2** and **4** are largely complete; **3** (live audio) and **5** (n
| P10-4 | HyperDB schema v2 — P8-4 | [x] v0.5.7 schema v9 | | P10-4 | HyperDB schema v2 — P8-4 | [x] v0.5.7 schema v9 |
| P11-1 | Guild-wide search — P5-3 | [x] v0.5.7 | | P11-1 | Guild-wide search — P5-3 | [x] v0.5.7 |
| P11-2 | Contact block gossip | [x] v0.5.7 | | P11-2 | Contact block gossip | [x] v0.5.7 |
| P10-5 | Opus/udx live voice — P3-3 | | P10-5 | Opus/udx live voice — P3-3 | [x] v0.7.8 |
| P31-1 | Local automod (keywords, mentions, invites) | [x] v0.7.7 | | P31-1 | Local automod (keywords, mentions, invites) | [x] v0.7.7 |
| P32-1 | Live voice audio codec + capture | [x] v0.7.8 |
**Next tranches:** P3-3 Opus/udx live audio · P3-4 screen share · announcement follow/cross-post · P5-7 local cosmetic boosts. **Next tranches:** P3-4 screen share · announcement follow/cross-post · P5-7 local cosmetic boosts · Pear mic capture.
When Phase 5 checklist is `[x]`, copy open items here into tracked rows in `PLATFORM_ROADMAP.md`. When Phase 5 checklist is `[x]`, copy open items here into tracked rows in `PLATFORM_ROADMAP.md`.
PLATFORM_ROADMAP.md
+13 -1
View File
@@ -59,7 +59,7 @@ Track implementation status here. Update checkboxes as phases land.
|----|------|--------| |----|------|--------|
| P3-1 | `pearcord-voice` — signaling mesh | [x] occupancy + media hub | | P3-1 | `pearcord-voice` — signaling mesh | [x] occupancy + media hub |
| P3-2 | Voice channels join/leave | [x] | | P3-2 | Voice channels join/leave | [x] |
| P3-3 | WebRTC / udx-native media path | [~] protomux data plane + ping RTT; udx optional | | P3-3 | WebRTC / udx-native media path | [x] Opus/pcm live audio + optional udx streams (v0.7.8) |
| P3-4 | Screen share | [ ] | | P3-4 | Screen share | [ ] |
| P3-5 | Stage channels | [x] audience/speaker roles + request to speak (v0.7.6) | | P3-5 | Stage channels | [x] audience/speaker roles + request to speak (v0.7.6) |
@@ -143,6 +143,7 @@ See [PEARCORD_PARITY.md](./PEARCORD_PARITY.md) for full gap analysis and Phases
| `pearcord-forum` | Forum post tags + palette on channel topic | | `pearcord-forum` | Forum post tags + palette on channel topic |
| `pearcord-stage` | Stage speaker roles (audience/speaker/pending) | | `pearcord-stage` | Stage speaker roles (audience/speaker/pending) |
| `pearcord-automod` | Local automod rules + `AUTOMOD_CONFIG_UPDATE` gossip | | `pearcord-automod` | Local automod rules + `AUTOMOD_CONFIG_UPDATE` gossip |
| `pearcord-voice-codec` | Opus/pcm 20 ms voice frames for P2P audio |
--- ---
@@ -235,6 +236,16 @@ When completing work, change `[ ]` → `[x]` and add a one-line note under **Cha
--- ---
### Phase 32 — Live voice audio (v0.7.8)
| ID | Item | Status |
|----|------|--------|
| P32-1 | `pearcord-voice-codec` Opus/pcm 48 kHz frames | [x] |
| P32-2 | `audio` protomux frames + capture loop + speaking stats | [x] |
| P32-3 | Optional udx-native transport + smoke-voice-audio | [x] |
---
### Phase 31 — Automod (v0.7.7) ### Phase 31 — Automod (v0.7.7)
| ID | Item | Status | | ID | Item | Status |
@@ -398,6 +409,7 @@ When completing work, change `[ ]` → `[x]` and add a one-line note under **Cha
### Changelog ### Changelog
- **2026-05-21** — v0.7.8: live voice audio (`pearcord-voice-codec`, Opus/pcm `audio` frames, capture loop, speaking peers, optional udx transport, smoke-voice-audio, VOICE_MEDIA.md).
- **2026-05-21** — v0.7.7: `pearcord-automod` local rules (keywords, mentions, invite links), `AUTOMOD_CONFIG_UPDATE` gossip, server settings UI, smoke-automod, AUTOMOD.md. - **2026-05-21** — v0.7.7: `pearcord-automod` local rules (keywords, mentions, invite links), `AUTOMOD_CONFIG_UPDATE` gossip, server settings UI, smoke-automod, AUTOMOD.md.
- **2026-05-21** — v0.7.6: stage channels (`pearcord-stage`, STAGE_SPEAKER_UPDATE gossip, request/approve speakers, stage bar UI, smoke-stage, STAGE_CHANNELS.md). - **2026-05-21** — v0.7.6: stage channels (`pearcord-stage`, STAGE_SPEAKER_UPDATE gossip, request/approve speakers, stage bar UI, smoke-stage, STAGE_CHANNELS.md).
- **2026-05-21** — v0.7.5: announcement channels (mod-only post), UI bar, smoke-announcement, ANNOUNCEMENT_CHANNELS.md. - **2026-05-21** — v0.7.5: announcement channels (mod-only post), UI bar, smoke-announcement, ANNOUNCEMENT_CHANNELS.md.
VOICE_MEDIA.md
+24 -14
View File
@@ -1,16 +1,20 @@
# Voice media (v0.4.9+) # Voice media (v0.4.9+ / v0.7.8 live audio)
P2P voice **media plane** on the existing guild Hyperswarm mesh (no central SFU). P2P voice **media plane** on the existing guild Hyperswarm mesh (no central SFU).
## Module ## Modules
`modules/pearcord-voice-media` | Module | Role |
|--------|------|
| `pearcord-voice-media` | Sessions, protomux mesh, capture loop, udx transport |
| `pearcord-voice-codec` | Opus (`opusscript`, Bare-safe) or `pcm-linear` fallback |
| Piece | Role | | Piece | Role |
|-------|------| |-------|------|
| `VoiceMediaHub` | Per-user session, peers, mute/deafen, RTT stats | | `VoiceMediaHub` | Per-user session, peers, mute/deafen, RTT + audio stats |
| `VoiceAudioPlane` | 20 ms capture, encode, `audio` frames, speaking detection |
| `mesh.js` | Protomux protocol `pearcord-voice-v1` on each guild connection | | `mesh.js` | Protomux protocol `pearcord-voice-v1` on each guild connection |
| `udx-native` | Optional; detected at runtime for future UDP audio | | `udx-transport.js` | Optional udx-native streams for binary audio when binding loads |
## Signaling vs media ## Signaling vs media
@@ -20,30 +24,36 @@ P2P voice **media plane** on the existing guild Hyperswarm mesh (no central SFU)
| Media ready | `VOICE_MEDIA_READY` (23) | Session id + channel for P2P mesh | | Media ready | `VOICE_MEDIA_READY` (23) | Session id + channel for P2P mesh |
| Mute | `VOICE_MEDIA_MUTE` (24) | Mute/deafen state | | Mute | `VOICE_MEDIA_MUTE` (24) | Mute/deafen state |
| Leave | `VOICE_MEDIA_LEAVE` (25) | End media session | | Leave | `VOICE_MEDIA_LEAVE` (25) | End media session |
| Data plane | `pearcord-voice-v1` | `ready`, `ping`/`pong`, `mute`, `leave` JSON frames | | Control | `pearcord-voice-v1` | `ready`, `ping`/`pong`, `mute`, `leave`, `udx-offer`/`udx-answer` |
| Audio | `pearcord-voice-v1` | `audio` — Opus/pcm frames (base64 in JSON, 20 ms @ 48 kHz mono) |
Joining voice runs occupancy gossip **and** announces media readiness, then sends an initial ping on open protomux channels. Joining voice runs occupancy gossip **and** announces media readiness, starts the capture loop, and sends an initial ping on open protomux channels. When `udx-native` is available, peers may negotiate a UDP stream for binary audio after `ready`.
## Platform API ## Platform API
| Method | Description | | Method | Description |
|--------|-------------| |--------|-------------|
| `joinVoiceChannel(id)` | Signaling + media join + gossip | | `joinVoiceChannel(id)` | Signaling + media join + gossip + audio capture |
| `leaveVoiceChannel()` | Signaling + media leave | | `leaveVoiceChannel()` | Signaling + media leave + stop capture |
| `setVoiceMute({ muted, deafened })` | Local + gossip + protomux | | `setVoiceMute({ muted, deafened })` | Local + gossip + protomux |
`view.voiceMedia`: `{ connected, sessionId, engine, rttMs, peers, muted, deafened, packetsIn, packetsOut }` `view.voiceMedia`: `{ connected, sessionId, engine, codec, opusAvailable, udxAvailable, rttMs, peers, muted, deafened, packetsIn, packetsOut, audioFramesIn, audioFramesOut, speakingPeers }`
## UI ## UI
Voice bar shows P2P engine + RTT, **Mute** and **Deafen** toggles, **Disconnect**. Voice bar shows codec (Opus when available), P2P engine + RTT, frame counters, **Speaking** names, **Mute** / **Deafen**, **Disconnect**.
## Tests ## Tests
`npm run test:voice-media` → `VOICE_MEDIA_SMOKE_OK` ```bash
cd apps/pearcord
npm run test:voice-media # VOICE_MEDIA_SMOKE_OK
npm run test:voice-audio # VOICE_AUDIO_SMOKE_OK
npm run test:boot
```
## Future (P3-4+) ## Future (P3-4+)
- Opus frames over udx-native streams when binding available in Pear/Bare
- Screen share (separate topic) - Screen share (separate topic)
- Stage channel semantics - Real microphone capture in Pear/Electron (replace synthetic tone)
- Full udx path through NAT via blind-relay when needed