view dictation/README.md @ 282:6bd4a3990696 default tip

Add cross-platform canvas orchestration and typed composer Enable native ARM dependencies, macOS Copilot orchestration, portable service supervision, CPU dictation, and a shared N-key speak-or-type composer. Co-authored-by: Copilot <[email protected]> Copilot-Session: f68442b1-fa8f-46a0-9689-81710613bbd4
author MrJuneJune <me@mrjunejune.com>
date Thu, 20 Aug 2026 20:45:25 -0700
parents 49e9e591c9bb
children
line wrap: on
line source

# WebRTC dictation

This Bazel package runs a local WebRTC speech-to-text service using aiortc and
faster-whisper. The default multilingual Whisper `small` model uses CUDA with
`int8_float16` on Linux and CPU inference with `int8` on macOS.

## Setup

```bash
bazel run //dictation:preflight
bazel run //dictation:download_model
bazel run //dictation:server
```

Open <http://127.0.0.1:8090>, grant microphone permission, and start
dictation. Audio is carried by WebRTC and partial/final transcript events are
returned on the `transcripts` RTC data channel.

To transcribe an existing audio file:

```bash
bazel run //dictation:transcribe -- /path/to/audio.wav
```

To exercise the running HTTP signaling and WebRTC path with an audio file:

```bash
bazel run //dictation:webrtc_smoke -- /path/to/audio.wav
```

The model is stored under `~/.cache/zenbu/faster-whisper-small` by default.
Override it with `DICTATION_MODEL_DIR`.
Set `DICTATION_DEVICE=cpu|cuda` to override platform selection.

Useful tuning:

```bash
DICTATION_COMPUTE_TYPE=int8_float16 \
DICTATION_MAX_SESSIONS=1 \
DICTATION_PARTIAL_INTERVAL_MS=500 \
DICTATION_SILENCE_MS=400 \
  bazel run //dictation:server
```

The first version intentionally binds to loopback and does not configure STUN
or TURN. Remote WebRTC access requires HTTPS, authentication, origin policy,
and usually a TURN service.

Infinite Canvas hosts this page as a hidden CEF/iframe transport. Pressing `M`
shows partial and final transcript text in the retained Dictation entity;
pressing `Enter` sends the accumulated text to Copilot orchestration:

```bash
bazel run //infinite_canvas:agent_dev
```