LFM3-Audio-2B streaming voice demo

Web UI + FastAPI WebSocket server for LiquidAI/LFM3-Audio-2B: speak into the browser, get streamed text and speech back, with debug data for each turn.

Run

uv sync
uv run python server.py        # http://127.0.0.1:8000

Hold the button (or Space) to talk; release to send, or type a message and press Enter. Modes: voice chat (interleaved text + speech, multi-turn history), voice + tools, ASR, and a TTS button.

Tool calling

"Voice + tools" mode passes the schemas in tools.py to the chat template, with the system prompt Respond with function calls, and interleaved text and audio.\n\nList of tools: and no prefill. The model either emits <tool_call><function=…><parameter=…> blocks or answers with speech. The server parses the calls, runs them, appends the assistant tool_calls message and a tool result message, and generates again (up to 4 rounds). Test tools:

get_current_time(timezone): real clock
get_weather(city): simulated, stable per city
calculate(expression): safe arithmetic evaluator
set_theme_color(color): changes the page's accent color, so a successful call is visible right away

Calls and results appear as cards in the chat and in the debug panel's "Tool calls" box; the raw markup shows in the token stream.

Force speech (on by default): with greedy text decoding the model sometimes answers in plain text without speech. When on, rounds after a tool result are prefilled with <|mixed_start|>, and a first round whose first token isn't <tool_call> or a speech marker is stopped and restarted with that prefill (about 230 ms extra). The debug panel counts these as "speech restarts".

How it streams

Mic → server: AudioWorklet posts float32 PCM at the browser's native rate over the WebSocket while you hold; the server resamples to 16 kHz on release.
Server → browser: model.generate(streamer=…, audio_streamer=…) runs in a worker thread. Text tokens are sent as JSON events. Speech frames (12 codebooks, 80 ms each) go to a separate decoder thread that uses LiquidCodecStreamingCache and sends 24 kHz float32 PCM as binary messages, every LFM_CHUNK_FRAMES frames (default 3, i.e. 240 ms). The browser schedules them gaplessly.

Debug panel

Per turn: input duration/RMS, prompt tokens, time to first text token / speech frame / audio chunk, generate time, RTF (gen time ÷ audio seconds, < 1 = faster than real time), frames/s (12.5 = real time), codec decode time and backlog, and playback underruns. There is also a client-side timeline, a raw token stream with special tokens highlighted, the exact prompt, and an event log. Each reply and recording gets a playable/downloadable WAV.

Config (env vars)

var	default
`LFM_DEVICE`	cuda → mps → cpu	LM device
`LFM_DTYPE`	bfloat16 (float32 on cpu)	LM dtype
`LFM_CODEC_DEVICE`	cpu	codec device
`LFM_CHUNK_FRAMES`	3	frames per audio chunk sent
`HOST` / `PORT`	127.0.0.1 / 8000

Scripts

scripts/smoke_test.py: headless TTS → ASR → chat → ASR round trip; writes smoke_out/*.wav.
scripts/ws_client_test.py <wav>: drives the WebSocket like the browser does (2 turns).
scripts/tools_test.py: typed and spoken tool-calling requests against a running server.

3.3 KiB Raw Blame History