speech

SpeechServerEngine

SpeechServerEngine implements ReadiumSpeechPlaybackEngine (see Playback.md) against a remote TTS HTTP service — a Readium Speech Server instance. SpeechServerEngineProvider wraps it for use with a single provider or ReadiumSpeechProviderRegistry.

Construction

Via the provider (recommended — handles voice caching for you):

import { SpeechServerEngineProvider } from "@readium/speech";

const provider = new SpeechServerEngineProvider({
  endpoints: {
    voices: "http://localhost:8000/voices",
    synthesize: "http://localhost:8000/synthesize",
    service: "http://localhost:8000/service"
  }
});
const engine = await provider.createEngine();

Directly, if you don’t need the registry or provider-level voice caching:

import { SpeechServerEngine } from "@readium/speech";

const engine = new SpeechServerEngine({
  endpoints: { voices: "...", synthesize: "...", service: "..." }
});

SpeechServerEngineProviderOptions is the same shape as SpeechServerEngineOptions below — every option documented here works identically through either constructor.

Options

interface SpeechServerEngineOptions {
  endpoints: {
    voices: string;      // GET  — list available voices
    synthesize: string;  // POST — synthesize one utterance/chunk
    service: string;     // GET  — server capabilities (formats, maxTextLength, ...)
  };
  fetch?: typeof fetch;         // default: fetch bound to globalThis
  prefetchWindow?: number;      // utterances to keep pre-fetched ahead of playback, default 3
  readyBufferChars?: number;    // combined chars to buffer before "ready", default 400
  overLengthText?: "split" | "error"; // default "split"
  format?: SpeechServerFormatOptions;
  timeoutMs?: number; // default: undefined (never declares a stall)
}

Format selection

interface SpeechServerFormatOptions {
  preferredFormat?: string;         // e.g. "opus" — used only if advertised + browser-playable
  strategy?: "quality" | "bandwidth"; // default "quality"
  adaptBitrateToNetwork?: boolean;  // default false
}

By default the engine picks a format from the intersection of what the server advertises (/service’s output.formats) and what the browser can actually decode (HTMLAudioElement.canPlayType), ranked "quality" (lossless/higher-fidelity first) or "bandwidth" (smallest-transfer first). preferredFormat overrides that ranking outright, as long as it’s both advertised and playable — an unsupported preference is silently ignored rather than erroring.

adaptBitrateToNetwork reduces the requested bitrate for compressed formats when the browser’s Network Information API (navigator.connection, Chromium-only) reports Save-Data or a 2G-class connection. It’s opt-in and off by default, since that API doesn’t exist in every browser and this must never silently change behavior for callers who didn’t ask for it.

Errors

Failures surface as "error" events (see Playback.md). Every detail has at least { message, recoverable }; when the server rejected a request with an RFC 9457 Problem Details response, detail also carries { status, type, title, instance }. A network failure or a stall (see below) has no response to carry Problem Details from, so those only ever have { message, recoverable } — and so does SpeechServerAudioDecodeError, which does get a response, just one whose audio payload fails to decode rather than a Problem Details body. recoverable is true when the server never responded at all (network failure, or a stall), and false when it responded but rejected the request, or the audio payload couldn’t be decoded — see FallbackEngine.md, which uses this to decide whether swapping to another engine could help.

Stall detection

timeoutMs is a grace period, not a per-request cap: /synthesize requests aren’t given a flat timeout, because prefetching and gapless scheduling mean a single slow chunk is harmless as long as there’s still enough already-buffered audio ahead of the playhead to cover it. Instead, the engine only starts a clock once the buffer is projected to run dry with the next chunk still not ready, and only declares a stall — throwing a SpeechServerStallError (408, .../error#stall) and aborting that specific request — if timeoutMs elapses past that point. Every chunk that does resolve pushes the projected buffer-empty point further out, so the deadline effectively rolls forward with playback rather than being fixed at request start.

Left undefined (the default), a stalled /synthesize request waits forever, same as before this option existed.

Minimizing gaps between utterances

Known limitations

/synthesize returns a complete base64-encoded audio payload per request, not a stream — the engine waits for the full response, then decodes it whole via AudioContext.decodeAudioData. There’s no Media Source Extensions (MSE) usage and no incremental playback within a single chunk. prefetchWindow/readyBufferChars/overLengthText: "split" reduce the perceived gap between utterances by overlapping requests and keeping individual requests small, but a single long chunk still has to finish downloading before any of it can play.