Skip to content

Generate and Speak

The Generate-and-Speak node fuses LLM token generation with streaming TTS synthesis: it runs the same projection and language-model inference as a Generate node, but as tokens arrive it segments them into sentences, dispatches each sentence to a dedicated TTS strand, and emits PCM audio frames alongside the text deltas.

Because synthesis overlaps generation, audio begins after roughly the first sentence (min_sentence_chars) rather than after the whole answer — the TTS of one sentence runs while the LLM is still generating the next. This is the lowest-latency way to speak a generated answer (measured by the time_to_first_audio_ms diagnostic). Chaining a standalone Generate → Speak instead produces the same audio but cannot start until generation has fully finished, since Speak only sees the completed text. Prefer this fused node when you want a generated answer spoken with minimal delay, and the split form only when you need to branch on, inspect, or transform the text in between.

The voice is chosen by tts_model_name. Leave it empty to use the first TTS voice available in the server's model catalog, or set it to pick a specific voice.

NodeType: GenerateAndSpeak.

Parameters

Param Type Default Range Structural Description
model_name Optional[str] inherit model default — ✓ LLM model catalog name. Empty = use the agent's default_model_name.
context_size int 0 ≥ 0.0 ✓ KV-cache / context window (n_ctx) for the LLM in tokens. 0 = fall back to the model variant's context_size, else the server default_n_ctx. Validated against the model's trained maximum at agent creation.
tts_model_name Optional[str] inherit model default — ✓ TTS model catalog name. Optional — when empty the server uses the first available TTS voice in the model catalog. Set it to pick a specific voice.
system_prompt Optional[str] (multiline) inherit model default — — Prepended before the user turn during projection.
input Optional[str] inherit model default — ✓ Slot name this node consumes as its primary text. Empty = "user_message". Structural: immutable after creation — rebinding would re-wire the slot dataflow that is validated once at agent creation.
template Optional[str] (multiline) inherit model default — — Mustache template applied to the user-area message at projection time.
placement Placement Placement.BeforeUserAsSystem — — Where the rendered template body is placed relative to the resolved input.
output_name Optional[str] inherit model default — ✓ Name this node's output slot is stored under. Empty = the node's own name. Structural: immutable after creation — renaming would re-wire the slot dataflow that is validated once at agent creation.
history_role HistoryRole HistoryRole.Assistant — — Controls whether/how this node's slot replays in later turns' projected transcript. No send field — GenerateAndSpeak always streams text (streaming and non-streaming delivery would need incompatible projection-replay handling, so only the streamed mode is supported).
sampling Optional[Any] inherit model default — — Sparse sampling overrides applied on top of the model-catalog defaults.
grammar Optional[str] (multiline) inherit model default — — Optional GBNF grammar. When non-empty, the generated (and spoken) output is constrained to this grammar at every decode step. Empty = unconstrained. Must contain a root rule. Mutable — rebuilt per turn. Narrower fit than on Generate: use it to constrain spoken output to a fixed set of barks / canned lines. Validated at agent creation and on mutation (3007).
speaker_id int 0 ≥ 0.0 — TTS speaker / voice index (default 0). Valid range is 0 to the model's speaker count minus 1 — the shipped Supertonic 3 bundle has 10 voice styles (0–9); Pocket TTS is single-speaker, so it must stay 0 there (pick a Pocket voice with tts_voice instead). Agent creation fails with an out-of-range error for values the model cannot satisfy.
speed float 1.0 0.1 – 3.0 — TTS speech-rate multiplier. 1.0 = native speed.
tts_lang Optional[str] inherit model default — — TTS language hint for multilingual models that select the language per call (e.g. Supertonic 3: "en", "de", "fr", …). Empty = the model's catalog default. Ignored by families whose language is fixed by the model itself (e.g. Pocket TTS per-language models).
tts_voice Optional[str] inherit model default — — Reference-voice WAV for zero-shot voice-cloning families (Pocket TTS): a path relative to the session storage root pointing at a 10–20 s clean mono recording of the target voice. Empty = the model's default voice. Ignored by fixed-voice families (use speaker_id there).
min_sentence_chars int 12 ≥ 1.0 — Minimum characters before a sentence boundary fires. Lower values reduce first-utterance latency at the cost of more, smaller TTS calls.
substitute_agent_variables bool False — — When true, replace __VARIABLE__ markers in streamed LLM output with the rendered values of agent variables that declared allow_output_substitution. Matching is ASCII case-insensitive. Default false. Transformed text is authoritative for the slot, wire answer, history, and TTS input (substitution runs before synthesis). Appended after default_exit to keep FlatBuffers field ids additive.
output_filter Optional[Any] inherit model default — — Opt-in bounded artifact cleanup applied to streamed LLM output before variable substitution, slot/history write, wire delivery, and TTS. Missing/null = all flags off (defaults). Appended after substitute_agent_variables for additive field ids.

Exits

Each exit is a structural string field on the node's params; its value names the target node (empty = END).

Exit Param field Description
default default_exit Default exit target — routes here after generation + synthesis complete. Empty string = END.

Constrained output (GBNF grammar)

Like Generate, this node accepts an optional grammar (GBNF) that constrains the generated output at every decode step. The full rules there apply verbatim (must contain a root rule; fail-fast at agent creation and InvalidParamValue 3007 on a bad mutation; mutable per turn; constrain the wire, not the thinking).

The fit here is narrower than on Generate: the constrained output is also spoken, so grammar only makes sense when the constrained text is itself meant to be voiced — e.g. restricting an NPC to a fixed set of barks or canned lines:

root ::= "Halt!" | "Who goes there?" | "Move along." | "For the throne!"

Note the interaction with min_sentence_chars (default 12): very short constrained output may never cross the sentence-boundary threshold, so the whole (tiny) output flushes as a single TTS call — which is fine for barks.

See Concept: Constrained output (GBNF) for what a grammar does and doesn't guarantee, and How to constrain output with a grammar for a worked example.