Skip to content

Generate and Speak

The Generate-and-Speak node fuses LLM token generation with streaming TTS synthesis: it runs the same projection and language-model inference as a Generate node, but as tokens arrive it segments them into sentences, dispatches each sentence to a dedicated TTS strand, and emits PCM audio frames alongside the text deltas.

Because synthesis overlaps generation, audio begins after roughly the first sentence (min_sentence_chars) rather than after the whole answer — the TTS of one sentence runs while the LLM is still generating the next. This is the lowest-latency way to speak a generated answer (measured by the time_to_first_audio_ms diagnostic). Chaining a standalone Generate → Speak instead produces the same audio but cannot start until generation has fully finished, since Speak only sees the completed text. Prefer this fused node when you want a generated answer spoken with minimal delay, and the split form only when you need to branch on, inspect, or transform the text in between.

The voice is chosen by tts_model_name. Leave it empty to use the first TTS voice available in the server's model catalog, or set it to pick a specific voice.

NodeType: GenerateAndSpeak.

Leave context_size at 0 to inherit the model variant's window, then the server default_n_ctx (commonly 8192) — the same resolution as Generate. The context table is an advanced structural override. Do not set q4_0 KV expecting a VRAM win at a right-sized window.

Parameters

Param Type Default Range Structural Description
model_name Optional[str] inherit model default — ✓ LLM model catalog name. Empty = use the agent's default_model_name.
context_size int 0 ≥ 0.0 ✓ KV-cache / context window (n_ctx) for the LLM in tokens. 0 = fall back to the model variant's context_size, else the server default_n_ctx. Validated against the model's trained maximum at agent creation.
tts_model_name Optional[str] inherit model default — ✓ TTS model catalog name. Optional — when empty the server uses the first available TTS voice in the model catalog. Set it to pick a specific voice.
system_prompt Optional[str] (multiline) inherit model default — — Prepended before the user turn during projection.
input Optional[str] inherit model default — ✓ Slot name this node consumes as its primary text. Empty = "user_message". Structural: immutable after creation — rebinding would re-wire the slot dataflow that is validated once at agent creation.
template Optional[str] (multiline) inherit model default — — Mustache template applied to the user-area message at projection time.
placement Placement Placement.BeforeUserAsSystem — — Where the rendered template body is placed relative to the resolved input.
output_name Optional[str] inherit model default — ✓ Name this node's output slot is stored under. Empty = the node's own name. Structural: immutable after creation — renaming would re-wire the slot dataflow that is validated once at agent creation.
history_role HistoryRole HistoryRole.Assistant — — Controls whether/how this node's slot replays in later turns' projected transcript. No send field — GenerateAndSpeak always streams text (streaming and non-streaming delivery would need incompatible projection-replay handling, so only the streamed mode is supported).
sampling Optional[Any] inherit model default — — Sparse sampling overrides applied on top of the model-catalog defaults.
grammar Optional[str] (multiline) inherit model default — — Optional GBNF grammar. When non-empty, the generated (and spoken) output is constrained to this grammar at every decode step. Empty = unconstrained. Must contain a root rule. Mutable — rebuilt per turn. Narrower fit than on Generate: use it to constrain spoken output to a fixed set of barks / canned lines. Validated at agent creation and on mutation (3007).
speaker_id int 0 ≥ 0.0 — TTS speaker / voice index (default 0). Valid range is 0 to the model's speaker count minus 1 — the shipped Supertonic 3 bundle has 10 voice styles (0–9); Pocket TTS is single-speaker, so it must stay 0 there (pick a Pocket voice with tts_voice instead). Agent creation fails with an out-of-range error for values the model cannot satisfy.
speed float 1.0 0.1 – 3.0 — TTS speech-rate multiplier. 1.0 = native speed.
tts_lang Optional[str] session locale -> model default — — TTS language hint for multilingual models that select the language per call (e.g. Supertonic 3: "en", "de", "fr", …). Resolution order: this param, then the session locale (CreateSessionRequest.locale), then the model's catalog default. Set it only to override the session — one character who keeps speaking French in a German playthrough. For a whole-game language switch, set the session locale instead and leave this empty: it reaches every agent and every node with no per-node mutation. Ignored by families whose language is fixed by the model itself (e.g. Pocket TTS per-language models).
tts_voice Optional[str] inherit model default — — Reference-voice WAV for zero-shot voice-cloning families (Pocket TTS): a path relative to the session storage root pointing at a 10–20 s clean mono recording of the target voice. Empty = the model's default voice. Ignored by fixed-voice families (use speaker_id there).
min_sentence_chars int 12 ≥ 1.0 — Minimum characters before a sentence boundary fires. Lower values reduce first-utterance latency at the cost of more, smaller TTS calls.
substitute_agent_variables bool False — — When true, replace __VARIABLE__ markers in streamed LLM output with the rendered values of agent variables that declared allow_output_substitution. Matching is ASCII case-insensitive. Default false. Transformed text is authoritative for the slot, wire answer, history, and TTS input (substitution runs before synthesis). Appended after default_exit to keep FlatBuffers field ids additive.
output_filter Optional[Any] inherit model default — — Opt-in bounded artifact cleanup applied to streamed LLM output before variable substitution, slot/history write, wire delivery, and TTS. Missing/null = all flags off (defaults). Appended after substitute_agent_variables for additive field ids.
tts_voice_description Optional[str] (multiline) inherit model default — — Natural-language stable voice identity: age, accent, pitch, timbre and baseline pacing. Empty = use speaker_id / catalog default. Appended after output_filter so existing FlatBuffers field IDs do not move.
tts_delivery_instruction Optional[str] (multiline) inherit model default — — Natural-language sustained delivery direction for this synthesis call. Empty = neutral / model default.
context Optional[Any] inherit model default — ✓ Backend context construction. Null table inherits, except CreateContext still defaults n_outputs_max to 1. Structural. Appended after tts_delivery_instruction so existing field IDs do not move.

Exits

Each exit is a structural string field on the node's params; its value names the target node (empty = END).

Exit Param field Description
default default_exit Default exit target — routes here after generation + synthesis complete. Empty string = END.

Constrained output (GBNF grammar)

Like Generate, this node accepts an optional grammar (GBNF) that constrains the generated output at every decode step. The full rules there apply verbatim (must contain a root rule; fail-fast at agent creation and InvalidParamValue 3007 on a bad mutation; mutable per turn; constrain the wire, not the thinking).

The fit here is narrower than on Generate: the constrained output is also spoken, so grammar only makes sense when the constrained text is itself meant to be voiced — e.g. restricting an NPC to a fixed set of barks or canned lines:

root ::= "Halt!" | "Who goes there?" | "Move along." | "For the throne!"

Note the interaction with min_sentence_chars (default 12): very short constrained output may never cross the sentence-boundary threshold, so the whole (tiny) output flushes as a single TTS call — which is fine for barks.

See Concept: Constrained output (GBNF) for what a grammar does and doesn't guarantee, and How to constrain output with a grammar for a worked example.

Diagnostics

When enable_diagnostics = true, this node contributes to debug_info.nodes[].diagnostics. See Turn Diagnostics JSON for the envelope, the shared input.prompt[] shape, and the engine counters.

This is the only node that emits a fifth section, tts — the voice half of the fused step is kept separate from the language-model parameters.

Key Meaning Absent when
parameters.model_name The language model that actually ran, after fallback. The name could not be resolved.
parameters.temperature, top_p, top_k, max_tokens The resolved sampling set. Unlike Generate, the penalty and seed keys are not recorded. —
parameters.history_role How the answer replays into later turns. —
parameters.substitute_agent_variables true. Output substitution is off.
parameters.output_filter.strip_speaker_prefix, parameters.output_filter.strip_asterisk_spans, parameters.output_filter.strip_paren_spans, parameters.output_filter.strip_bracket_spans true, one key per enabled rule. That rule is off.
parameters.output_filter.max_prefix_bytes, parameters.output_filter.max_span_bytes The scan limits the filter ran with. No filter rule is enabled.
parameters.input The configured input slot name. Using the user_message default.
parameters.output_name The configured output slot name. Defaulting to the node name.
tts.speaker_id, speed, tts_lang, tts_voice The voice settings the synthesis ran with. —
tts.tts_voice_description The voice description prompt. Not set.
tts.tts_delivery_instruction The delivery instruction. Not set.
tts.sample_rate The TTS model's native sample rate, in Hz. —
tts.min_sentence_chars Sentence-boundary threshold before a chunk is synthesized. —
tts.time_to_first_audio_ms Delay before the first audio chunk was produced — the voice counterpart of time_to_first_token_ms. No audio was produced.
input.prompt[] The rendered prompt as the model received it. —
output.text The text this node produced (and spoke). The node produced no output slot.
engine.* Grammar state and scheduler counters. Server config include_engine_diagnostics is off.

The node-level variable_replacements[] and output_filter_removals[] arrays behave exactly as on Generate, and matter more here: what the filter removed is also what the voice did not say.