Skip to content

v0.5.0

This release makes an agent's turn composable and game-driven: a per-turn slot blackboard lets any node's output feed another node or template later in the same turn, agent variables let game code push live state (level, mood, inventory) straight into prompts, retrieval filters, and generated text, and tool results can now flow back into the model mid-turn instead of being faked with an acknowledgement. It also adds GBNF grammar constraints for output your game code has to parse, dialog-history editing for scripted openers and rollbacks, explicit KV-cache control, and a visual workflow graph editor for Unreal.

Several node params were replaced, a node was renamed, and the wire protocol changed — existing integrations must be updated and rebuilt. See Breaking changes and behavior changes before upgrading.

Highlights

  • Agent variables — a typed per-agent key→value store the game writes over the wire and the server reads from prompt templates ({{var.<name>}}), retrieval filters, and (opt-in) markers in generated text. See Agent Variables.
  • Slots and inter-node value passing — every producing node writes into a named slot any later node can consume via input or {{slot.<name>}}, plus a model-free Transform node for composing text. See Slots and inter-node value passing.
  • Model-visible tool results — pause on a tool call, run it client-side, and hand the real payload back so a downstream Generate answers from it. See Pause and resume a turn.
  • Constrained output (GBNF) — give Generate a grammar and every reply is guaranteed to be one of your legal strings: no parse failures, no preamble to strip. See Constrain output with a grammar.
  • Seed and edit dialog history — append scripted user/assistant turns or tail-remove whole interactions without running the graph. See Seed and edit dialog history.
  • KV-cache control — prefill an agent's caches at creation or on demand, evict them to reclaim memory, and query residency. See Manage an agent's KV cache.
  • Unreal workflow graph editor — author workflows visually in Unreal, as Unity has since v0.3.0. See Edit workflows in the Unreal graph editor.

Agent variables

  • Declare once, mutate freely. CreateAgent gains a variables field — the agent's complete, type-locked declaration (int64 / double / string / bool / set<string>), each with an initial value. Writes to an undeclared name are rejected. See Agent Parameters.
  • Read from three places, server-side. Mustache templates via {{var.<name>}} (see Use Mustache templates), the Retrieve / ClassifyIntent filter grammar via a {"var": "<name>"} operand (see Retrieve filter grammar), and — opt-in — __NAME__ markers in generated text. All three re-read the current value on every use, so a mutation lands on the very next turn with no ChangeParam round-trip and no recompile.
  • A typed mirror on every client. Variables() (C++) / .variables (Python) / .Variables (Unity) / Get Variables (Unreal Blueprint) offer SetInt / SetFloat / SetString / SetBool / SetStringSet / AddToSet / RemoveFromSet / Reset and matching getters. Writes validate locally against the declaration — an unknown name or wrong type fails immediately with no round-trip — update the local mirror synchronously, and are batched into one update flushed just before the next send, resume, or param change.
  • Output substitution. With allow_output_substitution on the declaration and substitute_agent_variables on a Generate / GenerateAndSpeak node, __PRICE__-style markers in the model's streamed output are replaced with the live variable value before the slot, the wire answer, history, and TTS see the text — so game code owns the number while the model writes the sentence. See Substitute agent variables in LLM output and Keep merchant prices deterministic.
  • Editor authoring. Declare variables on the Unity TryllWorkflowAsset / Unreal workflow asset, then override initial values per instance on the agent component (a Material-Instance-style Inspector row per declaration). Override edits update a live Play-mode or editor-Chat agent immediately, with a Flush Now button for eager sends.
  • Guide: Drive prompts and retrieval from game state.

Slots and inter-node value passing

  • Slots replace fixed message roles. Every producing node (Generate, GenerateAndSpeak, CannedResponse, Transform, Instruction / IntentToInstruction) writes its output into a named slot on the turn's shared blackboard. Any later node in the same turn can read it via input; any template can reference it via {{slot.<name>}}. See Slots and inter-node value passing.
  • New Transform node. A model-free node that renders a Mustache template into its own slot — no inference, no wire emission. The flagship use is query rewriting for RAG: fold the conversation into a standalone search query before Retrieve runs. See Query rewriting for RAG.
  • input selector. Generate, GenerateAndSpeak, ToolCall, Retrieve, ClassifyIntent, ClassifyIntentLLM, RegexGuardrail, and Speak gain an input param naming the slot to consume (empty = the reserved user_message slot), replacing the old Placement.InPlaceOfUser and "voice the latest character message" behaviors.
  • output_name override. Producing nodes can name their slot explicitly instead of using the node's own name, so a graph can be rewired without renaming nodes.
  • send replaces stream. A three-way SendAnswer enum (None, Whole, Streamed) on generation / canned-response nodes controls wire delivery; None lets a node compute a value used only internally this turn (e.g. a query rewrite) without ever emitting AnswerText.
  • history_role controls cross-turn replay. A HistoryRole enum (None, Assistant) on the same nodes controls whether a slot's text is replayed as history on later turns, independent of whether it was sent on the wire this turn. Together with send, this is how you add a hidden draft or reasoning pass — see Add a hidden reasoning or draft step.
  • Multi-sender AnswerText.node_name. Every AnswerText frame now carries the name of the node that produced it, so a client can attribute chunks when a turn routes through more than one text-producing node — two NPCs speaking, or a thinking channel plus a reply. Each producing node emits its own final chunk. See Send multiple answers in one turn.
  • Slot-aware editor authoring. The Unity and Unreal graph editors render input as a dropdown of only the slots reachable upstream of that node (no free text), and renaming a node or its output_name override now automatically rewrites every exit, input selector, and {{slot.<name>}} template reference to the old name in one step — see Slots and inter-node value passing.

Tool calling

  • One disposition param replaces the notify/pause/history trio. A single seven-value ToolCallDisposition enum (RouteOnly, Acknowledge (default), Notify, NotifyAndAcknowledge, Pause, PauseAndAcknowledge, AwaitResult) now controls whether a ToolCall node fires a tool_call event, pauses the turn, and how the call projects into later-turn history. The full matrix is in the ToolCall reference; the history contract is in Tool-call history: complete pairs, never dangling.
  • Model-visible tool results (AwaitResult). ResumeAgentRequest gains tool_results: [ToolResult {call_id, result}]. An AwaitResult pause is resumed with a complete, atomically-applied batch — one entry per pending call_id, taken from the tool_call event — so a downstream Generate can condition on the real payload instead of a synthetic acknowledgement. Invalid batches (unknown, duplicate, or missing call_id; a batch on a non-AwaitResult pause; or over the 64-entry / 128-byte-call_id / 256 KiB-result / 512 KiB-aggregate limits) return the new 3016 InvalidToolResults and leave the turn paused for retry. See Pause and resume a turn.
  • acknowledge_text. The acknowledgement the server writes into history for the Acknowledge dispositions is now configurable per node (empty = "ok").
  • New client APIs for the result-bearing resume. C++: ResumeWithToolResult(s) / *Async, SetOnToolCallWithId, and the preferred owning-payload SetOnToolCallEvent (safe to capture beyond the callback). Python: resume(..., tool_results=) plus the non-blocking resume_async(...), which is required from a reader-thread tool-call or paused callback — the blocking resume() / change_params() deadlock the sole reader thread if called from there. Unity / Unreal: ResumeWithToolResult(s), plus RegisterTool auto-resume sugar that now retains the paused batch (and any already-succeeded handler results) across a failed attempt — see HasPausedToolBatch / RetryPausedToolBatch.
  • Pick a tool-capable model before downloading it. models.json variants now declare tool_call_support ("supported" / "unsupported"), surfaced by ListModels and in the Unity / Unreal Model Manager detail pane. It is advisory — whether a ToolCall node accepts a model is still decided at agent creation from the model's own chat template. See Model Management.

Constrained output (GBNF)

  • grammar on Generate and GenerateAndSpeak. Supply a GBNF grammar and output is constrained at every decode step — malformed output becomes unsampleable. Ideal for closed-set command parsing, dialogue-choice enums, emotion tags, or anything consumed by game code rather than read by a human. See Generate and the concept page Constrained output (GBNF).
  • Fail fast, flip at runtime. A grammar must contain a root rule; an invalid one is rejected at agent creation, and an invalid mutation is rejected with 3007 InvalidParamValue while the previous grammar is kept. grammar is mutable and rebuilt per turn, so a client can flip a node between a strict "command turn" and free chat with a param change. See Constrain output with a grammar.

Dialog history and turn control

  • Append or trim history without running a turn. Two new operations — AppendInteractions (batches of {user_message, assistant_message}; either side may be empty) and RemoveInteractionsFromEnd (saturating whole-interaction tail removal) — let you seed a scripted opener, restore a save game, or roll back the last exchange. Both return applied/removed counts, emit no TurnComplete, and require a strictly idle agent (rejected with 3004 AgentBusy while a turn is running or paused, which is stricter than ChangeAgentParam). Exposed on every client (AppendInteractions / RemoveInteractionsFromEnd; Python append_interactions / remove_interactions_from_end). See Seed and edit dialog history.
  • Stateless agents can be seeded. With maintain_dialogue_history = false, appended history acts as a one-shot seed for a single turn and clears when that turn completes.
  • Cancelling a turn now has a guide. Cancel a turn covers the StopAndKeep / StopAndDiscard modes on every client, and how a discard differs from an explicit tail removal.

KV-cache control

  • Choose how caches are prepared at creation. CreateAgent gains kv_cache_initialization: AllocateOnly (default), Prefill (allocate and decode the reusable prefix up front, so the first turn skips that cost), or DeferAllocation (defer until the first send). See Agent Parameters.
  • Prefill, evict, and inspect on demand. Three new operations report aggregate residency (Evicted / Resident), reusable-prefix status (NotCurrent / Current), eligible node count, and — for prefill — contexts created plus tokens decoded and reused. Exposed as PrefillKvCache / EvictKvCache / GetKvCacheStatus (C++ and Unity, with async variants), prefill_kv_cache() / evict_kv_cache() / get_kv_cache_status() (Python), and Prefill KV Cache / Evict KV Cache / Get KV Cache Status with matching result events (Unreal Blueprint).
  • Eviction is safe. Evicted eligible contexts are restored automatically before the next send — there is no restore call to make. Cache operations are idle-only; a conflict with a running or paused turn, or another cache operation, returns 3004 AgentBusy. Only language-model contexts are affected. See Manage an agent's KV cache.

Editor tooling

  • Unreal workflow graph editor. Unreal now ships its own visual workflow graph editor with an asset factory, matching the Unity window: add nodes, wire exits, edit params, validate. See Edit workflows in the Unreal graph editor.
  • Unreal Chat window. The editor Chat panel is a dockable tab that targets a scene actor's agent component, creating its own preview agent from that component's resolved graph, variable declarations, and overrides — so you can iterate on a workflow without entering PIE. Variable-value edits stay live; structural changes need a restart. See Test an agent in the editor.
  • Variable overrides Inspector. Both editors draw one optional override row per declared variable — checkbox, read-only name, typed value — rather than a free-form list, and warn about stale entries left behind by a renamed or removed declaration.

Models, memory, and server configuration

  • Pinning and sweeping now cover every model kind. LoadModel / UnloadModel retention applies to language, embedding, STT, and TTS caches alike, and unused on-demand models are swept across all four caches immediately before the next load — not just after DestroyAgent — so idle models stop lingering. VAD is catalog/download-only: LoadModel for a VAD entry returns 6005 ModelResolutionFailed. See Pin and unpin models and Model Management.
  • models.json reference filled in. The catalog reference now documents model_type, audience, hidden, default_sampling, the required tool_call_support, the llama.cpp knobs (disable_thinking, use_jinja, reasoning_budget, chat_template_file), and the STT / TTS variant fields (stt_family, tts_family, tts_files, tts_lang, device_preference, num_threads).
  • New server-config fields. include_engine_diagnostics (per-node engine metrics in debug_info; never prompt or generated text), default_vad_model, stt_debug_dump_dir, idle_shutdown_timeout (a managed server self-exits after this many idle seconds), plus documented crash_dump, monitor, and debug_commands groups. See Server Configuration.
  • Managed-server shutdown documented. Stopping a managed server closes the client side and lets the server self-exit once idle (default 60 s) rather than hard-killing it, and the client's port setting is passed on the command line, so it overrides server-config.json. See Auto-launch the server.

New and expanded documentation

Breaking changes and behavior changes

  • HumanMessageGuardrail is renamed RegexGuardrail. The node type, params type, and builder method are renamed on every client (AddRegexGuardrail / RegexGuardrailParams / TryllRegexGuardrailParams / UTryllRegexGuardrailParams), and the node gains an input param so it can guardrail any slot, not just the user's message. See Regex Guardrail.
  • Fixed message components are removed. HumanMessageComponent, CharacterMessageComponent, and InstructionComponent are replaced by a single SlotComponent {name, text, kind, history_role, producer}, and TurnDiagnostics.human_message / character_message are gone; slot text appears under debug_info.interaction.components[] as SlotComponent entries. Update any tooling that parses debug_info JSON.
  • GenerateParams.stream / CannedResponseParams.stream (bool) are removed — replaced by send: SendAnswer. stream=True → send=SendAnswer.Streamed; stream=False → send=SendAnswer.Whole (the old stream=False path still emitted one whole-text AnswerText frame — it just skipped per-token deltas; it never meant "don't send"). Use send=SendAnswer.None_ only for a node whose text should never reach the wire at all (e.g. an internal query-rewrite step), which has no equivalent in the old boolean. GenerateAndSpeak has no send field — it always streams.
  • Placement.InPlaceOfUser is removed, and the Unity enum is renamed TryllKnowledgePlacement → TryllPlacement. Compose the replacement text with a Transform node and have the downstream node consume it via input instead. See Query rewriting for RAG for the migration pattern.
  • Speak's implicit input changed. It no longer voices "the latest character message" — it voices its input-resolved slot, and an empty input defaults to user_message (the same default as every other input-bearing node). A CannedResponse → Speak graph must therefore set Speak's input to the upstream canned node's slot; leaving it empty voices the user's message instead. Graphs relying on Placement.InPlaceOfUser upstream need the Transform migration above.
  • Instruction lookup tag renamed. {{instruction_<name>}} is removed; use {{slot.<name>}}. {{#instructions}} is unchanged.
  • ToolCall's notify_client, pause_after_tool_call, and history_policy are removed — set disposition: ToolCallDisposition instead (notify_client=true → Notify or NotifyAndAcknowledge; pause_after_tool_call=true → Pause, PauseAndAcknowledge, or AwaitResult). The tool_call event now carries a disposition value in place of history_policy. Rewrite any graph or config that set the old fields — see Tool Calling.
  • on_answer_text / OnAnswerText callback signature changed on every client — it gains a leading node_name parameter (Python: Callable[[str, str, bool, bool], None]; C++ / Unity / Unreal equivalents updated to match). Update callback signatures when upgrading the client libraries.
  • AnswerText.is_final is now per node, not per turn. A turn that routes through more than one text-producing node emits one final chunk per producing node. A client that treats the first is_final as "the turn is over" must key off node_name, or wait for TurnComplete.
  • tool_call_support is required in models.json. Every language-model variant must declare "supported" or "unsupported"; the server rejects a catalog that omits it, and the field is not allowed on other model types. Add it to any custom catalog before upgrading.
  • Breaking wire-protocol change. Rebuild your integration against the new schema; a client built against the v0.4.0 schema is rejected at connection time with 5004 ProtocolVersionMismatch.

New error codes

  • 3013 UnknownVariable — a variable update targeted a name not declared in CreateAgent.variables.
  • 3014 VariableTypeMismatch — an update's value type did not match the variable's declared type.
  • 3015 InvalidVariableValue — an invalid variable declaration (bad name, duplicate, or over a size limit), an unknown operation, or a set that would exceed its 4096-element cap.
  • 3016 InvalidToolResults — tool_results on a resume is not a complete, unique batch for the agent's currently paused AwaitResult calls. The turn stays paused for retry.
  • 4004 KvCacheOperationFailed — an explicit prefill, or agent creation with kv_cache_initialization = Prefill, failed while allocating, formatting, or decoding the reusable prefix.
  • 4005 KvCacheManagementUnsupported — the selected inference backend does not support the requested KV-cache operation.

See Error Codes for the full list.