How-to Guides

Task recipes. Each page solves one concrete problem and shows a minimal working example in Unity, Unreal, C++, and Python where applicable. If you are new to Tryll, start with First inference; if you want the why, see Concepts.

Guides marked ★ are the ones to read first in their area.

Setup and connection

• Run the Tryll Server — install or build from source, edit server-config.json, start, watch logs.
• Auto-launch the Server — let the client spawn and manage the server process for you (Unity, Unreal, C++, Python).
• Connect and Manage a Session — connect, create the session, reconnect, clean shutdown.

Editor tools

• Test an Agent in the Editor ★ — iterate on a workflow in the Unity/Unreal Chat window without entering Play/PIE.
• Edit Workflows in the Unity Graph Editor and in the Unreal Graph Editor — author a workflow visually: add nodes, wire exits, validate, save.
• Manage Models in the Editor — browse, download, load/unload, and register models for builds from the GUI.
• Choose Where Models Run — put each model in RAM or VRAM, and understand why placement is a hint rather than a guarantee.
• Use the Agent Log — the per-agent send/receive narrative in the editor: what your game asked for, and what came back.
• Inspect Server Memory — Unity and Unreal Memory windows: live RAM/VRAM for the whole server process (Refresh only).
• Inspect and Edit Knowledge Stores — Unity and Unreal Knowledge windows: list, query, and edit experimental mutable stores, save and load game memory, and export or import one store.
• Compare Dialog Variants in the Lab — run one scripted dialog across parameter variants and seeds, and read the spread instead of guessing from a single reply.
• Test Tool Calls in Chat and the Dialog Lab — register a handler on the scene agent, talk to it in Chat, then script canned or live-tool Lab runs.
• Run the Dialog Lab in CI — run a Lab script or suite from the command line and fail the build when an NPC gets worse.
• Agentic Coding With Tryll — have an AI coding agent build Tryll agents: AGENTS.md, the Unreal MCP tools, and how to write a prompt it can finish.

Build an agent

• Build a Chat Agent with a Graph ★ — compose a Generate + guardrail graph. Start here when writing your first agent.
• Design an NPC Prompt ★ — measurement-backed recipes for writing a role-play system_prompt: what actually improves character dialogue, what provably does not, and how model choice changes the answer.
• Design an Intent-Classifier Prompt — the ClassifyIntentLLM template contract ({{intents_block}} required; {{letters}} optional), how to write and order the labels, and how to tune threshold/margin.
• Use Mustache Templates — control exactly how instructions, slots, variables, and retrieved chunks land in the prompt.
• Stream Answers to a UI — wire streaming tokens into your application's view layer.
• Send Multiple Answers in One Turn — two speakers, or a thinking channel plus a reply, routed by AnswerText.node_name.

Dialogue and turn control

• Change Agent Parameters at Runtime — mutate a node's params (prompt, sampling, threshold, …) between turns, and which params are structural and cannot change.
• Seed and Edit Dialog History — append scripted user/assistant turns, write an opener, or retry by removing the last turn — all without running the graph.
• Export and Import Dialog — capture a full-fidelity snapshot and restore it from a game-owned save file.
• Save and Load Agent State — coordinate quiescent multi-agent snapshots, variables, game state, atomic slot writes, strict restore, and KV-cache warm-up.
• Cancel a Turn — stop an in-flight turn (a chat "Stop" button, or a rollback), keeping or discarding the partial reply.
• Pause and Resume a Turn — suspend a turn between nodes and resume it after acting client-side.
• Manage an Agent's KV Cache — prefill, evict, and inspect language-model KV caches to trade memory against first-turn latency.
• Run Many Agents on a Small GPU — size every LLM node's context window, then evict the agents nobody is talking to: the two levers that fit several NPC agents into limited VRAM.
• Keep Long Conversations Coherent — add a Summarize node so ageing turns fold into a rolling summary instead of being trimmed out of the prompt.

Knowledge and retrieval

• Create a Simple RAG Assistant ★ — the hero recipe: prepare a knowledge base, add a Retrieve node in front of Generate, see grounded answers.
• Query Rewriting and HyDE for RAG — rewrite the user's message before retrieval, while still answering what they actually asked. Covers both registers: a keyword query when your chunks read like questions, and a hypothetical answer (HyDE) when they read like statements.
• Build an Intent-Driven NPC — classify the player's intent against a labelled knowledge base and route to tone-specific replies.
• Classify Intent Without a Knowledge Base — the same NPC using ClassifyIntentLLM: describe each intent in a sentence, no embedding model and no corpus to author.
• Score Several Attributes of One Line — label one player line on several axes (Choice, Boolean, Score) in one batched ClassifyAttributesLLM call, then route on the result.
• Give an NPC a Memory It Can Search — store what the player says with WriteToMemory, find it again with Retrieve on later turns, and keep it across game saves (experimental).

Game state and control flow

The mental model for this section is the per-turn slot blackboard and the per-agent variable store.

• Drive Prompts and Retrieval from Game State ★ — inject live game state (level, mood, quests) into a prompt template and a Retrieve filter via agent variables, updated each turn with no ChangeParam round-trip.
• Branch on a Slot — conditional routing with the Branch node, including reacting to a verdict the model produced earlier in the same turn.
• Add a Hidden Reasoning or Draft Step — an internal draft or "thinking" pass the conversation never remembers, using send and history_role.
• Substitute Agent Variables in LLM Output — have the model emit __PRICE__-style markers and replace them with live variable values before the answer, history, or TTS sees the text.
• Filter LLM Output Artifacts — strip speaker prefixes and roleplay markup spans out of streamed output.
• Keep Merchant Prices Deterministic ★ — worked example: game code owns the number, the model owns the wording, and the two never disagree.

Structured output and tools

• Define and Handle Tool Calls ★ — declare tool definitions on a ToolCall node, route on tool_called / no_tool_called, handle the call client-side, and hand the real result back to the model.
• Test Tool Calls in Chat and the Dialog Lab — exercise that workflow in the editors without Play/PIE.
• Compare Tool-Call Models ★ — exact-accuracy on the Tryll-authored suites and the BFCL slice; pick a catalog model by the job (commands, log query, parallel calls), not by parameter count.
• Design a Tool-Call Prompt ★ — measurement-backed schema and prompt recipes: name "unfiltered", put it first in the enum, and do not add parameters the model cannot fill.
• Fix Tool-Call Text in Answers — the player sees <|tool_call>… instead of dialogue: find which node wrote it, and fix the prompt, the routing, or the tool node.
• Constrain Output with a Grammar — force a Generate node to emit exactly a fixed shape (command, choice, JSON) with a GBNF grammar; flip between command turn and free chat at runtime.

Guardrails and safety

• Use Canned Responses and Guardrails — short-circuit jailbreaks and off-topic prompts to scripted replies, before spending a model call.
• Build an Immersion Guard — gate out-of-character and out-of-world player lines before they reach the NPC, and drop the rejected turn from history.

Voice

• Add Voice Output to an Agent ★ — make an NPC speak with GenerateAndSpeak / Speak and a playback component.
• Add Lip Sync to a Character — move a character's mouth with its voice using the server's viseme track.
• Clone a Voice from an Audio Sample — point a voice profile at a short reference WAV to speak in that voice.
• Design a Voice Clip with Qwen3-TTS — make a reference clip from a text description, without an actor.
• Add Emotions to a Voice — one clip per mood as delivery modes, switched at runtime.
• Use Voice Input — create a VoiceInput handle, stream PCM audio, receive transcripts, push-to-talk vs. hands-free.
• Bias Voice Input with Hotwords — nudge the STT decoder toward game-specific proper nouns, spell names, and other lexicon items without retraining the model.
• Add Lip Sync to a MetaHuman (Unreal) — make a MetaHuman's mouth follow an agent's TTS voice with the separate Tryll MetaHuman Lip Sync plugin, using Epic's neural model or the viseme track.

Models and shipping

• Use Your Own Local Model — register a GGUF file that lives on disk without a Hugging Face download.
• Use a LoRA Adapter — run an LLM node with a fine-tuned GGUF adapter, and tune how strongly it applies.
• Pin and Unpin Models — keep a model warm (LoadModelRequest) or release it (UnloadModelRequest).
• Verify Required Models Before Play — gate Play/PIE on every registered model being downloaded first, and fail the build when a workflow references a model that will not ship.
• Estimate Memory Footprint — work out the RAM and VRAM a model line-up needs before you ship, so it fits your target hardware. Check the live numbers in Inspect Server Memory.
• Ship Storage Folders for Builds — lay out canned-response / guardrail / RAG / hotword files so they travel with a packaged build and resolve against the process storage root.
• Ship an Agent in Another Language — set one session language and have speech, speech recognition, content files and prompt directives all follow it; plus what the language setting deliberately does not do.
• Localize Prompts with Your Localization System — tick Localized on a prompt, store a Text ID, and let Unity or Unreal ask your string table for the text before the agent is created.

Downloading and registering models from the editor GUI is covered in Manage Models in the Editor.

Help and support

• Report a Bug — work out what kind of bug you hit, collect the logs, crash dump or Agent Log export it needs, and post it in the Discord Bug Hunt channel.

Related

• Concepts — the mental model behind each recipe.
• Reference — field-level detail for every knob.