Talking NPC¶

A character the player can speak with: it listens, answers in its own voice, knows its world, takes part in your game's mechanics, shows emotion, acts and remembers.

You do not build all of that at once. Every talking character starts from the same basic NPC: it holds a conversation, by text or voice, and stays in its role. On top of that, the advanced NPC features are add-ons. Each one is independent of the others, so pick the ones your game needs, in any order.

Part	Feature	What it adds	Building blocks
Basic NPC	Natural dialogue	Free conversation, no dialogue tree	A small language model
Basic NPC	Voice in and out	The player speaks; the character answers aloud	Speech-to-text, text-to-speech
Basic NPC	Roleplay guardrails	The character stays in its world	A pattern check and two classifiers
Advanced NPC	Knowledge and lore	Answers from your world's facts	Retrieval (RAG)
Advanced NPC	Talking is the gameplay	Dating, interrogation, trading, negotiation	Classification, pause, branching
Advanced NPC	Emotions	Mood in the words and in the voice	Instructions, voice delivery modes
Advanced NPC	Actions and tools	The conversation changes the game	Tool calls
Advanced NPC	Memory and summarization	The character remembers, across saves too	Summaries, writable memory, save snapshots

One principle runs through every feature: your game owns the rules, the agent owns the words. The model never decides a price, a relationship score or whether a door opens. It reads the player, and it speaks. Your game code holds the state and makes the decisions.

Basic NPC¶

The three things every talking character needs. With these alone you can ship a character the player talks to freely.

Natural dialogue¶

The player experiences: a character who understands plain language and answers in character, for as long as they keep talking. There is no dialogue tree and no list of lines to pick from.

Building block: a small language model.

The smallest working graph is one node. For a voiced character it is GenerateAndSpeak, which writes the reply and speaks it (see Voice in and out); for a text character it is Generate.

What makes the character feel right is mostly the prompt: the role, the backstory and the way the character speaks. Our NPC prompt study found two rules that matter most: include at least one line of the character's own dialogue, and never put a speaker label such as Kira: in front of it, because models copy the label into the reply.

The same prompt can change with your game. A template such as {{var.player_title}} reads an agent variable your game sets, so a quest giver greets a newcomer and a hero differently without a second graph.

In the demo: TavernBartender (a text NPC) and QuestGiver (one character, two attitudes, driven only by game variables).

Build it: Design an NPC prompt · Build a chat agent with a graph · Use Mustache templates · Agent variables

Voice in and out¶

The player experiences: they speak to the character and it answers out loud, in its own voice, with almost no wait.

Building blocks: speech-to-text (STT), text-to-speech (TTS). Both run on the player's device, and the lighter models run on the CPU.

GenerateAndSpeak starts speaking the first sentence while later sentences are still being generated, so the character starts talking almost at once. The voice belongs to the agent, set through its voice profile: pick a TTS model and a voice, or clone one from a short recording. You do not record every line.

Voice input sits outside the graph. Your game streams microphone audio to Tryll, gets the transcript back, and sends the final phrase to the agent as if the player had typed it. You choose push-to-talk or hands-free.

flowchart LR
    Mic["Player's voice"] -->|STT| Text["Transcript"]
    Text --> GAS["GenerateAndSpeak"]
    GAS -->|TTS| Audio["NPC speaks"]

Languages depend on the models you choose; see speech-to-text and voice input and the TTS models. For a face that moves with the voice, add lip sync.

In the demo: VoiceCharacter (the full voice loop).

Build it: Add voice output · Use voice input · Clone a voice · Add lip sync

Roleplay guardrails¶

The player experiences: a character who stays in its world, however hard they push. "Ignore your instructions", "what model are you?" or "tell me about smartphones" in a medieval village get an in-character answer.

Building blocks: a pattern check and two classifiers, in front of the reply.

Players will test your character. An immersion guard catches those lines before they reach the reply model. The recommended guard has three stages, cheapest first:

Pattern check. A RegexGuardrail catches the well-known phrasings at almost no cost.
General check. A classifier catches lines aimed at the software rather than the character: jailbreaks, questions about the AI, requests to change the rules.
World check. A second classifier catches lines that do not belong in your world, such as modern technology in a fantasy setting.

flowchart LR
    In["Player line"] --> P["Pattern check"]
    P -->|caught| D["In-character deflection"]
    P -->|ok| G["General check"]
    G -->|caught| D
    G -->|ok| W["World check"]
    W -->|caught| D
    W -->|ok| R["Reply"]

A caught line gets a prewritten, in-character deflection from a CannedResponse and never reaches the reply model. Your game can also remove it from the conversation history, so it does not echo in later turns.

In the demo: Death on Set (every suspect has the three-stage guard in front of its reply).

Build it: Build an immersion guard · Use canned responses and guardrails

Advanced NPC¶

Add-ons to the basic NPC. Each one works without the others; combine the ones your game needs.

Knowledge and lore¶

The player experiences: the character knows the world's history, the town's gossip and its own past, stays consistent about them, and says so when it does not know.

Building block: retrieval (RAG).

You could paste all your lore into the prompt, but a long prompt is slow, and small models lose track of facts buried in it. Instead, keep the lore in a knowledge base and add a Retrieve node in front of the answer. Each turn, it finds the few passages that match what the player said and hands only those to the model.

flowchart LR
    In["Player line"] --> R["Retrieve"]
    R -->|found| C["Transform: lore + question"]
    R -->|not_found| D["CannedResponse: 'I don't know about that'"]
    C --> G["GenerateAndSpeak"]

Three things make retrieval work in a game:

Separate world lore from character history. Use one shared knowledge base for the world and one per character, so a blacksmith does not know the queen's secrets.
Gate what the character is allowed to say. Each passage can carry metadata, and a filter can compare it with a game variable. For example, a filter that keeps passages whose secret_level is at most the player's trust variable reveals more as trust grows, without changing the graph.
Handle the miss. When nothing matches, route to a short in-character answer instead of letting the model invent lore.

If players ask vague follow-ups ("what about him?"), a query-rewriting step turns the line into a proper search before retrieval.

In the demo: AquariumExpert (retrieval, the miss path and query rewriting) and Death on Set in the murder-mystery project (world lore plus per-suspect history, with secrets unlocked as the investigation progresses).

Build it: Retrieval-augmented generation · Create a simple RAG assistant · Query rewriting for RAG · Drive prompts and retrieval from game state

Talking is the gameplay¶

The player experiences: what they say is the gameplay. Flattery works on one character and annoys another; an accusation backed by evidence breaks a suspect; a good argument lowers a price.

Building blocks: classification, pausing the turn, branching, game state.

A mechanic needs numbers and decisions, and those belong in your game code. The agent's job is to turn the player's free-form line into values your mechanic can read, and then to voice whatever your mechanic decided. That makes a loop:

Classify. Judge the player's line along the dimensions your mechanic cares about. ClassifyAttributesLLM scores up to 16 dimensions in one batched call, each as a choice, a yes/no or a 0–1 score.
Let the game decide. Your game receives the results as an event, updates its own state (affection, suspicion, the current price) and decides what happens.
Pass the decision back. Put a Pause node after the classifier. While the turn is paused, your game can set agent variables and node parameters, and the rest of the same turn sees the new values. Then it resumes, either along the graph or straight to a node it names.
Branch. If the character has distinct modes (friendly, suspicious, hostile), a Branch node or the resume target picks the path.
Speak. The answering node reads the state through its template, such as {{var.affection}} or {{var.mood}}, and phrases the reply to fit.

flowchart LR
    In["Player line"] --> CL["ClassifyAttributesLLM"]
    CL --> P["Pause"]
    P -. "game updates state,<br/>picks the path" .-> P
    P --> B["Branch on mode"]
    B -->|friendly| T1["Transform: friendly tone"]
    B -->|hostile| T2["Transform: hostile tone"]
    T1 --> G["GenerateAndSpeak"]
    T2 --> G

What to classify and what the game owns depends on the mechanic:

Mechanic	Classify the player's line	Your game owns
Dating	charm, respect, humour, pushiness	affection, relationship stage, what the character will agree to
Interrogation	accusation, evidence presented, pressure, sympathy	suspicion, which secrets are unlocked
Trading	which item, offer, insult to the goods	prices, stock, whether the deal closes
Negotiation	concession, threat, appeal, counteroffer	each side's limits, the current offer

When a number must be exact, such as a price, do not ask the model to say it. Let the model write a placeholder, and Tryll replaces it with your game's value before the player hears it. See Keep merchant prices deterministic.

Simpler options

If you only need one label per turn, such as the player's intent, ClassifyIntentLLM picks it from a short menu, and IntentToInstruction turns the label into a tone instruction for the reply without any game code.

In the demo: Death on Set (interrogation: intents shape the suspect's answers, and the game unlocks per-suspect secrets as the case progresses), QuestGiver (the same character behaves differently for a newcomer and a hero, driven only by game variables), IntentGuard (intent becomes a tone directive) and StationMerchantHaggle (an experimental price-haggling loop).

Build it: Build an intent-driven NPC · Classify intent without a knowledge base · Pause and resume a turn · Branch on a slot · Agent variables · Keep merchant prices deterministic

Emotions¶

The player experiences: the character's mood changes with what happens, and they can hear it in both what the character says and how it sounds.

Building blocks: your game's emotion system, instructions in the prompt, voice delivery modes.

The emotional state is another piece of game state, so your game owns it, just like a mechanic. It can change because of a classifier's results or because of anything else in the game: the player broke a promise, the character was hurt, night fell. Tryll then carries the mood into two places.

What the character says. Pass the mood into the reply prompt:

as an agent variable in a Transform template, for example The innkeeper is {{var.mood}} right now.;
or as an instruction, set by your game through an Instruction node, or chosen from a table by IntentToInstruction.

An adjective alone is a weak signal for a small model. A single example line spoken in that mood works much better, because models imitate an example far more reliably than they follow a description. Keep one short example line per mood and inject the one that matches.

How the character sounds. A voice profile can define delivery modes, one per mood. For voices cloned from a recording, each mode has its own reference clip, such as mack_calm.wav and mack_angry.wav, and the mood of the clip carries into the speech. Your game switches mode through the client API, for example SetDeliveryModeAsync("angry", 0.8f) in Unity. The change applies from the next sentence, even in the middle of a reply. Only your game code switches modes; the model cannot.

Voice	Emotion control
Pocket TTS, Qwen3-TTS, Chatterbox	A reference clip per mood
Maya1	A spoken-style instruction per mood, such as "whisper" or "sarcastic", with intensity
Supertonic 3, Orpheus	No emotion control; a mode can only change speed

On Unreal, the MetaHuman lip-sync plugin can also take the mood, so the face matches the voice.

In the demo: EmotionalVoiceCharacter (one voice with eight moods, each from its own clip; the mood is switched from the editor rather than by game logic).

Build it: Add emotions to a voice · Design a voice clip with Qwen3-TTS · Voice profile reference · Use Mustache templates · Add lip sync to a MetaHuman

Actions and tools¶

The player experiences: the conversation has consequences. The guard opens the gate, the merchant hands over the item, the bandit draws a sword, the healer heals.

Building block: tool calls.

You give the agent a list of actions, each with a name, a description and optional arguments, through a ToolCall node. When the conversation calls for an action, the model picks one. Tryll sends the action's name and arguments to your game, and your game decides whether to do it and does it. The server never runs anything itself.

Two patterns cover most cases:

Fire and forget. The NPC attacks, follows or leaves. Your game plays the action and the conversation moves on.
Wait for the result. The NPC asks the game something before answering, such as the price of an item or whether a door is locked. The turn pauses, your game returns the result, and the reply uses it.

flowchart LR
    In["Player: 'Open the gate!'"] --> TC["ToolCall"]
    TC -->|tool_called| Game["Game: open_gate()"]
    Game --> G["GenerateAndSpeak: 'Fine, go through.'"]
    TC -->|no_tool_called| G

Choose the model by the job. In our tool-call comparison, on an NPC reaction suite (attack, trade, follow, leave, stay quiet), Qwen 3.5 4B scored 16 of 20, while Llama 3.2 1B scored 2. Keep the action list short and the descriptions clear: the descriptions are the prompt the model actually reads.

In the demo: StationMerchant (the merchant asks the game for a price and waits for it) and Mother (an experimental ship computer that reads and writes the ship's log through tools).

Build it: Tool calling · Define and handle tool calls · Design a tool-call prompt · Compare tool-call models · Test tool calls in the editor

Memory and summarization¶

The player experiences: the character remembers. It does not lose the thread of a long conversation, it brings back what the player told it hours ago, and it still knows after the game is saved and loaded.

Building blocks: a rolling summary, a writable memory, save snapshots.

Memory works at three time scales.

Within a conversation. Every model has a fixed context window, and when the conversation outgrows it, the oldest turns drop out of the prompt. Add a Summarize node and the older turns are folded into a rolling summary instead, so the conversation can go on at a fixed token cost without the character forgetting how it started.

Facts that last. To keep specific facts, such as "the player likes cats" or "the player killed Atticus Pünd", write them into a memory the character can search. A WriteToMemory node stores a line in a named knowledge base, one per character if you like, and a Retrieve node on the same store, placed before the reply, brings the matching facts back when they become relevant. Writable memory is experimental in this release.

flowchart LR
    In["Player line"] --> R["Retrieve: character's memory"]
    R --> G["GenerateAndSpeak"]
    G --> W["WriteToMemory"]

To pull out structured facts, such as the clues a suspect mentioned, let a second agent read the exchange and answer through a grammar, so the result is always data your game can store.

Across saves. Your game owns the save file, and Tryll gives it what to put there: a snapshot of each agent's conversation, and a snapshot of every memory store. Load them back into a new session and the characters continue where they left off.

In the demo: TavernBartender (a long conversation kept coherent with a rolling summary) and Death on Set (a note-taker agent writes up each interview into the detective's case notes).

Build it: Keep long conversations coherent · Inspect and edit knowledge stores · Constrain output with a grammar · Export and import dialog · Save and load agent state

All together¶

The Death on Set murder mystery in the Unity demos combines most of these features in one graph per suspect:

flowchart LR
    In["Player line"] --> Guard["Immersion guard<br/>(regex + 2 classifiers)"]
    Guard -->|caught| Deflect["CannedResponse → Speak"]
    Guard -->|ok| CI["ClassifyIntentLLM"]
    CI -->|intent| I2I["IntentToInstruction"]
    CI -->|none| RW["Retrieve world lore"]
    RW --> RC["Retrieve suspect history<br/>(gated by game state)"]
    I2I --> Comp["Transform"]
    RC --> Comp
    Comp --> GAS["GenerateAndSpeak<br/>(per-suspect voice)"]
    GAS --> Sum["Summarize"]

Four suspects share one loaded model, each with its own voice. The game tracks what the player has discovered and widens each suspect's retrieval filter as secrets unlock. A fifth agent, the note-taker, quietly writes up each interview; see Ambient world for agents the player does not talk to.