Skip to content

First Inference in Unity

Unpack a Tryll release, drop the package and the server into your Unity project, configure one project setting, and watch a single Generate node print its answer to a text element during Play Mode — no chat UI required.

What you'll build

A scene with a TryllAgentComponent on a GameObject that, on Start, connects to a locally-spawned Tryll server, downloads a model on first run, sends the prompt "In one sentence: what is Tryll?", and streams the answer into a UI Text element.

Before you start

  • Unity 6 (6000.x) or later, with a blank 3D project.
  • ~5 GB free disk space for the auto-downloaded model and a working internet connection on first run.
  • Familiarity with creating GameObjects, adding components, and writing a short C# MonoBehaviour.

You do not need to build anything from source — the distribution ships pre-generated FlatBuffers bindings.

Step 1 — Get the distribution

  1. Download tryll-<version>.7z from the releases page and extract it somewhere temporary (e.g. C:\Downloads\tryll\).

  2. You'll get three sibling folders:

    tryll/
├── server/
│   ├── tryll_server.exe
│   ├── llama.dll, ggml*.dll, ggml-vulkan.dll
│   └── data/
│       ├── server-config.json
│       ├── models.json
│       ├── default-canned-responses.txt
│       └── default-guardrail-patterns.txt
├── UnityPlugin/
│   ├── package.json
│   ├── Runtime/
│   └── Editor/
└── UnrealPlugin/
    └── ...

The archive ships no model weights — the server lazy-downloads whatever the graph references, into .app-data/models/ next to the exe.

Step 2 — Install the Unity package

Copy both the server/ and UnityPlugin/ folders into your Unity project as a top-level Tryll/ folder:

<YourProject>/
├── Assets/
├── Packages/
│   └── manifest.json
├── Tryll/
│   ├── tryll_server.exe
│   ├── llama.dll, ggml*.dll, ggml-vulkan.dll
│   ├── data/
│   └── UnityPlugin/       <-- copy UnityPlugin/* here
│       ├── package.json
│       ├── Runtime/
│       └── Editor/
└── <YourProject>.sln

Then register the package in Packages/manifest.json:

{
  "dependencies": {
    "com.tryll.client": "file:../Tryll/UnityPlugin",
    ...
  }
}

Save the file. Unity will import the package; you should see no compile errors.

Add this to .gitignore to keep the model cache out of source control:

/Tryll/.app-data/

Why one Tryll/ folder?

Keeping the server binary and the package together means one path to set in Project Settings, one entry to gitignore, and one folder to update on a new release.

Step 3 — Configure Tryll Client in Project Settings

Open Edit → Project Settings → Tryll Client and set:

Setting Value Why
Auto Launch Server true (default) TryllClient spawns the server on Awake in standalone builds.
Editor Server Exe Path Tryll/tryll_server.exe Resolves relative to the Unity project root. Used by the editor (Play Mode and the Chat Window).
Build Server Exe Path tryll_server.exe (default) Used in packaged builds, relative to the game exe.
Allow Auto Model Downloading true First-inference convenience: CreateAgent will download missing models automatically.
Game Name e.g. my-game Short slug sent to the server on every session for telemetry grouping. Please fill it for convenience.

Settings are saved to Assets/Resources/TryllRuntimeSettings.asset (created on first access).

Allow Auto Model Downloading is a development flag

It removes friction during the first-inference loop, but it lets a CreateAgent call block for minutes while a model downloads. For shipped builds, drive downloads explicitly with the DownloadModel API and a progress UI. See Enable Auto Model Downloading.

Step 4 — Set up the scene

  1. Create a new empty GameObject — call it TryllDemo.
  2. Add a Tryll Agent Component (Add Component → Tryll → Tryll Agent Component).
  3. In the Inspector, expand Inline Graph Description:
    • Under Nodes, add one entry. Set Name = answer, Type = Generate. Leave Params empty for now. The node's Default Exit field can be left blank — an empty exit routes the turn to END automatically.
    • Set Start Node = answer.
    • Set Default Model Name to a catalog entry — spelling must match Tryll/data/models.json exactly. Recommended: Gemma 3 4B Instruct (Q4_K_M) (~3 GB download). Smaller / quicker: Llama 3.2 3B Instruct (Q4_K_M) (~2 GB download).
  4. Leave Workflow Asset empty — the inline description is used when no asset is assigned.
  5. Un-check Auto Create On Connect — this tutorial wires the lifecycle explicitly so the session is configured before the agent is created.
  6. Add a UI → Text (TMP) element to your scene — name it AnswerText.
  7. Create a C# script TryllDemo.cs and attach it to the same GameObject:
using UnityEngine;
using TMPro;
using Tryll.Client;

public class TryllDemo : MonoBehaviour
{
    public TMP_Text AnswerText;

    private TryllAgentComponent _agent;
    private string _accumulated = "";

    void Start()
    {
        _agent = GetComponent<TryllAgentComponent>();

        _agent.OnAnswerText.AddListener((text, isDelta, isFinal) =>
        {
            _accumulated = isDelta ? _accumulated + text : text;
            AnswerText.text = _accumulated;
        });

        _agent.OnTurnComplete.AddListener((status, debugInfo, tokens) =>
        {
            if (status != TryllTurnStatus.Success)
                Debug.LogError($"Turn failed: {status}");
        });

        _agent.OnError.AddListener(error =>
            Debug.LogError($"Tryll error: {error}"));

        _agent.OnAgentCreated.AddListener(() =>
            _agent.SendMessage("In one sentence: what is Tryll?"));

        var client = TryllClient.Instance;
        client.ConfigureSessionComplete += err =>
        {
            if (!err.IsOk) { Debug.LogError($"ConfigureSession failed: {err}"); return; }
            _agent.CreateAgent();
        };

        client.Connect();
        client.ConfigureSession(TryllInferenceEngine.LlamaCpp,
            TryllRuntimeSettings.Load().AllowAutoModelDownloading);
    }
}
  1. Assign the AnswerText reference in the Inspector.

Step 5 — Test without entering Play Mode

Before pressing Play, use the Chat Window to verify your setup is working:

  1. Open Window → Tryll → Chat.
  2. Confirm the Tryll Settings field shows your TryllRuntimeSettings asset (auto-found).
  3. Drag the TryllDemo GameObject's Tryll Agent Component into the Agent Component slot.
  4. Click Start — the window launches the server, connects, and creates the agent. The status label turns green: Ready.
  5. Type a message and press Enter. You should see a streamed reply.
  6. Click Stop when done.

See Test an Agent in the Editor for the full walkthrough including troubleshooting.

Step 6 — Enter Play Mode

Press Play.

On the first run you'll see:

  1. A brief pause while the server starts and the client connects.
  2. A longer pause (minutes, depending on your network) while the model downloads — watch the Console for [Tryll]-prefixed progress lines.
  3. The model loads from disk (a few seconds the first time it is used in a session).
  4. The answer streams into AnswerText.

Subsequent runs skip the download and load steps — the model stays in the server process, and .app-data/models/ persists across Play Mode sessions.

Where to see what's happening

Open the Console window and filter by [Tryll]:

Prefix Source Useful for
[TryllClient] TryllClient singleton Connection attempts, session config, agent lifecycle.
[TryllAgent] TryllAgentComponent Agent created/destroyed, DownloadProgress, errors.
[TryllServer] tryll_server.exe stdout (re-emitted by the client) Listening on 0.0.0.0:9100, model load lines, server errors.

A rotating server log is also written to Tryll/.app-data/logs/tryll.log (production log mode is the default in the shipped server-config.json).

What you built

  • A session connected to a Tryll server the client spawned for you.
  • One agent with a one-node graph.
  • Model output surfaced as OnAnswerText events (set stream="true" on the node's Params for token-by-token chunks instead of a single full-reply frame).

Where to go next

Troubleshooting

  • Compile errors on import — check that Tryll/UnityPlugin/Runtime/Generated/ contains .cs files. If that folder is empty, the distribution was packaged without the FlatBuffers step; contact support or re-download.
  • [TryllServer] Failed to find server exe — Editor Server Exe Path doesn't point at your copy. Remember it resolves relative to the Unity project root; with the layout above it should be Tryll/tryll_server.exe.
  • Stuck at connecting — the server crashed or didn't start in time. Check the Console for [TryllServer] lines and Tryll/.app-data/logs/tryll.log as a fallback.
  • ConfigureSession fails — usually a server-side error reading models.json or initialising the inference engine. Check [TryllServer] Console output.
  • CreateAgent hangs for minutes — the model is downloading because Allow Auto Model Downloading is on. Watch the Console for DownloadProgress lines, or pre-download from a small UI using the DownloadModel API instead.
  • OnError fires during CreateAgent — read the error message. Common causes: Default Model Name doesn't match any entry in models.json, or a route targets a node that doesn't exist.
  • Nothing appears in AnswerText — confirm OnAnswerText is wired and that _accumulated is being assigned to AnswerText.text. With the default stream="false", OnAnswerText fires once with the full reply immediately before OnTurnComplete.