Constrain Output with a Grammar

Force a Generate node to emit exactly one of a fixed set of machine-readable answers — a command, a choice, a tiny JSON object — by attaching a GBNF grammar. Every reply is then guaranteed to match the grammar; there are no parse failures to handle.

Prerequisites

• A connected session, with CreateSession already called.
• A language model on disk. For command parsing, prefer a model with thinking off (per-variant disable_thinking) — see Model Management.
• Read Constrained Output (GBNF) for what a grammar does and does not guarantee (syntax, never semantics).

Step 1 — write the grammar

A grammar is a set of rules. It must have a rule named root, and it must not be left-recursive. Keep the value sets small and use words that tokenise cleanly:

root   ::= action " " object
action ::= "grab" | "pull" | "cut" | "activate"
object ::= "wire" | "enemy" | "lever" | "door"

Every output is one of the 4×4 = 16 strings. To let the model signal a missing/unrecognised slot, add a sentinel as an alternative in each slot (e.g. object ::= "wire" | … | "unknown") — but note the model only uses the sentinel if you tell it to (Step 2) and it follows the instruction (verify in Step 3).

Step 2 — put it on a Generate node

Set grammar alongside a system_prompt that spells out the allowed words and the output order. The grammar enforces the shape; the prompt drives which words get chosen.

PythonC++

from tryll_client.graph import GraphDescription, GenerateParams, SamplingOverrides

GRAMMAR = (
    'root   ::= action " " object\n'
    'action ::= "grab" | "pull" | "cut" | "activate"\n'
    'object ::= "wire" | "enemy" | "lever" | "door"'
)
PROMPT = (
    "You control a game character. Reply with the single best command as "
    "'<action> <object>', choosing one action from {grab, pull, cut, "
    "activate} and one object from {wire, enemy, lever, door}. "
    "Output nothing else."
)

graph = (
    GraphDescription()
    .add_node("command", GenerateParams(
        system_prompt=PROMPT,
        grammar=GRAMMAR,
        sampling=SamplingOverrides(temperature=0.0),  # extraction, not creativity
    ))
    .set_start_node("command")
    .set_default_model_name("My Local Model")
)
agent = client.create_agent(graph)

```cpp using namespace Tryll::Client; using namespace Tryll::NodeParams;

GenerateParamsT gp; gp.system_prompt = "You control a game character. Reply with the single " "best command as '