Python Client API Reference¶
Full API reference for the tryll_client Python package, auto-generated from
Google-style docstrings. The package lives at tryll/clients/python/tryll_client/.
The public surface — the __all__ re-exported from tryll_client.__init__ — is:
TryllClient, ConnectedSession, AgentProxy, AgentVariables, VariableDecl,
TryllError, WavWriter, GraphDescription, InferenceEngine, ManagedServer,
ModelInfo, ModelStatus, NodeType, and the KV-cache lifecycle types
AgentKvCacheInitialization, AgentKvCacheResidency,
AgentKvCachePrefixStatus, AgentKvCacheStatus,
AgentKvCachePrefillResult, and AgentKvCacheEvictResult.
Node-param types (GenerateParams, SamplingOverrides, SendAnswer, …) are
imported directly from tryll_client.graph / tryll_client._generated.node_params,
not from the top-level package.
ConnectedSession is the recommended entry point for local-server deployments;
obtain it via TryllClient.run_and_connect.
Internal modules (wire, codec, _generated) are excluded.
tryll_client¶
tryll_client ¶
tryll_client — Python client library for the Tryll server.
Quick start::
from tryll_client import TryllClient, InferenceEngine, GraphDescription
from tryll_client.graph import GenerateParams, SamplingOverrides
from tryll_client._generated.node_params import SendAnswer
client = TryllClient.connect("127.0.0.1", 9100)
client.create_session(InferenceEngine.LlamaCpp)
graph = (GraphDescription()
.add_generate("generate", GenerateParams(
system_prompt="You are a helpful assistant.",
send=SendAnswer.Streamed,
sampling=SamplingOverrides(temperature=0.0, seed=42),
# default_exit defaults to "" = END
))
.set_start_node("generate")
.set_default_model_name("Llama 3.2 3B Instruct (Q4_K_M)"))
agent = client.create_agent(graph)
response = agent.send_message("Hello!")
agent.destroy()
client.shutdown()
Prerequisites
The CMake build for server/ must have run at least once to generate FlatBuffers Python code into tryll_client/_generated/. Run: cmake --build server/build
AgentProxy ¶
Handle for one server-side agent created via TryllClient.create_agent().
The proxy exposes the user-facing turn API — :meth:send_message and
:meth:destroy — and caches diagnostics from the last completed turn
on last_* properties.
Bind the proxy to its owning client and server-side agent id.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
'TryllClient'
|
Parent :class: |
required |
agent_id
|
int
|
Server-assigned agent identifier returned in the
|
required |
node_baselines
|
dict[str, 'NodeParamsBase'] | None
|
Deep-copied params per graph node name, captured
at |
None
|
variables
|
dict[str, tuple[int, Any]] | None
|
|
None
|
last_debug_info
property
¶
JSON diagnostics string from the most recent send_message call.
Returns:
| Type | Description |
|---|---|
str
|
Server-produced JSON document attached to |
str
|
when diagnostics were enabled on agent creation; otherwise an |
str
|
empty string. Empty string also before the first turn. |
last_ttft_s
property
¶
Time-to-first-token in seconds for the most recent turn.
Returns:
| Type | Description |
|---|---|
float | None
|
Seconds elapsed from |
float | None
|
streamed |
float | None
|
arrived (e.g. canned-response paths that skip streaming). |
last_answer_chunk_count
property
¶
Number of AnswerText chunks received for the last turn.
Typically one chunk per generated token when streaming; useful for verifying the stream was delivered incrementally.
last_tokens_generated
property
¶
Server-reported generated token count for the last turn.
Authoritative for both streaming and non-streaming modes. Zero if the server has not yet completed a turn.
set_on_answer_text ¶
Register (or clear) a persistent callback for streaming text chunks.
The callback is invoked on the reader thread for every AnswerText
frame received for this agent — including frames from server-initiated
turns (e.g. voice autosend after VoiceInput.end_utterance).
The callback signature is (node_name: str, text: str, is_delta: bool,
is_final: bool). node_name identifies which node in the workflow
graph produced this text (multi-sender attribution); existing callers
that only need the text can ignore it. Must return quickly and must
not call blocking client methods.
Call with None to unregister the current callback.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cb
|
Callable[[str, str, bool, bool], None] | None
|
Callable invoked with |
required |
set_on_turn_complete ¶
Register (or clear) a persistent callback for turn-complete notifications.
The callback is invoked on the reader thread once per TurnComplete
frame for this agent. Covers both client-initiated turns (via
:meth:send_message) and server-initiated turns (voice autosend).
The callback signature is
(status: int, debug_info: str, tokens_generated: int)
where status maps to the wire TurnStatus enum
(0 = Ok, 1 = Error, 2 = Cancelled).
Call with None to unregister the current callback.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cb
|
Callable | None
|
Callable invoked with |
required |
set_on_tool_call ¶
Register (or clear) a callback for tool-call notifications.
The callback is invoked on the reader thread for every
NodeEvent frame with event_type == "tool_call" the server
sends for this agent — i.e. when the graph has a ToolCall node
with a notifying disposition (Notify / NotifyAndAcknowledge /
Pause / PauseAndAcknowledge / AwaitResult). It must return
quickly and must not call any blocking :class:TryllClient or
:class:AgentProxy methods.
The callback signature is (tool_name: str, arguments_json: str)
where arguments_json is a compact JSON object, e.g.
'{"city": "Berlin"}'. Parse it with :func:json.loads as
needed.
Call with None to unregister the current callback.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cb
|
Callable[[str, str], None] | None
|
Callable invoked with |
required |
set_on_tool_call_with_id ¶
Register a call-ID-aware tool-call callback.
The callback receives (call_id, tool_name, arguments_json) on the
reader thread. Echo call_id in a :class:ToolResult passed through
the keyword-only tool_results argument of :meth:resume_async
(preferred from this callback) or :meth:resume (only from a
non-reader thread). The legacy :meth:set_on_tool_call callback
remains supported.
set_on_error ¶
Register (or clear) a per-agent client-originated error callback.
Fired on the reader thread when a user callback (tool_call, paused,
etc.) raises. Does not replace server ErrorResponse delivery for
pending requests. The error callback itself must not raise; secondary
failures are swallowed to protect the reader.
set_on_intent_classified ¶
Register (or clear) a callback for intent-classification notifications.
Fired on the reader thread for every NodeEvent with
event_type == "intent_classified" — emitted by
ClassifyIntentNode on its "found" path when
notify_client = "true".
The callback signature is
(intent: str, record_id: str, record_index: int, distance: float).
Call with None to unregister.
set_on_paused ¶
Register (or clear) a callback for pause notifications.
Fired on the reader thread for every NodeEvent with
event_type == "paused" — emitted by the executor when it pauses
the turn between nodes (see the Pause node and pausing
ToolCallDisposition values). While paused, the agent still
counts as busy for :meth:send_message, but
:meth:change_params / :meth:change_param are allowed.
The callback signature is (node_name: str, pending_exit: str)
where pending_exit is the exit route that will be taken on a
plain (non-jump) :meth:resume.
Call with None to unregister.
set_on_tts_audio_format ¶
Register (or clear) a callback for the TTS audio format frame.
Emitted by the server exactly once per turn that produces TTS audio,
before the first :meth:set_on_tts_audio callback. The callback
signature is (node_name: str, sample_rate: int, channels: int,
bits_per_sample: int) — node_name is the producing node's name
(multi-sender attribution; empty string if the server omitted it).
Fired on the reader thread — must return quickly and must not call
blocking client methods.
Call with None to unregister.
set_on_tts_audio ¶
Register (or clear) a callback for streaming TTS audio chunks.
Fired on the reader thread for every TtsAudioFrame received for
this agent. The callback signature is (node_name: str, pcm:
memoryview) — node_name is the producing node's name
(multi-sender attribution; matches the preceding
:meth:set_on_tts_audio_format call for the same producer). The
pcm argument is a zero-copy memoryview over int16 LE PCM at
the format declared by the preceding format callback. The view is
valid only during the call — copy with bytes(pcm) or
np.frombuffer(pcm, dtype=np.int16).copy() if needed.
TtsAudioFrame carries no is_final flag — end-of-stream is
signalled by TurnComplete. Wire :meth:set_on_turn_complete
(or close your sink in the surrounding with-block) to flush.
Call with None to unregister.
set_on_node_event ¶
Register (or clear) a generic NodeEvent fallback callback.
Fired on the reader thread for any NodeEvent whose
event_type is unrecognised or whose typed callback (e.g.
:meth:set_on_tool_call, :meth:set_on_intent_classified) is
not registered. When a typed callback handles the event, this
one is NOT invoked for that event.
The callback signature is
(node_name: str, event_type: str, kv_pairs: list[tuple[str, str]]).
Call with None to unregister.
send_message ¶
Send a user message and return the complete assistant response.
Blocks until TurnComplete is received from the server,
accumulating all AnswerText chunks into a single string.
Diagnostics from TurnComplete (debug_info,
tokens_generated) are cached on the last_* properties.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
User-turn text to send to the agent. |
required |
timeout
|
float
|
Maximum seconds to wait for |
120.0
|
Returns:
| Type | Description |
|---|---|
str
|
The full concatenated assistant response text. |
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors, decode failures, or
if |
send_message_async ¶
Send a user message without blocking; return a Future for the result.
Unlike :meth:send_message, this returns immediately so the caller can act
mid-turn (e.g. watch streaming callbacks and call :meth:cancel). The
Future resolves with a :class:TurnResult on TurnComplete (including a
cancelled turn, where status == TurnStatus.Cancelled) or raises
:class:~tryll_client.errors.TryllError on a server error.
Await it via fut.result(timeout) (raises concurrent.futures.TimeoutError
if the turn does not finish in time). Streaming chunks still arrive via the
set_on_answer_text / set_on_tts_audio callbacks.
Note: use :meth:cancel to cancel the turn — do not call fut.cancel(),
which only cancels a scheduled call, not the server-side turn.
change_param ¶
Apply a single-field mutation using a dotted attribute path.
Clones the baseline params captured at create_agent, coerces
value to the leaf field type (when value is a string), assigns
via :func:~tryll_client._generated.node_params_codec.set_dotted,
and sends the full params object. On success the baseline is updated
so later mutations compose.
Example::
agent.change_param("generate", "sampling.temperature", "0.5")
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node_name
|
str
|
Instance name of the target node. |
required |
path
|
str
|
Dotted path to the field (e.g. |
required |
value
|
Any
|
New value (string from JSON/dialog scripts, or a native Python value). |
required |
timeout
|
float
|
Maximum seconds to wait for the server |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
If node_name is unknown (code 3005) or the server reports another error. |
change_params ¶
Apply a typed node-parameter update to a workflow node at runtime.
The typed params object must match the concrete type of the target node.
Structural fields must match the create-time values or the server
returns ParamNotMutable. On success, updates the internal baseline
used by :meth:change_param.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node_name
|
str
|
Instance name of the target node. |
required |
params
|
'NodeParamsBase'
|
Typed node parameters (generated DTO). |
required |
timeout
|
float
|
Maximum seconds to wait for the server |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors ( |
destroy ¶
Request agent destruction on the server and wait for Ack.
After this call returns, the proxy is no longer usable; further
send_message calls will raise :class:TryllError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
timeout
|
float
|
Maximum seconds to wait for the |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors or timeout. |
cancel ¶
Cooperatively cancel the agent's in-flight turn and wait for Ack.
The active send_message ends with TurnStatus.Cancelled. Call this
from a different thread than the blocking send_message to interrupt a
turn mid-flight. No-op server-side if the agent is idle.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mode
|
int
|
:class: |
0
|
timeout
|
float
|
Maximum seconds to wait for the |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors or timeout. |
resume ¶
Resume a turn that is currently paused and wait for Ack.
Not safe on the reader thread (tool-call / paused callbacks):
blocking here deadlocks the sole frame reader. Use
:meth:resume_async from those callbacks instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
resume_node
|
str
|
Empty continues via the paused node's pending exit route; non-empty jumps to that node by name (must exist in the agent's graph). |
''
|
timeout
|
float
|
Maximum seconds to wait for the |
30.0
|
tool_results
|
Sequence[ToolResult | tuple[str, str]]
|
Complete result batch for the paused tool-call pass.
Each result must echo the |
()
|
Raises:
| Type | Description |
|---|---|
TryllError
|
|
resume_async ¶
Resume a paused turn without blocking; returns a Future for the Ack.
Safe to call from reader-thread callbacks. The Future resolves to
None on success or raises :class:TryllError on failure.
prefill_kv_cache ¶
Synchronise eligible language-model contexts to the reusable prefix.
evict_kv_cache ¶
Release eligible raw contexts while retaining agent state.
get_kv_cache_status ¶
Return aggregate residency and reusable-prefix state.
append_interactions ¶
Append scripted user/assistant interactions without running the graph.
Each item is (user_message, assistant_message). Empty / None
strings omit that side; both empty → skipped. Idle-only
(AgentBusy 3004 while running/paused).
Returns:
| Type | Description |
|---|---|
int
|
Number of interactions actually appended. |
remove_interactions_from_end ¶
Remove whole interactions from the end of the dialog.
count == 0 is a no-op; count >= size clears all. Idle-only.
Returns:
| Type | Description |
|---|---|
int
|
Number of interactions actually removed. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If count is negative. |
TryllError
|
On server-reported errors ( |
ToolResult
dataclass
¶
One model-visible result correlated to a tool-call call_id.
WavWriter ¶
Write streaming TTS audio to a 16-bit mono WAV file.
Lazily opens the output file on the first format callback, appends every
TTS audio chunk, and closes the file on :meth:on_turn_complete (or
explicit :meth:close).
TtsAudioFrame does not carry an is_final flag — end-of-stream is
signalled by TurnComplete. Use :meth:bind_to (which wires all three
callbacks) or register :meth:on_turn_complete yourself.
Single producer per instance. A WavWriter captures exactly one TTS
producer per turn — a second :meth:on_format call before :meth:close
raises :class:RuntimeError instead of silently reopening (and truncating)
the file. This is not a limitation in practice: the voice loop recreates a
fresh writer for every turn, and GenerateAndSpeak/Speak each emit at most
one producer per turn. A graph with two TTS-emitting nodes active in the
same turn needs one WavWriter per node (keyed by node_name), not
one shared instance.
Bind a sink path; the file is created lazily on the format callback.
on_format ¶
Open the output file using the announced format.
Raises :class:RuntimeError if called a second time before
:meth:close — multi-producer capture into a single WavWriter is
not supported (see the class docstring); allowing it would silently
truncate the first producer's audio.
on_audio ¶
Append one TTS chunk. Silently drops chunks that arrive before format.
node_name is accepted for signature compatibility (see
:meth:on_format) but unused.
on_turn_complete ¶
Close the file. Signature matches :meth:AgentProxy.set_on_turn_complete.
bind_to
classmethod
¶
Create a :class:WavWriter and wire it to agent's TTS callbacks.
Registers :meth:on_format, :meth:on_audio, and
:meth:on_turn_complete on agent. Note that this overrides any
previously registered turn_complete callback on the agent — wire
the three handlers manually if you need a custom turn-complete sink.
ConnectedSession ¶
RAII wrapper that owns a :class:~tryll_client.managed_server.ManagedServer
and a connected :class:TryllClient.
Obtain via :meth:TryllClient.run_and_connect. Use as a context manager
(recommended) to guarantee teardown in the right order — client first, then
server::
with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
session.client.create_session(InferenceEngine.LlamaCpp)
agent = session.client.create_agent(graph)
print(agent.send_message("hi"))
TryllClient ¶
Synchronous TCP session to the Tryll server.
Usage::
client = TryllClient.connect("127.0.0.1", 9100)
client.create_session(InferenceEngine.LlamaCpp)
graph = GraphDescription().add_node(...).set_start_node(...)
# Exit fields (e.g. default_exit="") live on each node's typed params
agent = client.create_agent(graph)
response = agent.send_message("Hello")
agent.destroy()
client.shutdown()
connect
classmethod
¶
Connect to the Tryll server and wait for ConnectionReady.
run_and_connect
classmethod
¶
run_and_connect(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, connect_timeout=30.0, idle_shutdown_timeout=60)
Spawn tryll_server and connect — recommended one-call factory.
Starts a local server process (passing --port <port> and, unless
idle_shutdown_timeout is 0, --idle-shutdown-timeout <N> on the
command line), waits for its TCP port to be ready, then opens a
session. The returned :class:ConnectedSession owns both the server
process and the client; use it as a context manager to guarantee
clean teardown::
with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
session.client.create_session(InferenceEngine.LlamaCpp)
agent = session.client.create_agent(graph)
If the connected server's advertised exe_path does not match exe
(a stranger raced us onto the port), the connection is torn down, the
wrong process is hard-killed, and a fresh server is spawned on a new
free port — up to a few attempts before raising.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exe
|
Path | str
|
Path to |
required |
host
|
str
|
Host for TCP probe and :meth: |
'127.0.0.1'
|
port
|
int
|
Port passed as |
9100
|
cwd
|
Path | None
|
Working directory for the child process (defaults to
|
None
|
extra_args
|
Sequence[str]
|
Additional CLI arguments after |
()
|
stdout
|
Path | None
|
File path for child stdout ( |
None
|
stderr
|
Path | None
|
File path for child stderr ( |
None
|
start_timeout
|
float
|
Seconds to wait for the TCP port to open (default 30). |
30.0
|
stop_timeout
|
float
|
Seconds to wait for graceful server exit (default 8). |
8.0
|
connect_timeout
|
float
|
Seconds to wait for :class: |
30.0
|
idle_shutdown_timeout
|
int
|
Seconds of idle after which the launched server
self-exits (default 60); see :meth: |
60
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
'ConnectedSession'
|
class: |
'ConnectedSession'
|
connected. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If exe does not exist. |
TimeoutError
|
If the port does not open within start_timeout. |
TryllError
|
If the session handshake fails, or identity verification keeps failing after a few retries. |
create_session ¶
create_session(engine, game_name='', timeout=30.0, stt_engine=0, tts_engine=0, embedding_engine=0, storage_data_folder='')
Create the logical session for this connection (mandatory, one-shot).
Sends CreateSessionRequest and waits for CreateSessionResponse, which
carries the server-allocated session_id (stored on :attr:session_id).
Must be called exactly once after :meth:connect and before
:meth:create_agent or any model/storage/voice call — those are rejected
with SessionNotReady until the session exists. A second call is
rejected by the server with SessionAlreadyExists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
engine
|
InferenceEngine | int
|
Inference backend for language models. |
required |
game_name
|
str
|
Integration identifier for telemetry grouping (e.g. |
''
|
timeout
|
float
|
Maximum seconds to wait for the server response. |
30.0
|
stt_engine
|
InferenceEngine | int
|
Inference backend for STT (speech-to-text) models.
Defaults to :attr: |
0
|
tts_engine
|
InferenceEngine | int
|
Inference backend for TTS (text-to-speech) models.
Defaults to :attr: |
0
|
embedding_engine
|
InferenceEngine | int
|
Inference backend for embedding models.
Defaults to :attr: |
0
|
storage_data_folder
|
str
|
Optional per-session storage data folder. Relative
storage paths in node params (and voice-input hotwords) resolve against
this folder. Empty (default) means the server falls back to its
configured |
''
|
set_inference_throttle ¶
Report how hard the server should yield the GPU back to the game.
Fire-and-forget: no response, no error surfaced locally. Never sent automatically — a session that never calls this runs unthrottled, identically to a client on an older protocol version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
level
|
float
|
0 = full speed (default), 1 = maximum yielding. Clamped server-side; out-of-range values do not raise here. |
required |
create_string_storage ¶
Create a named StringStorage on the server.
Provide either strings (inline list) or file_path (server-side file path).
The storage can then be referenced by name in node params via string_storage.
create_keyed_string_storage ¶
Create a Map or Multimap-kind StringStorage on the server.
keys and values must be the same length.
For Map kind all keys must be unique.
The storage can then be referenced by name in IntentToInstructionNode
(and other keyed-consumer nodes) via the string_storage param.
destroy_string_storage ¶
Destroy a named StringStorage on the server.
Nodes that already hold the storage are unaffected.
create_embedded_string_storage ¶
create_embedded_string_storage(name, config_path=None, strings=None, embedding_model=None, timeout=None)
Create a named EmbeddedStringStorage on the server.
Path A: supply config_path (server-side *.json).
Path B: supply strings (inline list) + embedding_model.
Returns an EmbeddedStorageInfo with record_count and embedding_dim.
destroy_embedded_string_storage ¶
Destroy a named EmbeddedStringStorage on the server.
Nodes that already hold the storage are unaffected.
create_voice_input ¶
create_voice_input(model_name, sample_rate=16000, channels=1, bits_per_sample=16, vad_threshold=0.5, vad_min_silence_ms=500, vad_speech_pad_ms=250, hotwords_storage_path='', hotwords_score=1.5, timeout=30.0)
Create a server-side VoiceInput session for speech-to-text.
Returns a :class:~tryll_client.voice_input.VoiceInput handle managing the
server-side session. Use as a context manager (recommended) for guaranteed
cleanup.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_name
|
str
|
Catalog name of the STT model, e.g.
|
required |
sample_rate
|
int
|
Sample rate of audio buffers you will push (Hz). |
16000
|
channels
|
int
|
Channel count (1 = mono, typical for mic capture). |
1
|
bits_per_sample
|
int
|
Bit depth of raw PCM samples (only 16-bit is supported). |
16
|
vad_threshold
|
float
|
Silero VAD speech-probability threshold (0.0–1.0). |
0.5
|
vad_min_silence_ms
|
int
|
Silence duration (ms) that closes a speech segment. |
500
|
vad_speech_pad_ms
|
int
|
Padding (ms) added around detected speech. |
250
|
hotwords_storage_path
|
str
|
Relative path (resolved against the session
storage folder) to a |
''
|
hotwords_score
|
float
|
Per-token bias strength applied to every phrase in the storage. 1.5 is a gentle default; 2.5 is aggressive. |
1.5
|
timeout
|
float
|
Maximum seconds to wait for the server response. |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
If the model cannot be found, the hotwords storage file is missing, or the request times out. |
create_agent ¶
create_agent(graph, enable_diagnostics=False, timeout=None, maintain_dialogue_history=True, variables=None, kv_cache_initialization=AgentKvCacheInitialization.AllocateOnly, workload=AgentWorkload.Interactive)
Create a server-side agent and return a proxy handle.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
GraphDescription
|
Fully-built graph description. |
required |
enable_diagnostics
|
bool
|
When True the server serialises per-node execution data into TurnComplete.debug_info for every turn. |
False
|
timeout
|
float | None
|
Maximum seconds to wait for the response. Defaults to 30 s. |
None
|
maintain_dialogue_history
|
bool
|
When True (default) the agent keeps its full dialogue history. When False the history is discarded after each turn — a stateless agent, useful for technical/classification agents that should not store or project prior turns. |
True
|
variables
|
dict[str, Any] | None
|
Declares this agent's per-agent variables — a
|
None
|
workload
|
AgentWorkload | int
|
:class: |
Interactive
|
list_models ¶
Request all models known to the server for the session's engine.
load_model ¶
Explicitly load and pin a model into memory.
The model stays in memory until :meth:unload_model is called,
regardless of whether any agents are using it.
Raises :class:TryllError if the model cannot be resolved or loaded.
unload_model ¶
Unpin a previously pinned model.
If no agents are currently using the model it is freed immediately; otherwise it will be freed when the last agent using it is destroyed.
download_model ¶
Initiate a model download on the server and block until complete.
on_progress is called with (model_name, bytes_downloaded, total_bytes, percent) for each progress frame received. Raises TryllError on failure.
send_debug_command ¶
Send a dev-only debug command to the server.
The server handles this only in non-Production builds with
debug_commands.enabled = true; otherwise it returns a TryllError.
command is a verb (e.g. "crash"); payload is verb-specific
(for crash it is the mode token: nullderef | abort | throw
| stackoverflow | fastfail; empty defaults to nullderef).
Set await_response=False for fatal verbs like crash that kill the
server and never reply — the call sends the frame and returns immediately;
the caller detects success via the ensuing TCP disconnect. When
await_response is True, returns the server's result string.
wait_for_disconnect ¶
Block until the server drops the connection, or timeout elapses.
The reader thread exits when the socket reaches EOF / resets (e.g. the
server process died), so joining it is a reliable disconnect signal.
Returns True if the connection dropped within timeout. Used after a
fatal debug command (crash) whose only success signal is the drop.
TryllError ¶
Bases: Exception
Raised on server errors, protocol errors, or timeouts.
Attributes:
| Name | Type | Description |
|---|---|---|
code |
Numeric error code from the server's |
Initialise a Tryll client error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
Human-readable description of the failure, typically
copied from the server's |
required |
code
|
int
|
Numeric error code from |
0
|
GraphDescription ¶
Fluent builder for a workflow graph sent to the server.
Typical usage::
graph = (GraphDescription()
.add_generate("gen", GenerateParams(
model_name="qwen2.5-0.5b-instruct",
system_prompt="You are helpful."))
.set_start_node("gen"))
agent = await client.create_agent(graph)
InferenceEngine ¶
Bases: IntEnum
Inference backend. Mirrors the FlatBuffers InferenceEngine enum.
ModelInfo
dataclass
¶
Summary information for one model returned by list_models().
ModelStatus ¶
Bases: IntEnum
Status of a model as reported by the server.
NodeType ¶
Bases: IntEnum
Node type ordinal. Mirrors the NodeParams union order in nodes.fbs (0-based; the wire discriminator is this value + 1).
The union discriminant in NodeParams supersedes this for the wire format;
NodeType is kept for client-side ergonomics and generated convenience APIs.
ManagedServer ¶
ManagedServer(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, idle_shutdown_timeout=_DEFAULT_IDLE_SHUTDOWN_TIMEOUT)
RAII handle around a child tryll_server process.
Use as a context manager (recommended), or call :meth:stop explicitly::
with ManagedServer.start(exe=Path("..."), port=9100) as srv:
client = TryllClient.connect(srv.host, srv.port)
...
No executable discovery is performed — pass the path explicitly.
The server is always launched with --port <port> so the caller's port
takes precedence over whatever is in server-config.json, and (unless
idle_shutdown_timeout is 0) with --idle-shutdown-timeout <N> so it
self-exits once no session is using it — :meth:stop releases our handle
but does not kill the process.
start
classmethod
¶
start(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, idle_shutdown_timeout=_DEFAULT_IDLE_SHUTDOWN_TIMEOUT)
Spawn the server and block until its TCP port is ready.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exe
|
Path | str
|
Path to |
required |
host
|
str
|
Host string for the TCP ready-probe (default |
'127.0.0.1'
|
port
|
int
|
TCP port; passed as |
9100
|
cwd
|
Path | None
|
Working directory for the child process (defaults to
|
None
|
extra_args
|
Sequence[str]
|
Additional CLI arguments appended after |
()
|
stdout
|
Path | None
|
File path for child stdout redirection ( |
None
|
stderr
|
Path | None
|
File path for child stderr redirection ( |
None
|
start_timeout
|
float
|
Seconds to wait for the TCP port to open (default 30). |
30.0
|
stop_timeout
|
float
|
Seconds to wait for clean exit on :meth: |
8.0
|
idle_shutdown_timeout
|
int
|
Seconds of idle (zero sessions, no in-flight
downloads) after which the server self-exits; passed as
|
_DEFAULT_IDLE_SHUTDOWN_TIMEOUT
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
'ManagedServer'
|
class: |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If exe does not exist. |
TimeoutError
|
If the port does not open within start_timeout. |
OSError
|
If the process cannot be spawned. |
stop ¶
Release the process handle without killing the server.
The server self-exits via its idle-shutdown timer (managed mode).
Callers that need an immediate, guaranteed stop (e.g. spawn succeeded
but connect/handshake failed, so no session was ever created and
nothing else can be relying on this process) should call
:meth:terminate instead. The QA harness uses
:func:stop_server_process directly for the same reason.
terminate ¶
Immediately hard-kill the child process, if still running.
Idempotent — safe to call more than once, including after :meth:stop.
Intended for failure paths where the caller knows this process is
theirs and nothing else can depend on it. Do not call this after a
successful handshake, since another consumer could be relying on the
server staying up until its own idle timeout.
AgentWorkload ¶
Bases: IntEnum
Whether a human ever waits on an agent's turns.
Purely a scheduling hint for the inference budget controller — it never changes node semantics. Set Background for world-logic/evaluator agents so their work is paced or deferred first under GPU pressure.
AgentVariables ¶
Mapping-like local mirror of one agent's declared per-agent variables.
Seed the mirror from the declaration passed to create_agent.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
'TryllClient'
|
Parent :class: |
required |
agent_id
|
int
|
Server-assigned agent id this store belongs to. |
required |
declared
|
Optional[dict[str, tuple[int, Any]]]
|
|
None
|
set ¶
Stage an Assign — value's inferred type must match the declaration.
A plain Python int is accepted for a variable declared float
(matches the filter grammar's int-to-float promotion); every other
type mismatch raises immediately, before anything is queued for the
wire.
add_to_set ¶
Stage an idempotent AddToSet — name must be a declared string set.
remove_from_set ¶
Stage an idempotent RemoveFromSet — name must be a declared string set.
stage_batch ¶
Stage a batch atomically: validate+stage every op, or roll back all.
ops is a list of (op_name, name, value) where op_name is one
of assign / add_to_set / remove_from_set / reset (the
value is ignored for reset). On the first invalid op the mirror
and staged queue are restored to their pre-call state and the error is
re-raised, so a rejected batch leaves nothing dirty — mirroring the
server's own validate-all-then-apply atomicity.
flush_if_dirty ¶
Send every staged mutation as one batched UpdateAgentVariablesRequest.
Called automatically at the top of send_message,
send_message_async, resume, change_param, and
change_params — deliberately not before cancel. No-op when
nothing is staged.
The batch is retained in-flight until acknowledged. On a transient
AgentBusy it is re-staged behind any newer writes and the error is
re-raised; on a hard rejection the affected names are re-synced from the
server before re-raising, so a rejected batch never leaves the mirror
silently diverged.
sync_from_server ¶
Refresh the local mirror from the server's authoritative current values.
Names with a locally newer (staged or in-flight) write are skipped, so a server snapshot never clobbers an unsent local change.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
names
|
Optional[list[str]]
|
Variable names to fetch; empty/ |
None
|
VariableDecl
dataclass
¶
One agent-variable value plus immutable declaration metadata.
tryll_client.client¶
tryll_client.client ¶
TryllClient — synchronous TCP client for the Tryll server.
Threading model
- A background reader thread continuously receives frames and dispatches them to registered pending requests via threading.Event.
- The main (calling) thread sends requests and blocks on the Event.
- All access to _pending is protected by _lock.
TryllClient ¶
Synchronous TCP session to the Tryll server.
Usage::
client = TryllClient.connect("127.0.0.1", 9100)
client.create_session(InferenceEngine.LlamaCpp)
graph = GraphDescription().add_node(...).set_start_node(...)
# Exit fields (e.g. default_exit="") live on each node's typed params
agent = client.create_agent(graph)
response = agent.send_message("Hello")
agent.destroy()
client.shutdown()
connect
classmethod
¶
Connect to the Tryll server and wait for ConnectionReady.
run_and_connect
classmethod
¶
run_and_connect(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, connect_timeout=30.0, idle_shutdown_timeout=60)
Spawn tryll_server and connect — recommended one-call factory.
Starts a local server process (passing --port <port> and, unless
idle_shutdown_timeout is 0, --idle-shutdown-timeout <N> on the
command line), waits for its TCP port to be ready, then opens a
session. The returned :class:ConnectedSession owns both the server
process and the client; use it as a context manager to guarantee
clean teardown::
with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
session.client.create_session(InferenceEngine.LlamaCpp)
agent = session.client.create_agent(graph)
If the connected server's advertised exe_path does not match exe
(a stranger raced us onto the port), the connection is torn down, the
wrong process is hard-killed, and a fresh server is spawned on a new
free port — up to a few attempts before raising.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exe
|
Path | str
|
Path to |
required |
host
|
str
|
Host for TCP probe and :meth: |
'127.0.0.1'
|
port
|
int
|
Port passed as |
9100
|
cwd
|
Path | None
|
Working directory for the child process (defaults to
|
None
|
extra_args
|
Sequence[str]
|
Additional CLI arguments after |
()
|
stdout
|
Path | None
|
File path for child stdout ( |
None
|
stderr
|
Path | None
|
File path for child stderr ( |
None
|
start_timeout
|
float
|
Seconds to wait for the TCP port to open (default 30). |
30.0
|
stop_timeout
|
float
|
Seconds to wait for graceful server exit (default 8). |
8.0
|
connect_timeout
|
float
|
Seconds to wait for :class: |
30.0
|
idle_shutdown_timeout
|
int
|
Seconds of idle after which the launched server
self-exits (default 60); see :meth: |
60
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
'ConnectedSession'
|
class: |
'ConnectedSession'
|
connected. |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If exe does not exist. |
TimeoutError
|
If the port does not open within start_timeout. |
TryllError
|
If the session handshake fails, or identity verification keeps failing after a few retries. |
create_session ¶
create_session(engine, game_name='', timeout=30.0, stt_engine=0, tts_engine=0, embedding_engine=0, storage_data_folder='')
Create the logical session for this connection (mandatory, one-shot).
Sends CreateSessionRequest and waits for CreateSessionResponse, which
carries the server-allocated session_id (stored on :attr:session_id).
Must be called exactly once after :meth:connect and before
:meth:create_agent or any model/storage/voice call — those are rejected
with SessionNotReady until the session exists. A second call is
rejected by the server with SessionAlreadyExists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
engine
|
InferenceEngine | int
|
Inference backend for language models. |
required |
game_name
|
str
|
Integration identifier for telemetry grouping (e.g. |
''
|
timeout
|
float
|
Maximum seconds to wait for the server response. |
30.0
|
stt_engine
|
InferenceEngine | int
|
Inference backend for STT (speech-to-text) models.
Defaults to :attr: |
0
|
tts_engine
|
InferenceEngine | int
|
Inference backend for TTS (text-to-speech) models.
Defaults to :attr: |
0
|
embedding_engine
|
InferenceEngine | int
|
Inference backend for embedding models.
Defaults to :attr: |
0
|
storage_data_folder
|
str
|
Optional per-session storage data folder. Relative
storage paths in node params (and voice-input hotwords) resolve against
this folder. Empty (default) means the server falls back to its
configured |
''
|
set_inference_throttle ¶
Report how hard the server should yield the GPU back to the game.
Fire-and-forget: no response, no error surfaced locally. Never sent automatically — a session that never calls this runs unthrottled, identically to a client on an older protocol version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
level
|
float
|
0 = full speed (default), 1 = maximum yielding. Clamped server-side; out-of-range values do not raise here. |
required |
create_string_storage ¶
Create a named StringStorage on the server.
Provide either strings (inline list) or file_path (server-side file path).
The storage can then be referenced by name in node params via string_storage.
create_keyed_string_storage ¶
Create a Map or Multimap-kind StringStorage on the server.
keys and values must be the same length.
For Map kind all keys must be unique.
The storage can then be referenced by name in IntentToInstructionNode
(and other keyed-consumer nodes) via the string_storage param.
destroy_string_storage ¶
Destroy a named StringStorage on the server.
Nodes that already hold the storage are unaffected.
create_embedded_string_storage ¶
create_embedded_string_storage(name, config_path=None, strings=None, embedding_model=None, timeout=None)
Create a named EmbeddedStringStorage on the server.
Path A: supply config_path (server-side *.json).
Path B: supply strings (inline list) + embedding_model.
Returns an EmbeddedStorageInfo with record_count and embedding_dim.
destroy_embedded_string_storage ¶
Destroy a named EmbeddedStringStorage on the server.
Nodes that already hold the storage are unaffected.
create_voice_input ¶
create_voice_input(model_name, sample_rate=16000, channels=1, bits_per_sample=16, vad_threshold=0.5, vad_min_silence_ms=500, vad_speech_pad_ms=250, hotwords_storage_path='', hotwords_score=1.5, timeout=30.0)
Create a server-side VoiceInput session for speech-to-text.
Returns a :class:~tryll_client.voice_input.VoiceInput handle managing the
server-side session. Use as a context manager (recommended) for guaranteed
cleanup.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_name
|
str
|
Catalog name of the STT model, e.g.
|
required |
sample_rate
|
int
|
Sample rate of audio buffers you will push (Hz). |
16000
|
channels
|
int
|
Channel count (1 = mono, typical for mic capture). |
1
|
bits_per_sample
|
int
|
Bit depth of raw PCM samples (only 16-bit is supported). |
16
|
vad_threshold
|
float
|
Silero VAD speech-probability threshold (0.0–1.0). |
0.5
|
vad_min_silence_ms
|
int
|
Silence duration (ms) that closes a speech segment. |
500
|
vad_speech_pad_ms
|
int
|
Padding (ms) added around detected speech. |
250
|
hotwords_storage_path
|
str
|
Relative path (resolved against the session
storage folder) to a |
''
|
hotwords_score
|
float
|
Per-token bias strength applied to every phrase in the storage. 1.5 is a gentle default; 2.5 is aggressive. |
1.5
|
timeout
|
float
|
Maximum seconds to wait for the server response. |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
If the model cannot be found, the hotwords storage file is missing, or the request times out. |
create_agent ¶
create_agent(graph, enable_diagnostics=False, timeout=None, maintain_dialogue_history=True, variables=None, kv_cache_initialization=AgentKvCacheInitialization.AllocateOnly, workload=AgentWorkload.Interactive)
Create a server-side agent and return a proxy handle.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
GraphDescription
|
Fully-built graph description. |
required |
enable_diagnostics
|
bool
|
When True the server serialises per-node execution data into TurnComplete.debug_info for every turn. |
False
|
timeout
|
float | None
|
Maximum seconds to wait for the response. Defaults to 30 s. |
None
|
maintain_dialogue_history
|
bool
|
When True (default) the agent keeps its full dialogue history. When False the history is discarded after each turn — a stateless agent, useful for technical/classification agents that should not store or project prior turns. |
True
|
variables
|
dict[str, Any] | None
|
Declares this agent's per-agent variables — a
|
None
|
workload
|
AgentWorkload | int
|
:class: |
Interactive
|
list_models ¶
Request all models known to the server for the session's engine.
load_model ¶
Explicitly load and pin a model into memory.
The model stays in memory until :meth:unload_model is called,
regardless of whether any agents are using it.
Raises :class:TryllError if the model cannot be resolved or loaded.
unload_model ¶
Unpin a previously pinned model.
If no agents are currently using the model it is freed immediately; otherwise it will be freed when the last agent using it is destroyed.
download_model ¶
Initiate a model download on the server and block until complete.
on_progress is called with (model_name, bytes_downloaded, total_bytes, percent) for each progress frame received. Raises TryllError on failure.
send_debug_command ¶
Send a dev-only debug command to the server.
The server handles this only in non-Production builds with
debug_commands.enabled = true; otherwise it returns a TryllError.
command is a verb (e.g. "crash"); payload is verb-specific
(for crash it is the mode token: nullderef | abort | throw
| stackoverflow | fastfail; empty defaults to nullderef).
Set await_response=False for fatal verbs like crash that kill the
server and never reply — the call sends the frame and returns immediately;
the caller detects success via the ensuing TCP disconnect. When
await_response is True, returns the server's result string.
wait_for_disconnect ¶
Block until the server drops the connection, or timeout elapses.
The reader thread exits when the socket reaches EOF / resets (e.g. the
server process died), so joining it is a reliable disconnect signal.
Returns True if the connection dropped within timeout. Used after a
fatal debug command (crash) whose only success signal is the drop.
ConnectedSession ¶
RAII wrapper that owns a :class:~tryll_client.managed_server.ManagedServer
and a connected :class:TryllClient.
Obtain via :meth:TryllClient.run_and_connect. Use as a context manager
(recommended) to guarantee teardown in the right order — client first, then
server::
with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
session.client.create_session(InferenceEngine.LlamaCpp)
agent = session.client.create_agent(graph)
print(agent.send_message("hi"))
tryll_client.agent¶
tryll_client.agent ¶
AgentProxy — client-side handle for a server-side agent.
An :class:AgentProxy is returned by
:meth:tryll_client.TryllClient.create_agent and represents a single
server-side agent running a pre-built workflow graph. The proxy is
single-session and not thread-safe beyond what the parent
:class:TryllClient provides.
TurnResult
dataclass
¶
Outcome of a turn started with :meth:AgentProxy.send_message_async.
Delivered by resolving the returned :class:concurrent.futures.Future once
TurnComplete arrives. status distinguishes a normal completion from a
cancelled one (:class:~tryll_client.graph.TurnStatus).
ToolResult
dataclass
¶
One model-visible result correlated to a tool-call call_id.
AgentProxy ¶
Handle for one server-side agent created via TryllClient.create_agent().
The proxy exposes the user-facing turn API — :meth:send_message and
:meth:destroy — and caches diagnostics from the last completed turn
on last_* properties.
Bind the proxy to its owning client and server-side agent id.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
'TryllClient'
|
Parent :class: |
required |
agent_id
|
int
|
Server-assigned agent identifier returned in the
|
required |
node_baselines
|
dict[str, 'NodeParamsBase'] | None
|
Deep-copied params per graph node name, captured
at |
None
|
variables
|
dict[str, tuple[int, Any]] | None
|
|
None
|
last_debug_info
property
¶
JSON diagnostics string from the most recent send_message call.
Returns:
| Type | Description |
|---|---|
str
|
Server-produced JSON document attached to |
str
|
when diagnostics were enabled on agent creation; otherwise an |
str
|
empty string. Empty string also before the first turn. |
last_ttft_s
property
¶
Time-to-first-token in seconds for the most recent turn.
Returns:
| Type | Description |
|---|---|
float | None
|
Seconds elapsed from |
float | None
|
streamed |
float | None
|
arrived (e.g. canned-response paths that skip streaming). |
last_answer_chunk_count
property
¶
Number of AnswerText chunks received for the last turn.
Typically one chunk per generated token when streaming; useful for verifying the stream was delivered incrementally.
last_tokens_generated
property
¶
Server-reported generated token count for the last turn.
Authoritative for both streaming and non-streaming modes. Zero if the server has not yet completed a turn.
set_on_answer_text ¶
Register (or clear) a persistent callback for streaming text chunks.
The callback is invoked on the reader thread for every AnswerText
frame received for this agent — including frames from server-initiated
turns (e.g. voice autosend after VoiceInput.end_utterance).
The callback signature is (node_name: str, text: str, is_delta: bool,
is_final: bool). node_name identifies which node in the workflow
graph produced this text (multi-sender attribution); existing callers
that only need the text can ignore it. Must return quickly and must
not call blocking client methods.
Call with None to unregister the current callback.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cb
|
Callable[[str, str, bool, bool], None] | None
|
Callable invoked with |
required |
set_on_turn_complete ¶
Register (or clear) a persistent callback for turn-complete notifications.
The callback is invoked on the reader thread once per TurnComplete
frame for this agent. Covers both client-initiated turns (via
:meth:send_message) and server-initiated turns (voice autosend).
The callback signature is
(status: int, debug_info: str, tokens_generated: int)
where status maps to the wire TurnStatus enum
(0 = Ok, 1 = Error, 2 = Cancelled).
Call with None to unregister the current callback.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cb
|
Callable | None
|
Callable invoked with |
required |
set_on_tool_call ¶
Register (or clear) a callback for tool-call notifications.
The callback is invoked on the reader thread for every
NodeEvent frame with event_type == "tool_call" the server
sends for this agent — i.e. when the graph has a ToolCall node
with a notifying disposition (Notify / NotifyAndAcknowledge /
Pause / PauseAndAcknowledge / AwaitResult). It must return
quickly and must not call any blocking :class:TryllClient or
:class:AgentProxy methods.
The callback signature is (tool_name: str, arguments_json: str)
where arguments_json is a compact JSON object, e.g.
'{"city": "Berlin"}'. Parse it with :func:json.loads as
needed.
Call with None to unregister the current callback.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cb
|
Callable[[str, str], None] | None
|
Callable invoked with |
required |
set_on_tool_call_with_id ¶
Register a call-ID-aware tool-call callback.
The callback receives (call_id, tool_name, arguments_json) on the
reader thread. Echo call_id in a :class:ToolResult passed through
the keyword-only tool_results argument of :meth:resume_async
(preferred from this callback) or :meth:resume (only from a
non-reader thread). The legacy :meth:set_on_tool_call callback
remains supported.
set_on_error ¶
Register (or clear) a per-agent client-originated error callback.
Fired on the reader thread when a user callback (tool_call, paused,
etc.) raises. Does not replace server ErrorResponse delivery for
pending requests. The error callback itself must not raise; secondary
failures are swallowed to protect the reader.
set_on_intent_classified ¶
Register (or clear) a callback for intent-classification notifications.
Fired on the reader thread for every NodeEvent with
event_type == "intent_classified" — emitted by
ClassifyIntentNode on its "found" path when
notify_client = "true".
The callback signature is
(intent: str, record_id: str, record_index: int, distance: float).
Call with None to unregister.
set_on_paused ¶
Register (or clear) a callback for pause notifications.
Fired on the reader thread for every NodeEvent with
event_type == "paused" — emitted by the executor when it pauses
the turn between nodes (see the Pause node and pausing
ToolCallDisposition values). While paused, the agent still
counts as busy for :meth:send_message, but
:meth:change_params / :meth:change_param are allowed.
The callback signature is (node_name: str, pending_exit: str)
where pending_exit is the exit route that will be taken on a
plain (non-jump) :meth:resume.
Call with None to unregister.
set_on_tts_audio_format ¶
Register (or clear) a callback for the TTS audio format frame.
Emitted by the server exactly once per turn that produces TTS audio,
before the first :meth:set_on_tts_audio callback. The callback
signature is (node_name: str, sample_rate: int, channels: int,
bits_per_sample: int) — node_name is the producing node's name
(multi-sender attribution; empty string if the server omitted it).
Fired on the reader thread — must return quickly and must not call
blocking client methods.
Call with None to unregister.
set_on_tts_audio ¶
Register (or clear) a callback for streaming TTS audio chunks.
Fired on the reader thread for every TtsAudioFrame received for
this agent. The callback signature is (node_name: str, pcm:
memoryview) — node_name is the producing node's name
(multi-sender attribution; matches the preceding
:meth:set_on_tts_audio_format call for the same producer). The
pcm argument is a zero-copy memoryview over int16 LE PCM at
the format declared by the preceding format callback. The view is
valid only during the call — copy with bytes(pcm) or
np.frombuffer(pcm, dtype=np.int16).copy() if needed.
TtsAudioFrame carries no is_final flag — end-of-stream is
signalled by TurnComplete. Wire :meth:set_on_turn_complete
(or close your sink in the surrounding with-block) to flush.
Call with None to unregister.
set_on_node_event ¶
Register (or clear) a generic NodeEvent fallback callback.
Fired on the reader thread for any NodeEvent whose
event_type is unrecognised or whose typed callback (e.g.
:meth:set_on_tool_call, :meth:set_on_intent_classified) is
not registered. When a typed callback handles the event, this
one is NOT invoked for that event.
The callback signature is
(node_name: str, event_type: str, kv_pairs: list[tuple[str, str]]).
Call with None to unregister.
send_message ¶
Send a user message and return the complete assistant response.
Blocks until TurnComplete is received from the server,
accumulating all AnswerText chunks into a single string.
Diagnostics from TurnComplete (debug_info,
tokens_generated) are cached on the last_* properties.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
User-turn text to send to the agent. |
required |
timeout
|
float
|
Maximum seconds to wait for |
120.0
|
Returns:
| Type | Description |
|---|---|
str
|
The full concatenated assistant response text. |
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors, decode failures, or
if |
send_message_async ¶
Send a user message without blocking; return a Future for the result.
Unlike :meth:send_message, this returns immediately so the caller can act
mid-turn (e.g. watch streaming callbacks and call :meth:cancel). The
Future resolves with a :class:TurnResult on TurnComplete (including a
cancelled turn, where status == TurnStatus.Cancelled) or raises
:class:~tryll_client.errors.TryllError on a server error.
Await it via fut.result(timeout) (raises concurrent.futures.TimeoutError
if the turn does not finish in time). Streaming chunks still arrive via the
set_on_answer_text / set_on_tts_audio callbacks.
Note: use :meth:cancel to cancel the turn — do not call fut.cancel(),
which only cancels a scheduled call, not the server-side turn.
change_param ¶
Apply a single-field mutation using a dotted attribute path.
Clones the baseline params captured at create_agent, coerces
value to the leaf field type (when value is a string), assigns
via :func:~tryll_client._generated.node_params_codec.set_dotted,
and sends the full params object. On success the baseline is updated
so later mutations compose.
Example::
agent.change_param("generate", "sampling.temperature", "0.5")
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node_name
|
str
|
Instance name of the target node. |
required |
path
|
str
|
Dotted path to the field (e.g. |
required |
value
|
Any
|
New value (string from JSON/dialog scripts, or a native Python value). |
required |
timeout
|
float
|
Maximum seconds to wait for the server |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
If node_name is unknown (code 3005) or the server reports another error. |
change_params ¶
Apply a typed node-parameter update to a workflow node at runtime.
The typed params object must match the concrete type of the target node.
Structural fields must match the create-time values or the server
returns ParamNotMutable. On success, updates the internal baseline
used by :meth:change_param.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
node_name
|
str
|
Instance name of the target node. |
required |
params
|
'NodeParamsBase'
|
Typed node parameters (generated DTO). |
required |
timeout
|
float
|
Maximum seconds to wait for the server |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors ( |
destroy ¶
Request agent destruction on the server and wait for Ack.
After this call returns, the proxy is no longer usable; further
send_message calls will raise :class:TryllError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
timeout
|
float
|
Maximum seconds to wait for the |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors or timeout. |
cancel ¶
Cooperatively cancel the agent's in-flight turn and wait for Ack.
The active send_message ends with TurnStatus.Cancelled. Call this
from a different thread than the blocking send_message to interrupt a
turn mid-flight. No-op server-side if the agent is idle.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
mode
|
int
|
:class: |
0
|
timeout
|
float
|
Maximum seconds to wait for the |
30.0
|
Raises:
| Type | Description |
|---|---|
TryllError
|
On server-reported errors or timeout. |
resume ¶
Resume a turn that is currently paused and wait for Ack.
Not safe on the reader thread (tool-call / paused callbacks):
blocking here deadlocks the sole frame reader. Use
:meth:resume_async from those callbacks instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
resume_node
|
str
|
Empty continues via the paused node's pending exit route; non-empty jumps to that node by name (must exist in the agent's graph). |
''
|
timeout
|
float
|
Maximum seconds to wait for the |
30.0
|
tool_results
|
Sequence[ToolResult | tuple[str, str]]
|
Complete result batch for the paused tool-call pass.
Each result must echo the |
()
|
Raises:
| Type | Description |
|---|---|
TryllError
|
|
resume_async ¶
Resume a paused turn without blocking; returns a Future for the Ack.
Safe to call from reader-thread callbacks. The Future resolves to
None on success or raises :class:TryllError on failure.
prefill_kv_cache ¶
Synchronise eligible language-model contexts to the reusable prefix.
evict_kv_cache ¶
Release eligible raw contexts while retaining agent state.
get_kv_cache_status ¶
Return aggregate residency and reusable-prefix state.
append_interactions ¶
Append scripted user/assistant interactions without running the graph.
Each item is (user_message, assistant_message). Empty / None
strings omit that side; both empty → skipped. Idle-only
(AgentBusy 3004 while running/paused).
Returns:
| Type | Description |
|---|---|
int
|
Number of interactions actually appended. |
remove_interactions_from_end ¶
Remove whole interactions from the end of the dialog.
count == 0 is a no-op; count >= size clears all. Idle-only.
Returns:
| Type | Description |
|---|---|
int
|
Number of interactions actually removed. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If count is negative. |
TryllError
|
On server-reported errors ( |
tryll_client.graph¶
tryll_client.graph ¶
GraphDescription builder and associated enums / dataclasses.
Mirrors the C++ Tryll::Client::GraphDescription / InferenceEngine
/ NodeType types. Enum values must stay in sync with
server/schema/messages.fbs.
This module exposes:
- :class:
GraphDescription— fluent builder for a workflow graph sent to the server onCreateAgent. - The enums used by graph descriptions: :class:
InferenceEngine, :class:ModelStatus. - The typed node-parameter DTOs from
tryll_client._generated.node_params.
NodeParamsBase ¶
Marker base for generated node-parameter DTOs.
SendAnswer ¶
Bases: IntEnum
Controls answer-text delivery to the client. Replaces the old stream: bool.
HistoryRole ¶
Bases: IntEnum
Controls whether/how a producer's slot output is replayed in later turns.
RetrievalMode ¶
Bases: IntEnum
RetrieveNode search path: dense HNSW, BM25 lexical, or RRF hybrid.
BranchTest ¶
Bases: IntEnum
Predicate policy for BranchNode.
NodeType ¶
Bases: IntEnum
Node type ordinal. Mirrors the NodeParams union order in nodes.fbs (0-based; the wire discriminator is this value + 1).
The union discriminant in NodeParams supersedes this for the wire format;
NodeType is kept for client-side ergonomics and generated convenience APIs.
InferenceEngine ¶
Bases: IntEnum
Inference backend. Mirrors the FlatBuffers InferenceEngine enum.
ModelStatus ¶
Bases: IntEnum
Status of a model as reported by the server.
ModelInfo
dataclass
¶
Summary information for one model returned by list_models().
StringStorageKind ¶
Bases: IntEnum
Kind of a StringStorage. Mirrors StringStorageKind in messages.fbs.
CancelMode ¶
Bases: IntEnum
How a mid-turn cancel treats the in-flight interaction. Mirrors CancelMode in messages.fbs.
TurnStatus ¶
Bases: IntEnum
Outcome of a workflow turn, as reported in TurnComplete. Mirrors TurnStatus in messages.fbs.
NodeDesc
dataclass
¶
GraphDescription ¶
Fluent builder for a workflow graph sent to the server.
Typical usage::
graph = (GraphDescription()
.add_generate("gen", GenerateParams(
model_name="qwen2.5-0.5b-instruct",
system_prompt="You are helpful."))
.set_start_node("gen"))
agent = await client.create_agent(graph)
tryll_client.variables¶
AgentProxy.variables — a mapping-like typed store; see
Agent Variables for the full model.
tryll_client.variables ¶
AgentVariables — per-agent typed variable mirror with deferred-flush staging.
Mirrors the store declared at create_agent(variables=...) time: a mapping-
like local cache seeded from the declaration, pre-validated locally (unknown
name / type mismatch raise immediately — no wire round-trip), with mutations
coalesced per name and flushed as one batched UpdateAgentVariablesRequest
immediately before the agent's next send_message/send_message_async/
resume/change_param/change_params call (never before cancel).
Staged writes are coalesced per name (last value wins for scalars; a minimal
add/remove delta, or a whole-set assign, whichever is smaller, for sets) so
rapid repeated writes never grow the batch unboundedly. A rejected flush is
recovered rather than silently dropped: a transient AgentBusy re-stages the
batch behind any newer writes; a hard rejection re-syncs the affected names from
the server so the optimistic mirror never stays diverged. Either way the error
propagates to the caller.
Same shape as the C++ client's Tryll::Client::AgentVariables — see
tryll/clients/cpp/include/tryll/AgentVariables.h.
VariableDecl
dataclass
¶
One agent-variable value plus immutable declaration metadata.
AgentVariables ¶
Mapping-like local mirror of one agent's declared per-agent variables.
Seed the mirror from the declaration passed to create_agent.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
'TryllClient'
|
Parent :class: |
required |
agent_id
|
int
|
Server-assigned agent id this store belongs to. |
required |
declared
|
Optional[dict[str, tuple[int, Any]]]
|
|
None
|
set ¶
Stage an Assign — value's inferred type must match the declaration.
A plain Python int is accepted for a variable declared float
(matches the filter grammar's int-to-float promotion); every other
type mismatch raises immediately, before anything is queued for the
wire.
add_to_set ¶
Stage an idempotent AddToSet — name must be a declared string set.
remove_from_set ¶
Stage an idempotent RemoveFromSet — name must be a declared string set.
stage_batch ¶
Stage a batch atomically: validate+stage every op, or roll back all.
ops is a list of (op_name, name, value) where op_name is one
of assign / add_to_set / remove_from_set / reset (the
value is ignored for reset). On the first invalid op the mirror
and staged queue are restored to their pre-call state and the error is
re-raised, so a rejected batch leaves nothing dirty — mirroring the
server's own validate-all-then-apply atomicity.
flush_if_dirty ¶
Send every staged mutation as one batched UpdateAgentVariablesRequest.
Called automatically at the top of send_message,
send_message_async, resume, change_param, and
change_params — deliberately not before cancel. No-op when
nothing is staged.
The batch is retained in-flight until acknowledged. On a transient
AgentBusy it is re-staged behind any newer writes and the error is
re-raised; on a hard rejection the affected names are re-synced from the
server before re-raising, so a rejected batch never leaves the mirror
silently diverged.
sync_from_server ¶
Refresh the local mirror from the server's authoritative current values.
Names with a locally newer (staged or in-flight) write are skipped, so a server snapshot never clobbers an unsent local change.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
names
|
Optional[list[str]]
|
Variable names to fetch; empty/ |
None
|
infer_variable_type ¶
Infer the VARIABLE_VALUE_* wire tag for a native Python value.
bool must be checked before int since bool is an int
subclass in Python.
tryll_client.kv_cache¶
AgentProxy.prefill_kv_cache(), evict_kv_cache(), and
get_kv_cache_status() operate on the aggregate lifecycle state. Create an
agent with kv_cache_initialization=AgentKvCacheInitialization.Prefill or
DeferAllocation when needed. Sends restore evicted contexts automatically.
See Manage an Agent's KV Cache.
tryll_client.kv_cache ¶
KV-cache lifecycle types for server-side agents.
Dialog mutation¶
AgentProxy.append_interactions() and AgentProxy.remove_interactions_from_end()
append or tail-remove scripted history without running the graph. Both are
strict idle-only (AgentBusy 3004 while running, paused, or during a
KV-cache lifecycle operation). Return applied/removed counts; no TurnComplete.
Each append item is (user_message, assistant_message) — None or empty strings
omit that side; both empty → skipped. See
Seed and edit dialog history.
tryll_client.throttle¶
TryllClient.set_inference_throttle(level) reports how hard the server should
yield the GPU back to your game — 0.0 = full speed (the default for a session
that never calls it), 1.0 = maximum yielding. Fire-and-forget: no response, no
error surfaced, safe to call every frame. It never changes what is generated,
only how fast.
create_agent(..., workload=AgentWorkload.Background) marks an agent as
background logic, so its inference gives way first under pressure. Everything
else the server classifies from the graph itself, per turn. See
Wire Protocol
and Server configuration.
tryll_client.throttle ¶
Manual inference throttle types for server-side agents and sessions.
AgentWorkload ¶
Bases: IntEnum
Whether a human ever waits on an agent's turns.
Purely a scheduling hint for the inference budget controller — it never changes node semantics. Set Background for world-logic/evaluator agents so their work is paced or deferred first under GPU pressure.
tryll_client.errors¶
tryll_client.errors ¶
Exception type for Tryll client errors.
All failures surfaced by the synchronous client — server-reported errors,
protocol-level decode failures, and timeouts — are raised as
:class:TryllError. Server-reported errors carry a numeric code that
matches the ranges defined in server/common/include/tryll/ErrorCodes.h;
client-originated errors (timeouts, closed socket) carry code == 0.
TryllError ¶
Bases: Exception
Raised on server errors, protocol errors, or timeouts.
Attributes:
| Name | Type | Description |
|---|---|---|
code |
Numeric error code from the server's |
Initialise a Tryll client error.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
Human-readable description of the failure, typically
copied from the server's |
required |
code
|
int
|
Numeric error code from |
0
|
tryll_client.managed_server¶
tryll_client.managed_server ¶
managed_server — spawn a tryll_server child process and wait for TCP readiness.
Typical usage::
from pathlib import Path
from tryll_client import TryllClient, ManagedServer
with ManagedServer.start(exe=Path("C:/tryll/tryll_server.exe"), port=9100) as srv:
client = TryllClient.connect(srv.host, srv.port)
client.create_session(...)
# … use client …
client.shutdown()
No executable discovery is performed here. Pass the path to the server
executable explicitly (or use the helpers in qa-and-eval/tryll_qa to
locate it).
ServerProcessExitedError ¶
Bases: RuntimeError
Raised by :func:wait_for_tcp when the watched process exits before its
TCP port ever opens.
exit_code is the process's exit code; port_conflict is True when
it equals :data:_SERVER_EXIT_PORT_IN_USE, so callers can special-case a
bind collision (retry on a fresh port) versus any other early exit
(report the real failure instead of a generic timeout).
ServerProcess ¶
Bases: NamedTuple
Return value of :func:start_server_process.
proc is the spawned child. port is the port it was actually told to
bind — when the caller left port at its default, :func:start_server_process
auto-allocates a free ephemeral one, so this is the only way a direct
caller can learn which port that turned out to be (the process was
launched with --port <port>, not the caller's original default).
ManagedServer ¶
ManagedServer(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, idle_shutdown_timeout=_DEFAULT_IDLE_SHUTDOWN_TIMEOUT)
RAII handle around a child tryll_server process.
Use as a context manager (recommended), or call :meth:stop explicitly::
with ManagedServer.start(exe=Path("..."), port=9100) as srv:
client = TryllClient.connect(srv.host, srv.port)
...
No executable discovery is performed — pass the path explicitly.
The server is always launched with --port <port> so the caller's port
takes precedence over whatever is in server-config.json, and (unless
idle_shutdown_timeout is 0) with --idle-shutdown-timeout <N> so it
self-exits once no session is using it — :meth:stop releases our handle
but does not kill the process.
start
classmethod
¶
start(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, idle_shutdown_timeout=_DEFAULT_IDLE_SHUTDOWN_TIMEOUT)
Spawn the server and block until its TCP port is ready.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exe
|
Path | str
|
Path to |
required |
host
|
str
|
Host string for the TCP ready-probe (default |
'127.0.0.1'
|
port
|
int
|
TCP port; passed as |
9100
|
cwd
|
Path | None
|
Working directory for the child process (defaults to
|
None
|
extra_args
|
Sequence[str]
|
Additional CLI arguments appended after |
()
|
stdout
|
Path | None
|
File path for child stdout redirection ( |
None
|
stderr
|
Path | None
|
File path for child stderr redirection ( |
None
|
start_timeout
|
float
|
Seconds to wait for the TCP port to open (default 30). |
30.0
|
stop_timeout
|
float
|
Seconds to wait for clean exit on :meth: |
8.0
|
idle_shutdown_timeout
|
int
|
Seconds of idle (zero sessions, no in-flight
downloads) after which the server self-exits; passed as
|
_DEFAULT_IDLE_SHUTDOWN_TIMEOUT
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
'ManagedServer'
|
class: |
Raises:
| Type | Description |
|---|---|
FileNotFoundError
|
If exe does not exist. |
TimeoutError
|
If the port does not open within start_timeout. |
OSError
|
If the process cannot be spawned. |
stop ¶
Release the process handle without killing the server.
The server self-exits via its idle-shutdown timer (managed mode).
Callers that need an immediate, guaranteed stop (e.g. spawn succeeded
but connect/handshake failed, so no session was ever created and
nothing else can be relying on this process) should call
:meth:terminate instead. The QA harness uses
:func:stop_server_process directly for the same reason.
terminate ¶
Immediately hard-kill the child process, if still running.
Idempotent — safe to call more than once, including after :meth:stop.
Intended for failure paths where the caller knows this process is
theirs and nothing else can depend on it. Do not call this after a
successful handshake, since another consumer could be relying on the
server staying up until its own idle timeout.
wait_for_tcp ¶
Block until host:port accepts a TCP connection or the wait expires.
If proc is given, it is polled each iteration; a process that has
already exited raises :class:ServerProcessExitedError immediately
instead of waiting (e.g. a port collision, exit code 3 — see
:data:_SERVER_EXIT_PORT_IN_USE). While the process is still alive the
wait is extended to :data:_ALIVE_HARD_CAP_SEC (a slow-but-alive cold
start is tolerated rather than treated as fatal); timeout_sec is the
floor and, when no proc is given, the only bound.
Raises :class:TimeoutError if the port does not open before the effective
cap and the process (if given) is still running.
connection_ready_identity ¶
Extract ConnectionReady identity fields from a connected TryllClient.
wait_for_connection_ready ¶
wait_for_connection_ready(host, port, timeout_sec, proc=None, connect_timeout=5.0, poll_interval=0.25)
Block until the server delivers ConnectionReady on host:port.
Stronger than :func:wait_for_tcp: readiness means the application-level
ConnectionReady hello was actually received, not merely that the TCP
port accepts. This closes the cold-start gap where the port accepts a
connection well before the server has finished starting and can service it
(the "ConnectionReady-late" failure mode) — a TCP-only check would declare
"ready" prematurely.
Like :func:wait_for_tcp, it keeps trying while proc (if given) is alive,
bounded by :data:_ALIVE_HARD_CAP_SEC, and short-circuits with
:class:ServerProcessExitedError the instant the child exits. Raises
:class:TimeoutError if the cap elapses while the child is still alive.
Returns the ConnectionReady identity dict (exe_path, build_config, version, run_id, managed, idle_timeout_seconds, protocol_version, codegen_fingerprint) captured from the successful probe connection.
start_server_process ¶
start_server_process(exe, port=_DEFAULT_PORT, cwd=None, env=None, log_dir=None, extra_args=(), host='127.0.0.1', idle_shutdown_timeout=_DEFAULT_IDLE_SHUTDOWN_TIMEOUT)
Start exe as a child process, passing --port <port> on the command line.
cwd defaults to the executable's directory (where data/ usually lives).
log_dir is the directory where server_stdout.txt and
server_stderr.txt are written. Defaults to the executable's directory.
Pass Path(os.devnull) to suppress output entirely.
extra_args is appended to the command line after --port <port>; use it
to pass server-startup flags such as --disable-telemetry.
When port is the default (9100) the function allocates a free ephemeral port automatically so every launcher gets its own isolated port. Pass an explicit port to override.
idle_shutdown_timeout is passed as --idle-shutdown-timeout <N> (default
60s) so the server self-exits once no session is using it — nothing calling
this function kills the process on teardown. Pass 0 for a standalone server
that should run until explicitly stopped (e.g. via :func:stop_server_process),
or if extra_args already includes the flag.
Returns:
| Name | Type | Description |
|---|---|---|
A |
ServerProcess
|
class: |
ServerProcess
|
was actually launched on — read |
|
ServerProcess
|
caller's port argument, since it may have been auto-allocated. |
stop_server_process ¶
Terminate proc; kill if it does not exit within terminate_timeout.
server_crash_info ¶
Classify whether proc crashed, based on whether it exited on its own.
The robust signal is "did the process exit before we asked it to stop":
poll()isNone— still running; the caller willterminate()it, which on Windows yields a nonzero exit code that must NOT be mistaken for a crash. Returns(False, None).- exited with code
0— clean self-exit (unusual for the managed server). Returns(False, 0). - exited with a nonzero code — crashed (e.g.
0xC0000005access violation). Returns(True, code).
Call this before :func:stop_server_process.