Python Client API Reference¶

Full API reference for the tryll_client Python package, auto-generated from Google-style docstrings. The package lives at server/client-python/tryll_client/.

The public surface — re-exported from tryll_client.__init__ — is: TryllClient, ConnectedSession, AgentProxy, TryllError, GraphDescription, InferenceEngine, NodeType, ManagedServer, and the knowledge-presentation config types. ConnectedSession is the recommended entry point for local-server deployments; obtain it via TryllClient.run_and_connect. Internal modules (wire, codec, _generated) are excluded.

tryll_client¶

tryll_client ¶

tryll_client — Python client library for the Tryll server.

Quick start::

from tryll_client import TryllClient, InferenceEngine, NodeType, GraphDescription

client = TryllClient.connect("127.0.0.1", 9100)
client.configure_session(InferenceEngine.LlamaCpp)

graph = (GraphDescription()
    .add_node("generate", NodeType.Generate, {
        "system_prompt": "You are a helpful assistant.",
        "stream": "true",
        "temperature": "0",
        "seed": "42",
    })
    .wire("generate", "default", "END")
    .set_start_node("generate")
    .set_default_model_name("Llama 3.2 3B Instruct (Q4_K_M)"))

agent = client.create_agent(graph)
response = agent.send_message("Hello!")
agent.destroy()
client.shutdown()

Prerequisites

The CMake build for server/ must have run at least once to generate FlatBuffers Python code into tryll_client/_generated/. Run: cmake --build server/build

AgentProxy ¶

AgentProxy(client, agent_id)

Handle for one server-side agent created via TryllClient.create_agent().

The proxy exposes the user-facing turn API — :meth:send_message and :meth:destroy — and caches diagnostics from the last completed turn on last_* properties.

Bind the proxy to its owning client and server-side agent id.

Parameters:

Name	Type	Description	Default
`client`	`'TryllClient'`	Parent :class:`TryllClient` used to send and receive wire messages on behalf of this agent.	required
`agent_id`	`int`	Server-assigned agent identifier returned in the `CreateAgentResponse`.	required

agent_id `property` ¶

agent_id

Server-assigned agent identifier for this proxy.

last_debug_info `property` ¶

last_debug_info

JSON diagnostics string from the most recent send_message call.

Returns:

Type	Description
`str`	Server-produced JSON document attached to `TurnComplete`
`str`	when diagnostics were enabled on agent creation; otherwise an
`str`	empty string. Empty string also before the first turn.

last_ttft_s `property` ¶

last_ttft_s

Time-to-first-token in seconds for the most recent turn.

Returns:

Type	Description
`float \| None`	Seconds elapsed from `send_message` invocation to the first
`float \| None`	streamed `AnswerText` chunk, or `None` if no chunks
`float \| None`	arrived (e.g. canned-response paths that skip streaming).

last_answer_chunk_count `property` ¶

last_answer_chunk_count

Number of AnswerText chunks received for the last turn.

Typically one chunk per generated token when streaming; useful for verifying the stream was delivered incrementally.

last_tokens_generated `property` ¶

last_tokens_generated

Server-reported generated token count for the last turn.

Authoritative for both streaming and non-streaming modes. Zero if the server has not yet completed a turn.

set_on_tool_call ¶

set_on_tool_call(cb)

Register (or clear) a callback for tool-call notifications.

The callback is invoked on the reader thread for every ToolCallNotification frame the server sends for this agent — i.e. when the graph has a ToolCall node with notify_client = "true". It must return quickly and must not call any blocking :class:TryllClient or :class:AgentProxy methods.

The callback signature is (tool_name: str, arguments_json: str) where arguments_json is a compact JSON object, e.g. '{"city": "Berlin"}'. Parse it with :func:json.loads as needed.

Call with None to unregister the current callback.

Parameters:

Name	Type	Description	Default
`cb`	`Callable[[str, str], None] \| None`	Callable invoked with `(tool_name, arguments_json)`, or `None` to clear.	required

send_message ¶

send_message(text, timeout=120.0)

Send a user message and return the complete assistant response.

Blocks until TurnComplete is received from the server, accumulating all AnswerText chunks into a single string. Diagnostics from TurnComplete (debug_info, tokens_generated) are cached on the last_* properties.

Parameters:

Name	Type	Description	Default
`text`	`str`	User-turn text to send to the agent.	required
`timeout`	`float`	Maximum seconds to wait for `TurnComplete`.	`120.0`

Returns:

Type	Description
`str`	The full concatenated assistant response text.

Raises:

Type	Description
`TryllError`	On server-reported errors, decode failures, or if `timeout` elapses before `TurnComplete` arrives.

change_param ¶

change_param(node_name, param_name, param_value, timeout=30.0)

Mutate a named parameter on a workflow node at runtime.

The agent must not be processing a turn when this is called. The change takes effect immediately for all subsequent turns. For system_prompt, the KV cache is lazily invalidated on the next :meth:send_message call.

Parameters:

Name	Type	Description	Default
`node_name`	`str`	Instance name of the target node (as declared in the graph).	required
`param_name`	`str`	Parameter key to set (e.g. `"temperature"`, `"system_prompt"`).	required
`param_value`	`str`	New value as a string.	required
`timeout`	`float`	Maximum seconds to wait for the server `Ack`.	`30.0`

Raises:

Type	Description
`TryllError`	On server-reported errors (`AgentBusy` 3004, `UnknownNode` 3005, `ParamNotMutable` 3006, `InvalidParamValue` 3007) or timeout.

destroy ¶

destroy(timeout=30.0)

Request agent destruction on the server and wait for Ack.

After this call returns, the proxy is no longer usable; further send_message calls will raise :class:TryllError.

Parameters:

Name	Type	Description	Default
`timeout`	`float`	Maximum seconds to wait for the `Ack` response.	`30.0`

Raises:

Type	Description
`TryllError`	On server-reported errors or timeout.

ConnectedSession ¶

ConnectedSession(*, server, client)

RAII wrapper that owns a :class:~tryll_client.managed_server.ManagedServer and a connected :class:TryllClient.

Obtain via :meth:TryllClient.run_and_connect. Use as a context manager (recommended) to guarantee teardown in the right order — client first, then server::

with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
    session.client.configure_session(InferenceEngine.LlamaCpp)
    agent = session.client.create_agent(graph)
    print(agent.send_message("hi"))

client `property` ¶

client

The connected :class:TryllClient.

server `property` ¶

server

The running :class:~tryll_client.managed_server.ManagedServer.

shutdown ¶

shutdown()

Shut the client down, then stop the server. Idempotent.

TryllClient ¶

TryllClient(sock)

Synchronous TCP session to the Tryll server.

Usage::

client = TryllClient.connect("127.0.0.1", 9100)
client.configure_session(InferenceEngine.LlamaCpp)
graph = GraphDescription().add_node(...).wire(...).set_start_node(...)
agent = client.create_agent(graph)
response = agent.send_message("Hello")
agent.destroy()
client.shutdown()

connect `classmethod` ¶

connect(host='127.0.0.1', port=9100, timeout=30.0)

Connect to the Tryll server and wait for SessionReady.

run_and_connect `classmethod` ¶

run_and_connect(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, connect_timeout=30.0)

Spawn tryll_server and connect — recommended one-call factory.

Starts a local server process (passing --port <port> on the command line), waits for its TCP port to be ready, then opens a session. The returned :class:ConnectedSession owns both the server process and the client; use it as a context manager to guarantee clean teardown::

with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
    session.client.configure_session(InferenceEngine.LlamaCpp)
    agent = session.client.create_agent(graph)

Parameters:

Name	Type	Description	Default
`exe`	`Path \| str`	Path to `tryll_server[.exe]`. Required.	required
`host`	`str`	Host for TCP probe and :meth:`connect` (default `127.0.0.1`).	`'127.0.0.1'`
`port`	`int`	Port passed as `--port` to the server (default 9100).	`9100`
`cwd`	`Path \| None`	Working directory for the child process (defaults to `exe.parent`).	`None`
`extra_args`	`Sequence[str]`	Additional CLI arguments after `--port <port>`.	`()`
`stdout`	`Path \| None`	File path for child stdout (`None` = discard).	`None`
`stderr`	`Path \| None`	File path for child stderr (`None` = discard).	`None`
`start_timeout`	`float`	Seconds to wait for the TCP port to open (default 30).	`30.0`
`stop_timeout`	`float`	Seconds to wait for graceful server exit (default 8).	`8.0`
`connect_timeout`	`float`	Seconds to wait for :class:`SessionReady` (default 30).	`30.0`

Returns:

Name	Type	Description
`A`	`'ConnectedSession'`	class:`ConnectedSession` whose server is running and client is
	`'ConnectedSession'`	connected.

Raises:

Type	Description
`FileNotFoundError`	If exe does not exist.
`TimeoutError`	If the port does not open within start_timeout.
`TryllError`	If the session handshake fails.

configure_session ¶

configure_session(engine, allow_auto_model_downloading=False, timeout=30.0)

Send ConfigureSessionRequest and wait for response.

Parameters:

Name	Type	Description	Default
`engine`	`InferenceEngine \| int`	Inference backend to use for this session.	required
`allow_auto_model_downloading`	`bool`	When True, :meth:`create_agent` will automatically download any missing models referenced by the graph instead of failing immediately. Intended for development and prototyping — not for production use. The default timeout for :meth:`create_agent` is extended to 30 minutes when this flag is set.	`False`
`timeout`	`float`	Maximum seconds to wait for the server response.	`30.0`

create_string_storage ¶

create_string_storage(name, strings=None, file_path=None, timeout=10.0)

Create a named StringStorage on the server.

Provide either strings (inline list) or file_path (server-side file path). The storage can then be referenced by name in node params via string_storage.

destroy_string_storage ¶

destroy_string_storage(name, timeout=10.0)

Destroy a named StringStorage on the server.

Nodes that already hold the storage are unaffected.

create_embedded_string_storage ¶

create_embedded_string_storage(name, config_path=None, strings=None, embedding_model=None, timeout=None)

Create a named EmbeddedStringStorage on the server.

Path A: supply config_path (server-side *.json). Path B: supply strings (inline list) + embedding_model. Returns an EmbeddedStorageInfo with record_count and embedding_dim.

When allow_auto_model_downloading=True was passed to :meth:configure_session the server may download the embedding model before building the storage. The default timeout is bumped to 30 minutes in that case; otherwise it is 120 seconds.

destroy_embedded_string_storage ¶

destroy_embedded_string_storage(name, timeout=10.0)

Destroy a named EmbeddedStringStorage on the server.

Nodes that already hold the storage are unaffected.

create_agent ¶

create_agent(graph, enable_diagnostics=False, timeout=None)

Create a server-side agent and return a proxy handle.

Parameters:

Name	Type	Description	Default
`graph`	`GraphDescription`	Fully-built graph description.	required
`enable_diagnostics`	`bool`	When True the server serialises per-node execution data into TurnComplete.debug_info for every turn.	`False`
`timeout`	`float \| None`	Maximum seconds to wait for the response. Defaults to 30 s normally, or 30 minutes when :meth:`configure_session` was called with `allow_auto_model_downloading=True` (to accommodate model downloads that may precede agent creation).	`None`

list_models ¶

list_models(timeout=10.0)

Request all models known to the server for the session's engine.

load_model ¶

load_model(model_name, timeout=300.0)

Explicitly load and pin a model into memory.

The model stays in memory until :meth:unload_model is called, regardless of whether any agents are using it. Raises :class:TryllError if the model cannot be resolved or loaded.

unload_model ¶

unload_model(model_name, timeout=30.0)

Unpin a previously pinned model.

If no agents are currently using the model it is freed immediately; otherwise it will be freed when the last agent using it is destroyed.

download_model ¶

download_model(model_name, on_progress=None, timeout=1800.0)

Initiate a model download on the server and block until complete.

on_progress is called with (model_name, bytes_downloaded, total_bytes, percent) for each progress frame received. Raises TryllError on failure.

shutdown ¶

shutdown()

Close the socket and stop the reader thread.

TryllError ¶

TryllError(message, code=0)

Bases: Exception

Raised on server errors, protocol errors, or timeouts.

Attributes:

Name	Type	Description
`code`		Numeric error code from the server's `ErrorCodes.h` ranges (1xxx connection, 2xxx session, 3xxx agent, 4xxx model, 5xxx node, 6xxx graph, 7xxx string storage, 8xxx embedded storage). Zero for client-originated errors such as timeouts or a closed socket.

Initialise a Tryll client error.

Parameters:

Name	Type	Description	Default
`message`	`str`	Human-readable description of the failure, typically copied from the server's `ErrorResponse.message` field or synthesised locally (e.g. "Timeout (30s) waiting…").	required
`code`	`int`	Numeric error code from `ErrorCodes.h` ranges; zero for client-originated errors.	`0`

str ¶

__str__()

Return the message, prefixed with [code] when non-zero.

GraphDescription ¶

GraphDescription()

Fluent builder for a workflow graph sent to the server.

Typical usage::

graph = (GraphDescription()
         .add_node("gen", NodeType.Generate, {"prompt": "..."})
         .set_start_node("gen")
         .set_default_model_name("qwen2.5-0.5b-instruct"))
agent = client.create_agent(graph)

Create an empty graph description with no nodes, wires, or start node.

add_node ¶

add_node(name, node_type, params=None)

Append a node to the graph.

Parameters:

Name	Type	Description	Default
`name`	`str`	Graph-unique node name; used by :meth:`wire` and :meth:`set_start_node` to reference this node.	required
`node_type`	`NodeType \| int`	Node kind; see :class:`NodeType`. Prefer :meth:`add_tool_call_node` when `node_type == ToolCall`.	required
`params`	`dict[str, str] \| None`	Key/value node parameters. All values are sent as strings on the wire.	`None`

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

add_tool_call_node ¶

add_tool_call_node(name, tools, params=None)

Append a ToolCall node with structured tool definitions.

Convenience over :meth:add_node that accepts a list of :class:ToolDef rather than encoding tools into string params.

Parameters:

Name	Type	Description	Default
`name`	`str`	Graph-unique node name.	required
`tools`	`list[ToolDef]`	Tool definitions available to the model at this node.	required
`params`	`dict[str, str] \| None`	Additional key/value node parameters.	`None`

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

wire ¶

wire(source, exit_name, target)

Add a wire (exit route) from one node to another.

Parameters:

Name	Type	Description	Default
`source`	`str`	Name of the node the wire leaves.	required
`exit_name`	`str`	Name of the exit route on `source` (exit names are node-type-specific; see the node reference).	required
`target`	`str`	Name of the node the wire enters.	required

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

set_start_node ¶

set_start_node(name)

Nominate which node receives the turn when the agent runs.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of a node previously added via :meth:`add_node` or :meth:`add_tool_call_node`.	required

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

set_default_model_name ¶

set_default_model_name(name)

Set the default catalog model name for nodes that need one.

Nodes that invoke a language model (Generate, ToolCall) use this value when their own model param is unset.

Parameters:

Name	Type	Description	Default
`name`	`str`	Catalog model name as registered in `models.json`.	required

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

InferenceEngine ¶

Bases: IntEnum

Inference backend that runs the model.

Passed to :meth:TryllClient.configure_session to select the engine for all agents on this session. Mirrors the FlatBuffers InferenceEngine enum.

Mock `class-attribute` `instance-attribute` ¶

Mock = 0

In-process mock engine used by tests. No real inference.

LlamaCpp `class-attribute` `instance-attribute` ¶

LlamaCpp = 1

llama.cpp — GGUF models. The default production engine.

OnnxGenAI `class-attribute` `instance-attribute` ¶

OnnxGenAI = 2

ONNX GenAI — not yet implemented (reserved).

WindowsML `class-attribute` `instance-attribute` ¶

WindowsML = 3

Windows ML — not yet implemented (reserved).

OpenVino `class-attribute` `instance-attribute` ¶

OpenVino = 4

Intel OpenVINO — not yet implemented (reserved).

TensorRtLlm `class-attribute` `instance-attribute` ¶

TensorRtLlm = 5

NVIDIA TensorRT-LLM — not yet implemented (reserved).

ModelInfo `dataclass` ¶

ModelInfo(name, status, hf_repo, size_bytes)

Summary information for one model returned by list_models().

name `instance-attribute` ¶

name

Catalog name of the model (key in models.json).

status `instance-attribute` ¶

status

Current status — see :class:ModelStatus.

hf_repo `instance-attribute` ¶

hf_repo

HuggingFace repository identifier, or empty string for local-only models.

size_bytes `instance-attribute` ¶

size_bytes

Total on-disk size in bytes; zero for Absent models.

ModelStatus ¶

Bases: IntEnum

Status of a model as reported by the server.

Mirrors ModelStatus in messages.fbs.

Absent `class-attribute` `instance-attribute` ¶

Absent = 0

No record of this model; call download_model first.

Local `class-attribute` `instance-attribute` ¶

Local = 1

User-supplied localPath model on disk; ready to load.

Downloading `class-attribute` `instance-attribute` ¶

Downloading = 2

Download in progress from HuggingFace.

Loaded `class-attribute` `instance-attribute` ¶

Loaded = 3

Model files are present and pinned into memory.

Downloaded `class-attribute` `instance-attribute` ¶

Downloaded = 4

Model files are present in the HuggingFace cache; not loaded.

NodeType ¶

Bases: IntEnum

Workflow node kind. Mirrors NodeType in messages.fbs.

Generate `class-attribute` `instance-attribute` ¶

Generate = 0

LLM-driven text generation.

HumanMessageGuardrail `class-attribute` `instance-attribute` ¶

HumanMessageGuardrail = 1

Pattern match on the incoming user message; routes accordingly.

CannedResponse `class-attribute` `instance-attribute` ¶

CannedResponse = 2

Emit a pre-written reply without invoking a model.

ToolCall `class-attribute` `instance-attribute` ¶

ToolCall = 3

Detect and emit a tool-call invocation (small-model-friendly).

Retrieve `class-attribute` `instance-attribute` ¶

Retrieve = 4

RAG: retrieve chunks from an embedded string storage.

Instruction `class-attribute` `instance-attribute` ¶

Instruction = 5

Attach a named instruction string to the current interaction.

ManagedServer ¶

ManagedServer(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0)

RAII handle around a child tryll_server process.

Use as a context manager (recommended), or call :meth:stop explicitly::

with ManagedServer.start(exe=Path("..."), port=9100) as srv:
    client = TryllClient.connect(srv.host, srv.port)
    ...

No executable discovery is performed — pass the path explicitly. The server is always launched with --port <port> so the caller's port takes precedence over whatever is in server-config.json.

host `property` ¶

host

Host string to pass to :func:TryllClient.connect.

port `property` ¶

port

TCP port the server is listening on.

pid `property` ¶

pid

PID of the child process, or -1 if not running.

is_running `property` ¶

is_running

True while the child process is alive.

start `classmethod` ¶

start(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0)

Spawn the server and block until its TCP port is ready.

Parameters:

Name	Type	Description	Default
`exe`	`Path \| str`	Path to `tryll_server[.exe]`. Required.	required
`host`	`str`	Host string for the TCP ready-probe (default `127.0.0.1`).	`'127.0.0.1'`
`port`	`int`	TCP port; passed as `--port` to the server (default 9100).	`9100`
`cwd`	`Path \| None`	Working directory for the child process (defaults to `exe.parent`).	`None`
`extra_args`	`Sequence[str]`	Additional CLI arguments appended after `--port <port>`.	`()`
`stdout`	`Path \| None`	File path for child stdout redirection (`None` = discard).	`None`
`stderr`	`Path \| None`	File path for child stderr redirection (`None` = discard).	`None`
`start_timeout`	`float`	Seconds to wait for the TCP port to open (default 30).	`30.0`
`stop_timeout`	`float`	Seconds to wait for clean exit on :meth:`stop` (default 8).	`8.0`

Returns:

Name	Type	Description
`A`	`'ManagedServer'`	class:`ManagedServer` whose port is accepting connections.

Raises:

Type	Description
`FileNotFoundError`	If exe does not exist.
`TimeoutError`	If the port does not open within start_timeout.
`OSError`	If the process cannot be spawned.

stop ¶

stop(*, timeout=None)

Terminate the child process.

Parameters:

Name	Type	Description	Default
`timeout`	`float \| None`	Seconds to wait before force-killing (defaults to the stop_timeout supplied at construction).	`None`

tryll_client.client¶

tryll_client.client ¶

TryllClient — synchronous TCP client for the Tryll server.

Threading model

A background reader thread continuously receives frames and dispatches them to registered pending requests via threading.Event.
The main (calling) thread sends requests and blocks on the Event.
All access to _pending is protected by _lock.

TryllClient ¶

TryllClient(sock)

Synchronous TCP session to the Tryll server.

Usage::

client = TryllClient.connect("127.0.0.1", 9100)
client.configure_session(InferenceEngine.LlamaCpp)
graph = GraphDescription().add_node(...).wire(...).set_start_node(...)
agent = client.create_agent(graph)
response = agent.send_message("Hello")
agent.destroy()
client.shutdown()

connect `classmethod` ¶

connect(host='127.0.0.1', port=9100, timeout=30.0)

Connect to the Tryll server and wait for SessionReady.

run_and_connect `classmethod` ¶

run_and_connect(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0, connect_timeout=30.0)

Spawn tryll_server and connect — recommended one-call factory.

Starts a local server process (passing --port <port> on the command line), waits for its TCP port to be ready, then opens a session. The returned :class:ConnectedSession owns both the server process and the client; use it as a context manager to guarantee clean teardown::

with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
    session.client.configure_session(InferenceEngine.LlamaCpp)
    agent = session.client.create_agent(graph)

Parameters:

Name	Type	Description	Default
`exe`	`Path \| str`	Path to `tryll_server[.exe]`. Required.	required
`host`	`str`	Host for TCP probe and :meth:`connect` (default `127.0.0.1`).	`'127.0.0.1'`
`port`	`int`	Port passed as `--port` to the server (default 9100).	`9100`
`cwd`	`Path \| None`	Working directory for the child process (defaults to `exe.parent`).	`None`
`extra_args`	`Sequence[str]`	Additional CLI arguments after `--port <port>`.	`()`
`stdout`	`Path \| None`	File path for child stdout (`None` = discard).	`None`
`stderr`	`Path \| None`	File path for child stderr (`None` = discard).	`None`
`start_timeout`	`float`	Seconds to wait for the TCP port to open (default 30).	`30.0`
`stop_timeout`	`float`	Seconds to wait for graceful server exit (default 8).	`8.0`
`connect_timeout`	`float`	Seconds to wait for :class:`SessionReady` (default 30).	`30.0`

Returns:

Name	Type	Description
`A`	`'ConnectedSession'`	class:`ConnectedSession` whose server is running and client is
	`'ConnectedSession'`	connected.

Raises:

Type	Description
`FileNotFoundError`	If exe does not exist.
`TimeoutError`	If the port does not open within start_timeout.
`TryllError`	If the session handshake fails.

configure_session ¶

configure_session(engine, allow_auto_model_downloading=False, timeout=30.0)

Send ConfigureSessionRequest and wait for response.

Parameters:

Name	Type	Description	Default
`engine`	`InferenceEngine \| int`	Inference backend to use for this session.	required
`allow_auto_model_downloading`	`bool`	When True, :meth:`create_agent` will automatically download any missing models referenced by the graph instead of failing immediately. Intended for development and prototyping — not for production use. The default timeout for :meth:`create_agent` is extended to 30 minutes when this flag is set.	`False`
`timeout`	`float`	Maximum seconds to wait for the server response.	`30.0`

create_string_storage ¶

create_string_storage(name, strings=None, file_path=None, timeout=10.0)

Create a named StringStorage on the server.

Provide either strings (inline list) or file_path (server-side file path). The storage can then be referenced by name in node params via string_storage.

destroy_string_storage ¶

destroy_string_storage(name, timeout=10.0)

Destroy a named StringStorage on the server.

Nodes that already hold the storage are unaffected.

create_embedded_string_storage ¶

create_embedded_string_storage(name, config_path=None, strings=None, embedding_model=None, timeout=None)

Create a named EmbeddedStringStorage on the server.

Path A: supply config_path (server-side *.json). Path B: supply strings (inline list) + embedding_model. Returns an EmbeddedStorageInfo with record_count and embedding_dim.

When allow_auto_model_downloading=True was passed to :meth:configure_session the server may download the embedding model before building the storage. The default timeout is bumped to 30 minutes in that case; otherwise it is 120 seconds.

destroy_embedded_string_storage ¶

destroy_embedded_string_storage(name, timeout=10.0)

Destroy a named EmbeddedStringStorage on the server.

Nodes that already hold the storage are unaffected.

create_agent ¶

create_agent(graph, enable_diagnostics=False, timeout=None)

Create a server-side agent and return a proxy handle.

Parameters:

Name	Type	Description	Default
`graph`	`GraphDescription`	Fully-built graph description.	required
`enable_diagnostics`	`bool`	When True the server serialises per-node execution data into TurnComplete.debug_info for every turn.	`False`
`timeout`	`float \| None`	Maximum seconds to wait for the response. Defaults to 30 s normally, or 30 minutes when :meth:`configure_session` was called with `allow_auto_model_downloading=True` (to accommodate model downloads that may precede agent creation).	`None`

list_models ¶

list_models(timeout=10.0)

Request all models known to the server for the session's engine.

load_model ¶

load_model(model_name, timeout=300.0)

Explicitly load and pin a model into memory.

The model stays in memory until :meth:unload_model is called, regardless of whether any agents are using it. Raises :class:TryllError if the model cannot be resolved or loaded.

unload_model ¶

unload_model(model_name, timeout=30.0)

Unpin a previously pinned model.

If no agents are currently using the model it is freed immediately; otherwise it will be freed when the last agent using it is destroyed.

download_model ¶

download_model(model_name, on_progress=None, timeout=1800.0)

Initiate a model download on the server and block until complete.

on_progress is called with (model_name, bytes_downloaded, total_bytes, percent) for each progress frame received. Raises TryllError on failure.

shutdown ¶

shutdown()

Close the socket and stop the reader thread.

ConnectedSession ¶

ConnectedSession(*, server, client)

RAII wrapper that owns a :class:~tryll_client.managed_server.ManagedServer and a connected :class:TryllClient.

Obtain via :meth:TryllClient.run_and_connect. Use as a context manager (recommended) to guarantee teardown in the right order — client first, then server::

with TryllClient.run_and_connect(exe=Path("..."), port=9100) as session:
    session.client.configure_session(InferenceEngine.LlamaCpp)
    agent = session.client.create_agent(graph)
    print(agent.send_message("hi"))

client `property` ¶

client

The connected :class:TryllClient.

server `property` ¶

server

The running :class:~tryll_client.managed_server.ManagedServer.

shutdown ¶

shutdown()

Shut the client down, then stop the server. Idempotent.

tryll_client.agent¶

tryll_client.agent ¶

AgentProxy — client-side handle for a server-side agent.

An :class:AgentProxy is returned by :meth:tryll_client.TryllClient.create_agent and represents a single server-side agent running a pre-built workflow graph. The proxy is single-session and not thread-safe beyond what the parent :class:TryllClient provides.

AgentProxy ¶

AgentProxy(client, agent_id)

Handle for one server-side agent created via TryllClient.create_agent().

The proxy exposes the user-facing turn API — :meth:send_message and :meth:destroy — and caches diagnostics from the last completed turn on last_* properties.

Bind the proxy to its owning client and server-side agent id.

Parameters:

Name	Type	Description	Default
`client`	`'TryllClient'`	Parent :class:`TryllClient` used to send and receive wire messages on behalf of this agent.	required
`agent_id`	`int`	Server-assigned agent identifier returned in the `CreateAgentResponse`.	required

agent_id `property` ¶

agent_id

Server-assigned agent identifier for this proxy.

last_debug_info `property` ¶

last_debug_info

JSON diagnostics string from the most recent send_message call.

Returns:

Type	Description
`str`	Server-produced JSON document attached to `TurnComplete`
`str`	when diagnostics were enabled on agent creation; otherwise an
`str`	empty string. Empty string also before the first turn.

last_ttft_s `property` ¶

last_ttft_s

Time-to-first-token in seconds for the most recent turn.

Returns:

Type	Description
`float \| None`	Seconds elapsed from `send_message` invocation to the first
`float \| None`	streamed `AnswerText` chunk, or `None` if no chunks
`float \| None`	arrived (e.g. canned-response paths that skip streaming).

last_answer_chunk_count `property` ¶

last_answer_chunk_count

Number of AnswerText chunks received for the last turn.

Typically one chunk per generated token when streaming; useful for verifying the stream was delivered incrementally.

last_tokens_generated `property` ¶

last_tokens_generated

Server-reported generated token count for the last turn.

Authoritative for both streaming and non-streaming modes. Zero if the server has not yet completed a turn.

set_on_tool_call ¶

set_on_tool_call(cb)

Register (or clear) a callback for tool-call notifications.

The callback is invoked on the reader thread for every ToolCallNotification frame the server sends for this agent — i.e. when the graph has a ToolCall node with notify_client = "true". It must return quickly and must not call any blocking :class:TryllClient or :class:AgentProxy methods.

The callback signature is (tool_name: str, arguments_json: str) where arguments_json is a compact JSON object, e.g. '{"city": "Berlin"}'. Parse it with :func:json.loads as needed.

Call with None to unregister the current callback.

Parameters:

Name	Type	Description	Default
`cb`	`Callable[[str, str], None] \| None`	Callable invoked with `(tool_name, arguments_json)`, or `None` to clear.	required

send_message ¶

send_message(text, timeout=120.0)

Send a user message and return the complete assistant response.

Blocks until TurnComplete is received from the server, accumulating all AnswerText chunks into a single string. Diagnostics from TurnComplete (debug_info, tokens_generated) are cached on the last_* properties.

Parameters:

Name	Type	Description	Default
`text`	`str`	User-turn text to send to the agent.	required
`timeout`	`float`	Maximum seconds to wait for `TurnComplete`.	`120.0`

Returns:

Type	Description
`str`	The full concatenated assistant response text.

Raises:

Type	Description
`TryllError`	On server-reported errors, decode failures, or if `timeout` elapses before `TurnComplete` arrives.

change_param ¶

change_param(node_name, param_name, param_value, timeout=30.0)

Mutate a named parameter on a workflow node at runtime.

The agent must not be processing a turn when this is called. The change takes effect immediately for all subsequent turns. For system_prompt, the KV cache is lazily invalidated on the next :meth:send_message call.

Parameters:

Name	Type	Description	Default
`node_name`	`str`	Instance name of the target node (as declared in the graph).	required
`param_name`	`str`	Parameter key to set (e.g. `"temperature"`, `"system_prompt"`).	required
`param_value`	`str`	New value as a string.	required
`timeout`	`float`	Maximum seconds to wait for the server `Ack`.	`30.0`

Raises:

Type	Description
`TryllError`	On server-reported errors (`AgentBusy` 3004, `UnknownNode` 3005, `ParamNotMutable` 3006, `InvalidParamValue` 3007) or timeout.

destroy ¶

destroy(timeout=30.0)

Request agent destruction on the server and wait for Ack.

After this call returns, the proxy is no longer usable; further send_message calls will raise :class:TryllError.

Parameters:

Name	Type	Description	Default
`timeout`	`float`	Maximum seconds to wait for the `Ack` response.	`30.0`

Raises:

Type	Description
`TryllError`	On server-reported errors or timeout.

tryll_client.graph¶

tryll_client.graph ¶

GraphDescription builder and associated enums / dataclasses.

Mirrors the C++ Tryll::Client::GraphDescription / InferenceEngine / NodeType types. Enum values must stay in sync with server/schema/messages.fbs.

This module exposes:

:class:GraphDescription — fluent builder for a workflow graph sent to the server on CreateAgent.
The enums used by graph descriptions: :class:InferenceEngine, :class:NodeType, :class:ModelStatus.
The value types: :class:ModelInfo, :class:ToolDef, :class:ToolParamDef.

InferenceEngine ¶

Bases: IntEnum

Inference backend that runs the model.

Passed to :meth:TryllClient.configure_session to select the engine for all agents on this session. Mirrors the FlatBuffers InferenceEngine enum.

Mock `class-attribute` `instance-attribute` ¶

Mock = 0

In-process mock engine used by tests. No real inference.

LlamaCpp `class-attribute` `instance-attribute` ¶

LlamaCpp = 1

llama.cpp — GGUF models. The default production engine.

OnnxGenAI `class-attribute` `instance-attribute` ¶

OnnxGenAI = 2

ONNX GenAI — not yet implemented (reserved).

WindowsML `class-attribute` `instance-attribute` ¶

WindowsML = 3

Windows ML — not yet implemented (reserved).

OpenVino `class-attribute` `instance-attribute` ¶

OpenVino = 4

Intel OpenVINO — not yet implemented (reserved).

TensorRtLlm `class-attribute` `instance-attribute` ¶

TensorRtLlm = 5

NVIDIA TensorRT-LLM — not yet implemented (reserved).

ModelStatus ¶

Bases: IntEnum

Status of a model as reported by the server.

Mirrors ModelStatus in messages.fbs.

Absent `class-attribute` `instance-attribute` ¶

Absent = 0

No record of this model; call download_model first.

Local `class-attribute` `instance-attribute` ¶

Local = 1

User-supplied localPath model on disk; ready to load.

Downloading `class-attribute` `instance-attribute` ¶

Downloading = 2

Download in progress from HuggingFace.

Loaded `class-attribute` `instance-attribute` ¶

Loaded = 3

Model files are present and pinned into memory.

Downloaded `class-attribute` `instance-attribute` ¶

Downloaded = 4

Model files are present in the HuggingFace cache; not loaded.

ModelInfo `dataclass` ¶

ModelInfo(name, status, hf_repo, size_bytes)

Summary information for one model returned by list_models().

name `instance-attribute` ¶

name

Catalog name of the model (key in models.json).

status `instance-attribute` ¶

status

Current status — see :class:ModelStatus.

hf_repo `instance-attribute` ¶

hf_repo

HuggingFace repository identifier, or empty string for local-only models.

size_bytes `instance-attribute` ¶

size_bytes

Total on-disk size in bytes; zero for Absent models.

NodeType ¶

Bases: IntEnum

Workflow node kind. Mirrors NodeType in messages.fbs.

Generate `class-attribute` `instance-attribute` ¶

Generate = 0

LLM-driven text generation.

HumanMessageGuardrail `class-attribute` `instance-attribute` ¶

HumanMessageGuardrail = 1

Pattern match on the incoming user message; routes accordingly.

CannedResponse `class-attribute` `instance-attribute` ¶

CannedResponse = 2

Emit a pre-written reply without invoking a model.

ToolCall `class-attribute` `instance-attribute` ¶

ToolCall = 3

Detect and emit a tool-call invocation (small-model-friendly).

Retrieve `class-attribute` `instance-attribute` ¶

Retrieve = 4

RAG: retrieve chunks from an embedded string storage.

Instruction `class-attribute` `instance-attribute` ¶

Instruction = 5

Attach a named instruction string to the current interaction.

ToolParamDef `dataclass` ¶

ToolParamDef(name, type, description)

Description of one parameter in a tool definition.

name `instance-attribute` ¶

name

Parameter name as the model will emit it.

type `instance-attribute` ¶

type

Parameter type hint (e.g. "string", "integer", "boolean").

description `instance-attribute` ¶

description

Human-readable description shown to the model in the tool prompt.

ToolDef `dataclass` ¶

ToolDef(name, description, parameters=list())

Definition of a callable tool for a ToolCall node.

name `instance-attribute` ¶

name

Tool name as the model will emit it.

description `instance-attribute` ¶

description

Human-readable description shown to the model in the tool prompt.

parameters `class-attribute` `instance-attribute` ¶

parameters = field(default_factory=list)

Ordered list of parameter definitions (may be empty).

NodeDesc `dataclass` ¶

NodeDesc(name, node_type, params, tools=list())

Internal serialisable description of a single node in the graph.

name `instance-attribute` ¶

name

Graph-unique node name.

node_type `instance-attribute` ¶

node_type

:class:NodeType value (stored as int for wire compatibility).

params `instance-attribute` ¶

params

Key/value node parameters (all string-typed on the wire).

tools `class-attribute` `instance-attribute` ¶

tools = field(default_factory=list)

Tool definitions; only populated for ToolCall nodes.

RouteDesc `dataclass` ¶

RouteDesc(source_node, exit_name, target_node)

Internal serialisable description of one wire (exit route) between two nodes.

source_node `instance-attribute` ¶

source_node

Node name this wire leaves from.

exit_name `instance-attribute` ¶

exit_name

Name of the exit route on the source node.

target_node `instance-attribute` ¶

target_node

Node name this wire enters.

GraphDescription ¶

GraphDescription()

Fluent builder for a workflow graph sent to the server.

Typical usage::

graph = (GraphDescription()
         .add_node("gen", NodeType.Generate, {"prompt": "..."})
         .set_start_node("gen")
         .set_default_model_name("qwen2.5-0.5b-instruct"))
agent = client.create_agent(graph)

Create an empty graph description with no nodes, wires, or start node.

add_node ¶

add_node(name, node_type, params=None)

Append a node to the graph.

Parameters:

Name	Type	Description	Default
`name`	`str`	Graph-unique node name; used by :meth:`wire` and :meth:`set_start_node` to reference this node.	required
`node_type`	`NodeType \| int`	Node kind; see :class:`NodeType`. Prefer :meth:`add_tool_call_node` when `node_type == ToolCall`.	required
`params`	`dict[str, str] \| None`	Key/value node parameters. All values are sent as strings on the wire.	`None`

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

add_tool_call_node ¶

add_tool_call_node(name, tools, params=None)

Append a ToolCall node with structured tool definitions.

Convenience over :meth:add_node that accepts a list of :class:ToolDef rather than encoding tools into string params.

Parameters:

Name	Type	Description	Default
`name`	`str`	Graph-unique node name.	required
`tools`	`list[ToolDef]`	Tool definitions available to the model at this node.	required
`params`	`dict[str, str] \| None`	Additional key/value node parameters.	`None`

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

wire ¶

wire(source, exit_name, target)

Add a wire (exit route) from one node to another.

Parameters:

Name	Type	Description	Default
`source`	`str`	Name of the node the wire leaves.	required
`exit_name`	`str`	Name of the exit route on `source` (exit names are node-type-specific; see the node reference).	required
`target`	`str`	Name of the node the wire enters.	required

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

set_start_node ¶

set_start_node(name)

Nominate which node receives the turn when the agent runs.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of a node previously added via :meth:`add_node` or :meth:`add_tool_call_node`.	required

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

set_default_model_name ¶

set_default_model_name(name)

Set the default catalog model name for nodes that need one.

Nodes that invoke a language model (Generate, ToolCall) use this value when their own model param is unset.

Parameters:

Name	Type	Description	Default
`name`	`str`	Catalog model name as registered in `models.json`.	required

Returns:

Type	Description
`'GraphDescription'`	`self` for fluent chaining.

tryll_client.errors¶

tryll_client.errors ¶

Exception type for Tryll client errors.

All failures surfaced by the synchronous client — server-reported errors, protocol-level decode failures, and timeouts — are raised as :class:TryllError. Server-reported errors carry a numeric code that matches the ranges defined in server/common/include/tryll/ErrorCodes.h; client-originated errors (timeouts, closed socket) carry code == 0.

TryllError ¶

TryllError(message, code=0)

Bases: Exception

Raised on server errors, protocol errors, or timeouts.

Attributes:

Name	Type	Description
`code`		Numeric error code from the server's `ErrorCodes.h` ranges (1xxx connection, 2xxx session, 3xxx agent, 4xxx model, 5xxx node, 6xxx graph, 7xxx string storage, 8xxx embedded storage). Zero for client-originated errors such as timeouts or a closed socket.

Initialise a Tryll client error.

Parameters:

Name	Type	Description	Default
`message`	`str`	Human-readable description of the failure, typically copied from the server's `ErrorResponse.message` field or synthesised locally (e.g. "Timeout (30s) waiting…").	required
`code`	`int`	Numeric error code from `ErrorCodes.h` ranges; zero for client-originated errors.	`0`

str ¶

__str__()

Return the message, prefixed with [code] when non-zero.

tryll_client.managed_server¶

tryll_client.managed_server ¶

managed_server — spawn a tryll_server child process and wait for TCP readiness.

Typical usage::

from pathlib import Path
from tryll_client import TryllClient, ManagedServer

with ManagedServer.start(exe=Path("C:/tryll/tryll_server.exe"), port=9100) as srv:
    client = TryllClient.connect(srv.host, srv.port)
    client.configure_session(...)
    # … use client …
    client.shutdown()

No executable discovery is performed here. Pass the path to the server executable explicitly (or use the helpers in qa-and-eval/tryll_qa to locate it).

ManagedServer ¶

ManagedServer(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0)

RAII handle around a child tryll_server process.

Use as a context manager (recommended), or call :meth:stop explicitly::

with ManagedServer.start(exe=Path("..."), port=9100) as srv:
    client = TryllClient.connect(srv.host, srv.port)
    ...

No executable discovery is performed — pass the path explicitly. The server is always launched with --port <port> so the caller's port takes precedence over whatever is in server-config.json.

host `property` ¶

host

Host string to pass to :func:TryllClient.connect.

port `property` ¶

port

TCP port the server is listening on.

pid `property` ¶

pid

PID of the child process, or -1 if not running.

is_running `property` ¶

is_running

True while the child process is alive.

start `classmethod` ¶

start(*, exe, host='127.0.0.1', port=9100, cwd=None, extra_args=(), stdout=None, stderr=None, start_timeout=30.0, stop_timeout=8.0)

Spawn the server and block until its TCP port is ready.

Parameters:

Name	Type	Description	Default
`exe`	`Path \| str`	Path to `tryll_server[.exe]`. Required.	required
`host`	`str`	Host string for the TCP ready-probe (default `127.0.0.1`).	`'127.0.0.1'`
`port`	`int`	TCP port; passed as `--port` to the server (default 9100).	`9100`
`cwd`	`Path \| None`	Working directory for the child process (defaults to `exe.parent`).	`None`
`extra_args`	`Sequence[str]`	Additional CLI arguments appended after `--port <port>`.	`()`
`stdout`	`Path \| None`	File path for child stdout redirection (`None` = discard).	`None`
`stderr`	`Path \| None`	File path for child stderr redirection (`None` = discard).	`None`
`start_timeout`	`float`	Seconds to wait for the TCP port to open (default 30).	`30.0`
`stop_timeout`	`float`	Seconds to wait for clean exit on :meth:`stop` (default 8).	`8.0`

Returns:

Name	Type	Description
`A`	`'ManagedServer'`	class:`ManagedServer` whose port is accepting connections.

Raises:

Type	Description
`FileNotFoundError`	If exe does not exist.
`TimeoutError`	If the port does not open within start_timeout.
`OSError`	If the process cannot be spawned.

stop ¶

stop(*, timeout=None)

Terminate the child process.

Parameters:

Name	Type	Description	Default
`timeout`	`float \| None`	Seconds to wait before force-killing (defaults to the stop_timeout supplied at construction).	`None`

wait_for_tcp ¶

wait_for_tcp(host, port, timeout_sec, poll_interval=0.25)

Block until host:port accepts a TCP connection or timeout_sec expires.

Raises :class:TimeoutError if the port does not open in time.

start_server_process ¶

start_server_process(exe, port=9100, cwd=None, env=None, log_dir=None)

Start exe as a child process, passing --port <port> on the command line.

cwd defaults to the executable's directory (where data/ usually lives).

log_dir is the directory where server_stdout.txt and server_stderr.txt are written. Defaults to the executable's directory. Pass Path(os.devnull) to suppress output entirely.

stop_server_process ¶

stop_server_process(proc, *, terminate_timeout=8.0)

Terminate proc; kill if it does not exit within terminate_timeout.

Python Client API Reference¶

tryll_client¶

tryll_client ¶

AgentProxy ¶

agent_id property ¶

last_debug_info property ¶

last_ttft_s property ¶

last_answer_chunk_count property ¶

last_tokens_generated property ¶

set_on_tool_call ¶

send_message ¶

change_param ¶

destroy ¶

ConnectedSession ¶

client property ¶

server property ¶

shutdown ¶

TryllClient ¶

connect classmethod ¶

run_and_connect classmethod ¶

configure_session ¶

create_string_storage ¶

destroy_string_storage ¶

create_embedded_string_storage ¶

destroy_embedded_string_storage ¶

create_agent ¶

list_models ¶

load_model ¶

unload_model ¶

download_model ¶

shutdown ¶

TryllError ¶

__str__ ¶

GraphDescription ¶

add_node ¶

add_tool_call_node ¶

wire ¶

set_start_node ¶

set_default_model_name ¶

InferenceEngine ¶

Mock class-attribute instance-attribute ¶

LlamaCpp class-attribute instance-attribute ¶

OnnxGenAI class-attribute instance-attribute ¶

WindowsML class-attribute instance-attribute ¶

OpenVino class-attribute instance-attribute ¶

TensorRtLlm class-attribute instance-attribute ¶

ModelInfo dataclass ¶

name instance-attribute ¶

status instance-attribute ¶

hf_repo instance-attribute ¶

size_bytes instance-attribute ¶

ModelStatus ¶

Absent class-attribute instance-attribute ¶

Local class-attribute instance-attribute ¶

Downloading class-attribute instance-attribute ¶

Loaded class-attribute instance-attribute ¶

Downloaded class-attribute instance-attribute ¶

NodeType ¶

Generate class-attribute instance-attribute ¶

HumanMessageGuardrail class-attribute instance-attribute ¶

CannedResponse class-attribute instance-attribute ¶

ToolCall class-attribute instance-attribute ¶

Retrieve class-attribute instance-attribute ¶

Instruction class-attribute instance-attribute ¶

ManagedServer ¶

host property ¶

port property ¶

pid property ¶

is_running property ¶

start classmethod ¶

stop ¶

tryll_client.client¶

tryll_client.client ¶

TryllClient ¶

connect classmethod ¶

run_and_connect classmethod ¶

configure_session ¶

create_string_storage ¶

destroy_string_storage ¶

create_embedded_string_storage ¶

agent_id `property` ¶

last_debug_info `property` ¶

last_ttft_s `property` ¶

last_answer_chunk_count `property` ¶

last_tokens_generated `property` ¶

client `property` ¶

server `property` ¶

connect `classmethod` ¶

run_and_connect `classmethod` ¶

str ¶

Mock `class-attribute` `instance-attribute` ¶

LlamaCpp `class-attribute` `instance-attribute` ¶

OnnxGenAI `class-attribute` `instance-attribute` ¶

WindowsML `class-attribute` `instance-attribute` ¶

OpenVino `class-attribute` `instance-attribute` ¶

TensorRtLlm `class-attribute` `instance-attribute` ¶

ModelInfo `dataclass` ¶

name `instance-attribute` ¶

status `instance-attribute` ¶

hf_repo `instance-attribute` ¶

size_bytes `instance-attribute` ¶

Absent `class-attribute` `instance-attribute` ¶

Local `class-attribute` `instance-attribute` ¶

Downloading `class-attribute` `instance-attribute` ¶

Loaded `class-attribute` `instance-attribute` ¶

Downloaded `class-attribute` `instance-attribute` ¶

Generate `class-attribute` `instance-attribute` ¶

HumanMessageGuardrail `class-attribute` `instance-attribute` ¶

CannedResponse `class-attribute` `instance-attribute` ¶

ToolCall `class-attribute` `instance-attribute` ¶

Retrieve `class-attribute` `instance-attribute` ¶

Instruction `class-attribute` `instance-attribute` ¶

host `property` ¶

port `property` ¶

pid `property` ¶

is_running `property` ¶

start `classmethod` ¶

connect `classmethod` ¶

run_and_connect `classmethod` ¶

client `property` ¶

server `property` ¶

agent_id `property` ¶

last_debug_info `property` ¶

last_ttft_s `property` ¶

last_answer_chunk_count `property` ¶

last_tokens_generated `property` ¶

Mock `class-attribute` `instance-attribute` ¶

LlamaCpp `class-attribute` `instance-attribute` ¶

OnnxGenAI `class-attribute` `instance-attribute` ¶

WindowsML `class-attribute` `instance-attribute` ¶

OpenVino `class-attribute` `instance-attribute` ¶

TensorRtLlm `class-attribute` `instance-attribute` ¶

Absent `class-attribute` `instance-attribute` ¶

Local `class-attribute` `instance-attribute` ¶

Downloading `class-attribute` `instance-attribute` ¶

Loaded `class-attribute` `instance-attribute` ¶

Downloaded `class-attribute` `instance-attribute` ¶

ModelInfo `dataclass` ¶

name `instance-attribute` ¶

status `instance-attribute` ¶

hf_repo `instance-attribute` ¶

size_bytes `instance-attribute` ¶

Generate `class-attribute` `instance-attribute` ¶

HumanMessageGuardrail `class-attribute` `instance-attribute` ¶

CannedResponse `class-attribute` `instance-attribute` ¶

ToolCall `class-attribute` `instance-attribute` ¶

Retrieve `class-attribute` `instance-attribute` ¶

Instruction `class-attribute` `instance-attribute` ¶

ToolParamDef `dataclass` ¶

name `instance-attribute` ¶

type `instance-attribute` ¶

description `instance-attribute` ¶

ToolDef `dataclass` ¶

name `instance-attribute` ¶

description `instance-attribute` ¶

parameters `class-attribute` `instance-attribute` ¶

NodeDesc `dataclass` ¶

name `instance-attribute` ¶

node_type `instance-attribute` ¶

params `instance-attribute` ¶