Class Tryll::TryllClient¶

ClassList > Tryll > TryllClient

TCP session to the Tryll server. More...

#include <TryllClient.h>

Classes¶

Type	Name
struct	EmbeddedStorageInfo Result returned by `CreateEmbeddedStringStorage` variants.

Public Functions¶

Type	Name
void	ConfigureSession (Client::InferenceEngine engine, bool allowAutoModelDownloading=false, std::chrono::milliseconds timeout=std::chrono::seconds{30}) Select the session's inference engine.
AgentProxy	CreateAgent (const Tryll::Client::GraphDescription & graph, bool enableDiagnostics=false, std::optional< std::chrono::milliseconds > timeout=std::nullopt) Blocking convenience wrapper over CreateAgentAsync .
std::future< AgentProxy >	CreateAgentAsync (const Tryll::Client::GraphDescription & graph, bool enableDiagnostics=false) Asynchronously create an agent with the given graph description.
EmbeddedStorageInfo	CreateEmbeddedStringStorage (std::string_view name, std::string_view configPath, std::string_view embeddingModel={}, std::optional< std::chrono::milliseconds > timeout=std::nullopt) Create an `EmbeddedStringStorage` from a server-side config.
EmbeddedStorageInfo	CreateEmbeddedStringStorageFromStrings (std::string_view name, const std::vector< std::string > & strings, std::string_view embeddingModel, std::optional< std::chrono::milliseconds > timeout=std::nullopt) Create an `EmbeddedStringStorage` from inline strings.
void	CreateStringStorage (std::string_view name, const std::vector< std::string > & strings, std::chrono::milliseconds timeout=std::chrono::seconds{10}) Create a named `StringStorage` on the server from an inline list.
void	CreateStringStorageFromFile (std::string_view name, std::string_view filePath, std::chrono::milliseconds timeout=std::chrono::seconds{10}) Create a named `StringStorage` on the server from a server-side file.
void	DestroyEmbeddedStringStorage (std::string_view name, std::chrono::milliseconds timeout=std::chrono::seconds{10}) Destroy a named `EmbeddedStringStorage` .
void	DestroyStringStorage (std::string_view name, std::chrono::milliseconds timeout=std::chrono::seconds{10}) Destroy a named `StringStorage` on the server.
void	DownloadModel (std::string_view modelName, Client::DownloadProgressCallback onProgress=nullptr, std::chrono::milliseconds timeout=std::chrono::minutes{30}) Start downloading a model on the server and block until complete.
std::uint64_t	GetSessionId () noexcept const Server-assigned session identifier.
std::vector< Client::ModelInfo >	ListModels (std::chrono::milliseconds timeout=std::chrono::seconds{10}) Request the list of models known to the server for the session's engine.
void	LoadModel (std::string_view modelName, std::chrono::milliseconds timeout=std::chrono::minutes{5}) Explicitly load and pin a model into memory.
void	Shutdown (std::chrono::milliseconds timeout=std::chrono::seconds{30}) Blocking convenience wrapper over ShutdownAsync .
std::future< void >	ShutdownAsync () Asynchronously shut down the connection.
	TryllClient (const TryllClient &) = default
	TryllClient (TryllClient &&) noexcept
void	UnloadModel (std::string_view modelName, std::chrono::milliseconds timeout=std::chrono::seconds{30}) Unpin a previously pinned model.
TryllClient &	operator= (const TryllClient &) = default
TryllClient &	operator= (TryllClient &&) noexcept
	~TryllClient ()

Public Static Functions¶

Type	Name
TryllClient	Connect (std::string_view host, std::uint16_t port, std::chrono::milliseconds timeout=std::chrono::seconds{30}) Blocking convenience wrapper over ConnectAsync .
std::future< TryllClient >	ConnectAsync (std::string_view host, std::uint16_t port) Asynchronously connect and wait for `SessionReady` .
ConnectedSession	RunAndConnect (Client::ManagedServerOptions serverOpts, std::chrono::milliseconds connectTimeout=std::chrono::seconds{30}) Spawn a local server and connect to it — recommended for single-process deployments where the client owns the server lifecycle.
std::future< ConnectedSession >	RunAndConnectAsync (Client::ManagedServerOptions serverOpts) Asynchronous variant of RunAndConnect .

Detailed Description¶

Copyable — all copies share the same underlying connection via shared_ptr. The connection stays open until the last copy goes out of scope or Shutdown is called.

Public Functions Documentation¶

function ConfigureSession¶

Select the session's inference engine.

void Tryll::TryllClient::ConfigureSession (
    Client::InferenceEngine engine,
    bool allowAutoModelDownloading=false,
    std::chrono::milliseconds timeout=std::chrono::seconds{30}
)

Must be called after Connect and before CreateAgent or any model-management call.

Parameters:

engine Inference backend; see Client::InferenceEngine.
allowAutoModelDownloading When true, the server automatically downloads any missing models referenced by a graph during CreateAgent instead of failing immediately. Intended for development and prototyping — not for production use. When this flag is set, CreateAgent and CreateAgentAsync use a 30-minute default timeout unless the caller overrides it explicitly.
timeout Maximum time to wait for the response.

Exception:

TryllError On server-reported errors or timeout.

function CreateAgent¶

Blocking convenience wrapper over CreateAgentAsync .

AgentProxy Tryll::TryllClient::CreateAgent (
    const Tryll::Client::GraphDescription & graph,
    bool enableDiagnostics=false,
    std::optional< std::chrono::milliseconds > timeout=std::nullopt
)

Parameters:

graph Fully-built graph description.
enableDiagnostics When true, the server serialises per-node execution data into TurnComplete.debug_info for every turn. Off by default; enable for QA/eval pipelines.
timeout Maximum time to wait for CreateAgentResponse. Defaults to 30 seconds normally, or to 30 minutes when ConfigureSession was called with allowAutoModelDownloading=true (to accommodate model downloads).

Returns:

AgentProxy bound to the new server-side agent.

Exception:

TryllError On validation failures, server-reported errors (including ModelAutoDownloadFailed when allowAutoModelDownloading=true and a model cannot be downloaded), or timeout.

function CreateAgentAsync¶

Asynchronously create an agent with the given graph description.

std::future< AgentProxy > Tryll::TryllClient::CreateAgentAsync (
    const Tryll::Client::GraphDescription & graph,
    bool enableDiagnostics=false
)

Parameters:

graph Fully-built graph description.
enableDiagnostics When true, the server serialises per-node execution data into TurnComplete.debug_info for every turn. Off by default for zero overhead in production; enable for QA/eval pipelines.

Note:

When ConfigureSession was called with allowAutoModelDownloading=true, this call may block internally while models are being downloaded. The returned future completes when creation succeeds or fails definitively. DownloadProgress frames are emitted during the download (observable via the DownloadModel progress callback path).

Returns:

Future yielding an AgentProxy bound to the new agent. Calling future::get() propagates any TryllError raised during validation or creation.

function CreateEmbeddedStringStorage¶

Create an EmbeddedStringStorage from a server-side config.

EmbeddedStorageInfo Tryll::TryllClient::CreateEmbeddedStringStorage (
    std::string_view name,
    std::string_view configPath,
    std::string_view embeddingModel={},
    std::optional< std::chrono::milliseconds > timeout=std::nullopt
)

Path A: the server loads records and (optionally) an on-disk HNSW index described by a *.json config referencing a *.kb.json records file.

If allow_auto_model_downloading was set in ConfigureSession and the required embedding model is absent, it will be downloaded before the storage is built. In that case the default timeout is automatically raised to 30 minutes; pass an explicit value to override.

Parameters:

name Session-unique storage name.
configPath Server-side *.json path.
embeddingModel Optional catalog embedding-model name for cross-check; when non-empty must match the config's embedding_model.
timeout Maximum time to wait for the response; embedding large corpora is slow.

Returns:

EmbeddedStorageInfo with name, record count, and dimension.

Exception:

TryllError On server-reported errors (including InvalidStringStorageData when embeddingModel mismatches the config) or timeout.

function CreateEmbeddedStringStorageFromStrings¶

Create an EmbeddedStringStorage from inline strings.

EmbeddedStorageInfo Tryll::TryllClient::CreateEmbeddedStringStorageFromStrings (
    std::string_view name,
    const std::vector< std::string > & strings,
    std::string_view embeddingModel,
    std::optional< std::chrono::milliseconds > timeout=std::nullopt
)

Path B: the server embeds strings in memory with the named embedding model; no on-disk index is written.

If allow_auto_model_downloading was set in ConfigureSession and the required embedding model is absent, it will be downloaded before embedding begins. In that case the default timeout is automatically raised to 30 minutes; pass an explicit value to override.

Parameters:

name Session-unique storage name.
strings Inline records to embed.
embeddingModel Catalog embedding-model name (required).
timeout Maximum time to wait; embedding is slow.

Returns:

EmbeddedStorageInfo with name, record count, and dimension.

Exception:

TryllError On server-reported errors or timeout.

function CreateStringStorage¶

Create a named StringStorage on the server from an inline list.

void Tryll::TryllClient::CreateStringStorage (
    std::string_view name,
    const std::vector< std::string > & strings,
    std::chrono::milliseconds timeout=std::chrono::seconds{10}
)

The storage may then be referenced by name in node params via string_storage (e.g. on CannedResponseNode or HumanMessageGuardrailNode). Must be called before CreateAgent on any graph that references this storage name.

Parameters:

name Session-unique storage name.
strings Inline list of strings to store.
timeout Maximum time to wait for the response.

Exception:

TryllError On server-reported errors or timeout.

function CreateStringStorageFromFile¶

Create a named StringStorage on the server from a server-side file.

void Tryll::TryllClient::CreateStringStorageFromFile (
    std::string_view name,
    std::string_view filePath,
    std::chrono::milliseconds timeout=std::chrono::seconds{10}
)

Parameters:

name Session-unique storage name.
filePath Server-side path to a newline-delimited text file.
timeout Maximum time to wait for the response.

Exception:

TryllError On server-reported errors, missing file, or timeout.

function DestroyEmbeddedStringStorage¶

Destroy a named EmbeddedStringStorage .

void Tryll::TryllClient::DestroyEmbeddedStringStorage (
    std::string_view name,
    std::chrono::milliseconds timeout=std::chrono::seconds{10}
)

Nodes that already hold the storage via shared_ptr keep it alive; this call only drops the session-level name mapping.

Parameters:

name Name passed to one of the Create variants.
timeout Maximum time to wait for the response.

Exception:

TryllError On server-reported errors or timeout.

function DestroyStringStorage¶

Destroy a named StringStorage on the server.

void Tryll::TryllClient::DestroyStringStorage (
    std::string_view name,
    std::chrono::milliseconds timeout=std::chrono::seconds{10}
)

Nodes that already hold the storage via shared_ptr keep it alive; this call only drops the session-level name mapping.

Parameters:

name Name passed to CreateStringStorage or CreateStringStorageFromFile.
timeout Maximum time to wait for the response.

Exception:

TryllError On server-reported errors or timeout.

function DownloadModel¶

Start downloading a model on the server and block until complete.

void Tryll::TryllClient::DownloadModel (
    std::string_view modelName,
    Client::DownloadProgressCallback onProgress=nullptr,
    std::chrono::milliseconds timeout=std::chrono::minutes{30}
)

Progress frames are delivered via onProgress on the reader thread (invoked synchronously, so keep the callback fast and non-blocking).

Parameters:

modelName Catalog model name from models.json.
onProgress Optional per-chunk progress callback.
timeout Maximum time to wait for DownloadComplete.

Exception:

TryllError On server-reported errors, a DownloadComplete reporting failure, or timeout.

function GetSessionId¶

Server-assigned session identifier.

std::uint64_t Tryll::TryllClient::GetSessionId () noexcept const

Returns:

The session_id field received in SessionReady.

function ListModels¶

Request the list of models known to the server for the session's engine.

std::vector< Client::ModelInfo > Tryll::TryllClient::ListModels (
    std::chrono::milliseconds timeout=std::chrono::seconds{10}
)

Parameters:

timeout Maximum time to wait for ListModelsResponse.

Returns:

Catalog entries, one per known model.

Exception:

TryllError On server-reported errors or timeout.

function LoadModel¶

Explicitly load and pin a model into memory.

void Tryll::TryllClient::LoadModel (
    std::string_view modelName,
    std::chrono::milliseconds timeout=std::chrono::minutes{5}
)

The model stays in memory until UnloadModel is called, regardless of whether any agents are using it.

Parameters:

modelName Catalog model name from models.json.
timeout Maximum time to wait; loading is slow for large GGUFs.

Exception:

TryllError If the model cannot be resolved or loaded, or on timeout.

function Shutdown¶

Blocking convenience wrapper over ShutdownAsync .

void Tryll::TryllClient::Shutdown (
    std::chrono::milliseconds timeout=std::chrono::seconds{30}
)

Parameters:

timeout Maximum time to wait for a graceful close.

Exception:

TryllError On timeout; socket errors during shutdown are suppressed.

function ShutdownAsync¶

Asynchronously shut down the connection.

std::future< void > Tryll::TryllClient::ShutdownAsync ()

Pending requests are cancelled with TryllError. Idempotent.

Returns:

Future completing when the socket is closed.

function TryllClient [1/3]¶

Tryll::TryllClient::TryllClient (
    const TryllClient &
) = default

function TryllClient [2/3]¶

Tryll::TryllClient::TryllClient (
    TryllClient &&
) noexcept

function UnloadModel¶

Unpin a previously pinned model.

void Tryll::TryllClient::UnloadModel (
    std::string_view modelName,
    std::chrono::milliseconds timeout=std::chrono::seconds{30}
)

Freed immediately if no active contexts reference it; freed lazily when the last one is destroyed.

Parameters:

modelName Catalog model name previously pinned with LoadModel.
timeout Maximum time to wait for the ack.

Exception:

TryllError On timeout.

function operator=¶

TryllClient & Tryll::TryllClient::operator= (
    const TryllClient &
) = default

function operator=¶

TryllClient & Tryll::TryllClient::operator= (
    TryllClient &&
) noexcept

function ~TryllClient¶

Tryll::TryllClient::~TryllClient ()

Public Static Functions Documentation¶

function Connect¶

Blocking convenience wrapper over ConnectAsync .

static TryllClient Tryll::TryllClient::Connect (
    std::string_view host,
    std::uint16_t port,
    std::chrono::milliseconds timeout=std::chrono::seconds{30}
)

Parameters:

host Hostname or IP of the Tryll server.
port TCP port.
timeout Maximum time to wait for SessionReady.

Returns:

A ready-to-use TryllClient.

Exception:

TryllError On connection failure, timeout, or a failed session handshake.

function ConnectAsync¶

Asynchronously connect and wait for SessionReady .

static std::future< TryllClient > Tryll::TryllClient::ConnectAsync (
    std::string_view host,
    std::uint16_t port
)

Parameters:

host Hostname or IP of the Tryll server.
port TCP port the server is listening on.

Returns:

Future yielding a ready-to-use TryllClient. Calling future::get() propagates any TryllError raised during the handshake.

function RunAndConnect¶

Spawn a local server and connect to it — recommended for single-process deployments where the client owns the server lifecycle.

static ConnectedSession Tryll::TryllClient::RunAndConnect (
    Client::ManagedServerOptions serverOpts,
    std::chrono::milliseconds connectTimeout=std::chrono::seconds{30}
)

Equivalent to calling Client::ManagedServer::Start followed by Connect, with correct cleanup if Connect throws. The returned ConnectedSession owns both the server process and the TCP session; on destruction it shuts the client down first, then terminates the server.

Parameters:

serverOpts Server launch options. serverOpts.exe is required.
connectTimeout Maximum time to wait for SessionReady after the server's TCP port becomes reachable.

Returns:

A ConnectedSession ready to use.

Exception:

TryllError If the server cannot be started, the TCP ready-probe times out, or the session handshake fails.

function RunAndConnectAsync¶

Asynchronous variant of RunAndConnect .

static std::future< ConnectedSession > Tryll::TryllClient::RunAndConnectAsync (
    Client::ManagedServerOptions serverOpts
)

Spawns a std::async task that calls RunAndConnect internally. Useful when the calling thread must not block during startup.

Parameters:

serverOpts Server launch options.

Returns:

Future yielding a ConnectedSession. Calling future::get() propagates any TryllError from the launch or handshake.

The documentation for this class was generated from the following file C:/_tryll/_monorepo/server/client-cpp/include/tryll/TryllClient.h