Skip to content

Enable Auto Model Downloading

By default, CreateAgent fails immediately if a model referenced by the graph is not already on disk. This is the right behaviour for production — you control exactly what is installed and when. During development and prototyping, however, it adds friction: you must list models, identify which ones are missing, download them, and only then create an agent.

The allow_auto_model_downloading session flag removes that ceremony. When enabled, CreateAgent automatically downloads any missing models before building the graph, at the cost of a longer first call.

Development flag

This flag is intended for local development and prototyping only. Do not enable it in production. Unpredictable network latency, disk-space exhaustion, or catalog mismatches are all harder to handle gracefully in production code than they are with an explicit download-then-create flow.

Quick start

C++

client.ConfigureSession(Tryll::Client::InferenceEngine::Llama,
                        /*allowAutoModelDownloading=*/true);

// No pre-download step needed — CreateAgent handles it.
auto agent = client.CreateAgent(graph);

Python

client.configure_session(
    engine=tryll_client.InferenceEngine.LLAMA,
    allow_auto_model_downloading=True,
)

# No pre-download step needed — create_agent handles it.
agent = client.create_agent(graph)

Unity

bool autoDownload = TryllRuntimeSettings.Load().AllowAutoModelDownloading;
TryllClient.Instance.ConfigureSession(TryllInferenceEngine.Llama, autoDownload);

// No pre-download step needed — CreateAgent handles it.
var agent = await TryllClient.Instance.CreateAgentAsync(graph);

Tick the Allow Auto Model Downloading checkbox in Project Settings → Tryll Client to enable it without changing code. This toggles the AllowAutoModelDownloading field on TryllRuntimeSettings, which you then pass into ConfigureSession as shown above.

Tip

Commit the TryllRuntimeSettings asset with the flag set to false so team members default to the explicit download flow. Individual developers can override it locally.

Unreal

TryllSubsystem->ConfigureSession(
    ETryllInferenceEngine::Llama,
    /*bAllowAutoModelDownloading=*/true);

// No pre-download step needed — CreateAgent handles it.
TryllSubsystem->CreateAgent(Graph, ...);

The flag is also exposed as UTryllRuntimeSettings::bAllowAutoModelDownloading in Project Settings → Plugins → Tryll Client, so you can flip it without touching code.

What happens under the hood

  1. ConfigureSession stores the flag for the lifetime of the session (until the next ConfigureSession call or the socket is closed).
  2. When CreateAgent arrives, the server inspects every node in the graph and collects the list of models it references (NodeFactory::CollectRequirements).
  3. Each model is classified:
  4. On disk — nothing to do, proceed immediately.
  5. Absent, repo configured — download it first.
  6. Absent, no repo — fail immediately with ErrorResponse(ModelAutoDownloadFailed).
  7. Downloads proceed sequentially. While each download runs, the server streams DownloadProgress frames back to the client using the same request_id as the CreateAgent call (see progress frames below).
  8. Once all downloads finish, the graph is compiled and CreateAgentResponse is sent.

Progress frames

Because download progress is reported through the same DownloadProgress message type used by DownloadModelRequest, existing client-side progress handlers already work without changes. The frames arrive on the CreateAgent request_id, not a separate one.

Note

The terminal frame for the CreateAgent request is always CreateAgentResponse (on success) or ErrorResponse (on failure) — never DownloadComplete.

Timeouts

Downloading a model can take minutes. When allow_auto_model_downloading is set, the Tryll client libraries automatically extend the default CreateAgent timeout from 30 seconds to 30 minutes. You can still pass an explicit timeout to override this:

// C++: explicit 10-minute override
auto agent = client.CreateAgent(graph, {}, std::chrono::minutes(10));

# Python: explicit 10-minute override
agent = client.create_agent(graph, timeout=600)

Error handling

Error Meaning Recovery
ModelAutoDownloadFailed (6006) A required model has no download source (no huggingface_repo configured in the catalog), or the transfer failed. Check ErrorResponse.message for the model name. Add the model to the catalog or fix the network issue, then retry.
GraphCompilationFailed (3003) The graph itself is invalid (independent of model availability). Fix the graph description.