Skip to content

Change Agent Parameters at Runtime

Adjust a node's configuration while the agent is idle — for example to switch personalities by updating system_prompt, or to tune retrieval precision by changing top_k — without recreating the agent.

Prerequisites

  • An agent created and ready to receive messages — see Build a Chat Agent with a Graph.
  • The agent must not be processing a turn when you call ChangeParams / change_params. The server rejects the request with error 3004 AgentBusy if a turn is in progress.

How it works

Parameters are now typed — each node type has a dedicated params object (e.g. GenerateParams, RetrieveParams) instead of a string key/value map.

The canonical mutation pattern is clone-set-send:

  1. Clone the baseline params object (preserves structural/immutable fields automatically).
  2. Set only the fields you want to change.
  3. Send the modified object via ChangeParams.

The server diffs the supplied params against the node's current params, vetoes any structural fields that have changed (3006 ParamNotMutable), validates values (3007 InvalidParamValue), and applies the delta.

Which fields can be changed?

Fields marked (structural) in the schema cannot be changed at runtime. All other fields are mutable unless a node-specific validator rejects the value.

Node Structural (immutable) fields Mutable fields
Generate model_name system_prompt, template, placement, stream, all sampling.* fields
ToolCall model_name, tool_call_format, tools system_prompt, generate_on_no_tool, notify_client, all sampling.* fields
Retrieve embedded_string_storage, embedding_model top_k, threshold, source, filter
CannedResponse inline_strings string_storage (rebind), selection_strategy
HumanMessageGuardrail inline_strings string_storage (rebind)
Instruction — instruction_name, instruction
ClassifyIntent embedded_string_storage, embedding_model metadata_field, threshold, filter, notify_client, diagnostic_topk
ClassifyIntentLLM model_name, intents_ids, intents_prompt system_prompt, history_turns, threshold, margin, notify_client, not_found_intent
IntentToInstruction inline_keys, inline_strings string_storage (rebind), instruction_name

Examples

using Tryll.Client;

var agentComp = GetComponent<TryllAgentComponent>();

// Clone the baseline, change one field, send.
var p = (TryllGenerateParams)agentComp.Agent.GetNodeParamsBaseline("answer").Clone();
p.SystemPrompt = "You are a pirate.";
agentComp.Agent.ChangeParams("answer", p,
    err => {
        if (err == null)
            Debug.Log("Persona updated");
        else
            Debug.LogError($"Error {err.Code}: {err.Message}");
    });

// Relax the retrieval threshold.
var rp = (TryllRetrieveParams)agentComp.Agent.GetNodeParamsBaseline("knowledge").Clone();
rp.Threshold = 0.9f;
agentComp.Agent.ChangeParams("knowledge", rp);

// Turn on streaming.
var gp = (TryllGenerateParams)agentComp.Agent.GetNodeParamsBaseline("answer").Clone();
gp.Stream = true;
agentComp.Agent.ChangeParams("answer", gp);

#include "Generated/Nodes/TryllGenerateParams.h"
#include "Generated/Nodes/TryllNodeParamsFactory.h"

// Clone the baseline, change one field, send.
UTryllNodeParamsBase* Base = AgentComponent->GetNodeParamsBaseline(TEXT("answer"));
UTryllGenerateParams* P = Cast<UTryllGenerateParams>(
    UTryllNodeParamsFactory::CloneParams(Base, this));
P->SystemPrompt = TEXT("You are a pirate.");
AgentComponent->ChangeParams(
    TEXT("answer"), P,
    [](const FTryllError& Err)
    {
        if (Err.Code == 0)
            UE_LOG(LogTemp, Log, TEXT("Persona updated"));
        else
            UE_LOG(LogTemp, Warning, TEXT("Error %d: %s"), Err.Code, *Err.Message);
    });

// Relax the retrieval threshold.
UTryllRetrieveParams* RP = Cast<UTryllRetrieveParams>(
    UTryllNodeParamsFactory::CloneParams(
        AgentComponent->GetNodeParamsBaseline(TEXT("knowledge")), this));
RP->Threshold = 0.9f;
AgentComponent->ChangeParams(TEXT("knowledge"), RP);
  1. Call Get Node Params Baseline on the TryllAgentComponent to get the current params object for a node.
  2. Call Clone Params (from TryllNodeParamsFactory) on the result.
  3. Set any fields on the cloned object.
  4. Call Change Params on the TryllAgentComponent and bind On Param Changed to handle success or error.

The C++ client does not cache a baseline copy of authored params. Keep your own copy of the params you last sent (or authored at CreateAgent) and start each mutation from that local copy. Structural fields must match the create-time values or the server returns ParamNotMutable.

#include <tryll/AgentProxy.h>
// XxxT PODs come from the flatc-generated messages_generated.h (--gen-object-api)
#include <NodeParams_generated.h>

using namespace Tryll::NodeParams;
using namespace Tryll::Client;

// Clone the caller-owned baseline, change system prompt, send asynchronously.
GenerateParamsT p = answerBaseline;          // caller-owned copy
p.system_prompt = "You are a pirate.";
agent.ChangeParamsAsync("answer", std::move(p)).get(); // or .then(...)

// Relax the retrieval threshold (synchronous).
RetrieveParamsT rp = knowledgeBaseline;      // caller-owned copy
rp.threshold = 0.9f;
agent.ChangeParams("knowledge", std::move(rp));

The Python client does not cache a baseline copy of authored params. Keep your own copy of the params you last sent (or authored at create_agent) and start each mutation from a deepcopy of that local copy.

from copy import deepcopy
from tryll_client.graph import GenerateParams, RetrieveParams

# Clone the caller-owned baseline, change system_prompt, send.
p = deepcopy(answer_baseline)        # caller-owned copy
p.system_prompt = "You are a pirate."
agent.change_params("answer", p)

# Relax the retrieval threshold.
rp = deepcopy(knowledge_baseline)
rp.threshold = 0.9
agent.change_params("knowledge", rp)

# Turn on streaming.
gp = deepcopy(answer_baseline)
gp.stream = True
agent.change_params("answer", gp)

change_params raises a TryllError on failure (unknown node, structural field changed, or invalid value).

system_prompt and KV-cache rewind

Changing system_prompt does not immediately flush the KV cache. The change is stored and picked up on the next send_message call. At that point the projection pipeline detects the new token sequence, trims the stale tail of the cache, and re-decodes only the changed prefix. This is the most efficient path: back-to-back change_params / send_message calls impose only one re-decode, not two.

Error codes

Code Name Cause
3004 AgentBusy A turn is in progress. Wait for TurnComplete before calling ChangeParams.
3005 UnknownNode node_name does not match any node instance name in the graph.
3006 ParamNotMutable A structural field was included in the mutation request.
3007 InvalidParamValue A mutable field failed schema-range or node-specific validation.