Skip to content

Model Benchmarks

Provisional · immersion guard, speed and memory

How catalog models do on the jobs Tryll gives them, measured by the Tryll team on our own test sets. Measured so far: the immersion guard, General guard (attempts to pull the NPC out of the game) and World guard (lines that do not belong in the setting), and speed and memory: how fast a model reads its prompt and writes its answer, and how much graphics memory it needs. NPC roleplay, tool calling and summarization will follow.

The numbers are provisional: 13 models. The guard is measured once per model (it reads probabilities and does not sample), at a 5 in 100 false-block budget, with World guard on one game setting. Speed and memory are measured on NVIDIA GeForce RTX 5070 Ti with Vulkan; other hardware gives other numbers.

Model cards

Leaderboards