Holds back, game tasks¶
Tryll game suites, one greedy pass
Only the game-task lines where no tool fits, such as small talk next to a door the player could open. A line counts when the model makes no call. A model that never calls anything scores full marks here, so read it beside Tool calls, game tasks.
Every model on this page was measured on NVIDIA GeForce RTX 4080 · Vulkan.
Holds back, game tasks¶
Measured models¶
| Model | Right | Interval | Rank range | Cases | Family |
|---|---|---|---|---|---|
| Bonsai 8B (Q1_0 1-bit) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Bonsai |
| Bonsai 8B (Q2_0 2-bit) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Bonsai |
| Gemma 4 E2B Instruct QAT (UD-Q4_K_XL) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Gemma |
| Gemma 4 E4B Instruct (Q4_K_M) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Gemma |
| Granite 4.2 3B Instruct (Q4_K_M) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Granite |
| Granite 4.2 8B Instruct (Q4_K_M) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Granite |
| Ministral 3 8B Instruct (Q4_K_M) | 100.0% | 100.0% to 100.0% | #1–10 of 12 | 24 of 24 | Ministral |
| Ministral 3 3B Instruct (Q4_K_M) | 95.8% | 87.5% to 100.0% | #1–10 of 12 | 23 of 24 | Ministral |
| Ling 3.0 Tiny (Q4_K_M) | 91.7% | 79.2% to 100.0% | #1–10 of 12 | 22 of 24 | Ling |
| Qwen 3.5 4B (Q4_K_M) | 87.5% | 75.0% to 100.0% | #1–10 of 12 | 21 of 24 | Qwen |
| Llama 3.1 8B Instruct (Q4_K_M) | 25.0% | 8.3% to 41.7% | #11–12 of 12 | 6 of 24 | Llama |
| Llama 3.2 3B Instruct (Q4_K_M) | 8.3% | 0.0% to 20.8% | #11–12 of 12 | 2 of 24 | Llama |
Model cards¶
- Bonsai 8B (Q1_0 1-bit)
- Bonsai 8B (Q2_0 2-bit)
- Gemma 4 E2B Instruct QAT (UD-Q4_K_XL)
- Gemma 4 E4B Instruct (Q4_K_M)
- Granite 4.2 3B Instruct (Q4_K_M)
- Granite 4.2 8B Instruct (Q4_K_M)
- Ministral 3 8B Instruct (Q4_K_M)
- Ministral 3 3B Instruct (Q4_K_M)
- Ling 3.0 Tiny (Q4_K_M)
- Qwen 3.5 4B (Q4_K_M)
- Llama 3.1 8B Instruct (Q4_K_M)
- Llama 3.2 3B Instruct (Q4_K_M)
No tool support: SmolLM2 1.7B Instruct (Q4_K_M).