World guard¶
measured on 1 setting
The share of lines that do not belong in the game world that the guard blocks, at the same 5 in 100 budget. Measured on one setting, a medieval merchant, so treat it as a first signal rather than a promise for every world.
World guard¶
Measured models¶
| Model | Block rate | Interval | Rank range | False blocks | Family |
|---|---|---|---|---|---|
| Gemma 4 E4B Instruct (Q4_K_M) | 90.1% | 83.3% to 96.1% | #1–6 of 13 | 3.4% | Gemma |
| Qwen 3.5 4B (Q4_K_M) | 90.1% | 83.8% to 96.2% | #1–4 of 13 | 4.5% | Qwen |
| Granite 4.2 8B Instruct (Q4_K_M) | 87.7% | 80.2% to 94.4% | #1–6 of 13 | 2.8% | Granite |
| Bonsai 8B (Q2_0 2-bit) | 86.4% | 78.7% to 93.6% | #1–7 of 13 | 4.5% | Bonsai |
| Ministral 3 8B Instruct (Q4_K_M) | 82.7% | 74.4% to 90.7% | #2–8 of 13 | 1.7% | Ministral |
| Llama 3.1 8B Instruct (Q4_K_M) | 80.2% | 71.6% to 89.1% | #2–9 of 13 | 2.8% | Llama |
| Gemma 4 E2B Instruct QAT (UD-Q4_K_XL) | 76.5% | 67.5% to 85.2% | #4–9 of 13 | 1.1% | Gemma |
| Granite 4.2 3B Instruct (Q4_K_M) | 72.8% | 62.8% to 81.9% | #5–12 of 13 | 2.3% | Granite |
| Ministral 3 3B Instruct (Q4_K_M) | 71.6% | 62.0% to 80.8% | #6–12 of 13 | 1.7% | Ministral |
| Llama 3.2 3B Instruct (Q4_K_M) | 64.2% | 53.6% to 75.0% | #8–12 of 13 | 2.8% | Llama |
| Ling 3.0 Tiny (Q4_K_M) | 61.7% | 51.2% to 72.6% | #8–12 of 13 | 0.0% | Ling |
| Bonsai 8B (Q1_0 1-bit) | 60.5% | 49.4% to 71.1% | #8–12 of 13 | 2.8% | Bonsai |
| SmolLM2 1.7B Instruct (Q4_K_M) | 28.4% | 18.9% to 38.6% | #13 of 13 | 0.6% | SmolLM2 |
Model cards¶
- Gemma 4 E4B Instruct (Q4_K_M)
- Qwen 3.5 4B (Q4_K_M)
- Granite 4.2 8B Instruct (Q4_K_M)
- Bonsai 8B (Q2_0 2-bit)
- Ministral 3 8B Instruct (Q4_K_M)
- Llama 3.1 8B Instruct (Q4_K_M)
- Gemma 4 E2B Instruct QAT (UD-Q4_K_XL)
- Granite 4.2 3B Instruct (Q4_K_M)
- Ministral 3 3B Instruct (Q4_K_M)
- Llama 3.2 3B Instruct (Q4_K_M)
- Ling 3.0 Tiny (Q4_K_M)
- Bonsai 8B (Q1_0 1-bit)
- SmolLM2 1.7B Instruct (Q4_K_M)