Holds back, BFCL¶
BFCL v4 slice, one greedy pass
Only the BFCL requests that none of the offered tools can answer. A case counts when the model makes no call. A model that never calls anything scores full marks here, so read it beside Tool calls, BFCL.
Every model on this page was measured on NVIDIA GeForce RTX 4080 · Vulkan.
Holds back, BFCL¶
Measured models¶
| Model | Right | Interval | Rank range | Cases | Family |
|---|---|---|---|---|---|
| Ministral 3 3B Instruct (Q4_K_M) | 98.1% | 94.4% to 100.0% | #1–7 of 12 | 53 of 54 | Ministral |
| Ministral 3 8B Instruct (Q4_K_M) | 98.1% | 94.4% to 100.0% | #1–7 of 12 | 53 of 54 | Ministral |
| Granite 4.2 3B Instruct (Q4_K_M) | 96.3% | 90.7% to 100.0% | #1–8 of 12 | 52 of 54 | Granite |
| Bonsai 8B (Q1_0 1-bit) | 94.4% | 87.0% to 100.0% | #1–8 of 12 | 51 of 54 | Bonsai |
| Bonsai 8B (Q2_0 2-bit) | 92.6% | 85.2% to 98.1% | #1–8 of 12 | 50 of 54 | Bonsai |
| Gemma 4 E2B Instruct QAT (UD-Q4_K_XL) | 92.6% | 85.2% to 98.1% | #1–8 of 12 | 50 of 54 | Gemma |
| Gemma 4 E4B Instruct (Q4_K_M) | 92.6% | 85.2% to 98.1% | #1–8 of 12 | 50 of 54 | Gemma |
| Qwen 3.5 4B (Q4_K_M) | 88.9% | 79.6% to 96.3% | #3–8 of 12 | 48 of 54 | Qwen |
| Granite 4.2 8B Instruct (Q4_K_M) | 77.8% | 66.7% to 88.9% | #9–10 of 12 | 42 of 54 | Granite |
| Ling 3.0 Tiny (Q4_K_M) | 75.9% | 64.8% to 87.0% | #9–10 of 12 | 41 of 54 | Ling |
| Llama 3.1 8B Instruct (Q4_K_M) | 31.5% | 20.4% to 44.4% | #11–12 of 12 | 17 of 54 | Llama |
| Llama 3.2 3B Instruct (Q4_K_M) | 24.1% | 13.0% to 35.2% | #11–12 of 12 | 13 of 54 | Llama |
Model cards¶
- Ministral 3 3B Instruct (Q4_K_M)
- Ministral 3 8B Instruct (Q4_K_M)
- Granite 4.2 3B Instruct (Q4_K_M)
- Bonsai 8B (Q1_0 1-bit)
- Bonsai 8B (Q2_0 2-bit)
- Gemma 4 E2B Instruct QAT (UD-Q4_K_XL)
- Gemma 4 E4B Instruct (Q4_K_M)
- Qwen 3.5 4B (Q4_K_M)
- Granite 4.2 8B Instruct (Q4_K_M)
- Ling 3.0 Tiny (Q4_K_M)
- Llama 3.1 8B Instruct (Q4_K_M)
- Llama 3.2 3B Instruct (Q4_K_M)
No tool support: SmolLM2 1.7B Instruct (Q4_K_M).