lm-studio
granite-4.1-8b
Run score 85.7% across 70 completed iterations.
- Score
- 85.7%
- Passed
- 60
- Failed
- 10
- Errors
- 0
- Started
- 5/8/2026, 4:38:38 PM
- Ended
- 5/8/2026, 4:41:30 PM
- Duration
- 173s
Category Breakdown
| Category | Total | Passed | Failed | Errors | Score |
|---|---|---|---|---|---|
| Basic File Reading | 40 | 40 | 0 | 0 | 100.0% |
| Basic Skills | 30 | 20 | 10 | 0 | 66.7% |
Cases
| Case | iterations | Passed | Failed | Errors | Score |
|---|---|---|---|---|---|
| find-file | 10 total | 10 | 0 | 0 | 100.0% |
| read-exact-file | 10 total | 10 | 0 | 0 | 100.0% |
| read-exact-file-with-at-reference | 10 total | 10 | 0 | 0 | 100.0% |
| read-file | 10 total | 10 | 0 | 0 | 100.0% |
| use-skill | 10 total | 10 | 0 | 0 | 100.0% |
| use-skill-with-refs | 10 total | 10 | 0 | 0 | 100.0% |
| use-skill-with-scripts | 10 total | 0 | 10 | 0 | 0.0% |
Iterations
70 matching iterations
| Iteration | Model | Category | Variant | Status | Duration | Tools | Tokens | Context |
|---|---|---|---|---|---|---|---|---|
| find-file / 001 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 2,280ms | 5 | 7,892 | 4.3% |
| find-file / 002 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,649ms | 4 | 6,419 | 4.2% |
| find-file / 003 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,605ms | 4 | 6,403 | 4.2% |
| find-file / 004 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,644ms | 4 | 6,415 | 4.2% |
| find-file / 005 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,885ms | 4 | 6,485 | 4.2% |
| find-file / 006 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,728ms | 4 | 6,434 | 4.2% |
| find-file / 007 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,751ms | 4 | 6,435 | 4.2% |
| find-file / 008 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 2,125ms | 5 | 7,837 | 4.3% |
| find-file / 009 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 2,022ms | 5 | 7,807 | 4.3% |
| find-file / 010 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,734ms | 4 | 6,413 | 4.2% |
| read-exact-file / 001 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 500ms | 2 | 4,083 | 4.3% |
| read-exact-file / 002 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,255ms | 2 | 4,085 | 4.3% |
| read-exact-file / 003 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,102ms | 2 | 4,076 | 4.3% |
| read-exact-file / 004 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,056ms | 2 | 4,076 | 4.3% |
| read-exact-file / 005 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,255ms | 2 | 4,086 | 4.3% |
| read-exact-file / 006 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,367ms | 2 | 4,096 | 4.3% |
| read-exact-file / 007 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,145ms | 2 | 4,081 | 4.3% |
| read-exact-file / 008 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,080ms | 2 | 4,076 | 4.3% |
| read-exact-file / 009 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,078ms | 2 | 4,076 | 4.3% |
| read-exact-file / 010 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,079ms | 2 | 4,076 | 4.3% |
| read-exact-file-with-at-reference / 001 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,759ms | 2 | 3,775 | 4.0% |
| read-exact-file-with-at-reference / 002 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,079ms | 2 | 3,753 | 3.9% |
| read-exact-file-with-at-reference / 003 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,098ms | 2 | 3,753 | 3.9% |
| read-exact-file-with-at-reference / 004 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,111ms | 2 | 3,753 | 3.9% |
| read-exact-file-with-at-reference / 005 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,268ms | 2 | 3,766 | 4.0% |
| read-exact-file-with-at-reference / 006 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,374ms | 2 | 3,773 | 4.0% |
| read-exact-file-with-at-reference / 007 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,077ms | 2 | 3,753 | 3.9% |
| read-exact-file-with-at-reference / 008 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,211ms | 2 | 3,762 | 4.0% |
| read-exact-file-with-at-reference / 009 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,285ms | 2 | 3,766 | 4.0% |
| read-exact-file-with-at-reference / 010 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,092ms | 2 | 3,753 | 3.9% |
| read-file / 001 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,675ms | 3 | 5,089 | 4.1% |
| read-file / 002 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,886ms | 4 | 6,489 | 4.2% |
| read-file / 003 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,917ms | 4 | 6,488 | 4.2% |
| read-file / 004 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,930ms | 4 | 6,492 | 4.3% |
| read-file / 005 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,586ms | 3 | 5,076 | 4.1% |
| read-file / 006 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,932ms | 4 | 6,487 | 4.2% |
| read-file / 007 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,565ms | 3 | 5,082 | 4.1% |
| read-file / 008 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,593ms | 3 | 5,084 | 4.1% |
| read-file / 009 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,911ms | 4 | 6,486 | 4.2% |
| read-file / 010 | lm-studio / granite-4.1-8b | Basic File Reading | Baseline (/skills) | passed | 1,462ms | 3 | 5,082 | 4.1% |
| use-skill / 001 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 1,459ms | 2 | 4,045 | 4.3% |
| use-skill / 002 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 964ms | 2 | 4,045 | 4.3% |
| use-skill / 003 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 987ms | 2 | 4,045 | 4.3% |
| use-skill / 004 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 980ms | 2 | 4,045 | 4.3% |
| use-skill / 005 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 981ms | 2 | 4,045 | 4.3% |
| use-skill / 006 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 975ms | 2 | 4,045 | 4.3% |
| use-skill / 007 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 987ms | 2 | 4,045 | 4.3% |
| use-skill / 008 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 979ms | 2 | 4,045 | 4.3% |
| use-skill / 009 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 966ms | 2 | 4,045 | 4.3% |
| use-skill / 010 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 964ms | 2 | 4,045 | 4.3% |
| use-skill-with-refs / 001 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 2,373ms | 4 | 7,219 | 4.7% |
| use-skill-with-refs / 002 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 862ms | 4 | 7,263 | 4.8% |
| use-skill-with-refs / 003 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 1,940ms | 4 | 7,225 | 4.7% |
| use-skill-with-refs / 004 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 1,870ms | 4 | 7,219 | 4.7% |
| use-skill-with-refs / 005 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 1,952ms | 4 | 7,226 | 4.7% |
| use-skill-with-refs / 006 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 2,061ms | 4 | 7,246 | 4.7% |
| use-skill-with-refs / 007 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 2,025ms | 4 | 7,248 | 4.7% |
| use-skill-with-refs / 008 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 2,127ms | 4 | 7,250 | 4.7% |
| use-skill-with-refs / 009 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 1,917ms | 4 | 7,224 | 4.7% |
| use-skill-with-refs / 010 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | passed | 2,019ms | 4 | 7,227 | 4.7% |
| use-skill-with-scripts / 001 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 3,072ms | 3 | 6,320 | 5.3% |
| use-skill-with-scripts / 002 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 1,012ms | 3 | 6,284 | 5.2% |
| use-skill-with-scripts / 003 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,722ms | 3 | 6,298 | 5.3% |
| use-skill-with-scripts / 004 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,311ms | 3 | 6,286 | 5.2% |
| use-skill-with-scripts / 005 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,360ms | 3 | 6,251 | 5.2% |
| use-skill-with-scripts / 006 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,163ms | 4 | 7,942 | 5.2% |
| use-skill-with-scripts / 007 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,681ms | 3 | 6,287 | 5.3% |
| use-skill-with-scripts / 008 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,484ms | 3 | 6,271 | 5.2% |
| use-skill-with-scripts / 009 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 2,848ms | 3 | 6,308 | 5.3% |
| use-skill-with-scripts / 010 | lm-studio / granite-4.1-8b | Basic Skills | Baseline (/skills) | failed | 1,435ms | 3 | 6,307 | 5.3% |