lm-studio
google/gemma-4-e4b
Run score 81.4% across 70 completed iterations.
- Score
- 81.4%
- Passed
- 57
- Failed
- 13
- Errors
- 0
- Started
- 5/8/2026, 4:57:24 PM
- Ended
- 5/8/2026, 5:06:40 PM
- Duration
- 556s
Category Breakdown
| Category | Total | Passed | Failed | Errors | Score |
|---|---|---|---|---|---|
| Basic File Reading | 40 | 38 | 2 | 0 | 95.0% |
| Basic Skills | 30 | 19 | 11 | 0 | 63.3% |
Cases
| Case | iterations | Passed | Failed | Errors | Score |
|---|---|---|---|---|---|
| find-file | 10 total | 10 | 0 | 0 | 100.0% |
| read-exact-file | 10 total | 10 | 0 | 0 | 100.0% |
| read-exact-file-with-at-reference | 10 total | 10 | 0 | 0 | 100.0% |
| read-file | 10 total | 8 | 2 | 0 | 80.0% |
| use-skill | 10 total | 6 | 4 | 0 | 60.0% |
| use-skill-with-refs | 10 total | 5 | 5 | 0 | 50.0% |
| use-skill-with-scripts | 10 total | 8 | 2 | 0 | 80.0% |
Iterations
70 matching iterations
| Iteration | Model | Category | Variant | Status | Duration | Tools | Tokens | Context |
|---|---|---|---|---|---|---|---|---|
| find-file / 001 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 7,241ms | 4 | 7,219 | 2.7% |
| find-file / 002 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 6,600ms | 4 | 7,010 | 2.6% |
| find-file / 003 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 4,123ms | 3 | 5,105 | 2.2% |
| find-file / 004 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 4,696ms | 4 | 6,971 | 2.5% |
| find-file / 005 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 6,328ms | 4 | 6,743 | 2.5% |
| find-file / 006 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 5,453ms | 4 | 6,638 | 2.4% |
| find-file / 007 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 7,401ms | 4 | 7,252 | 2.7% |
| find-file / 008 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,716ms | 3 | 5,063 | 2.2% |
| find-file / 009 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 4,480ms | 3 | 5,137 | 2.3% |
| find-file / 010 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 5,792ms | 4 | 6,725 | 2.5% |
| read-exact-file / 001 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,021ms | 2 | 3,877 | 2.1% |
| read-exact-file / 002 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,071ms | 2 | 3,960 | 2.2% |
| read-exact-file / 003 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,075ms | 2 | 3,990 | 2.2% |
| read-exact-file / 004 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,076ms | 3 | 5,340 | 2.2% |
| read-exact-file / 005 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,690ms | 2 | 3,995 | 2.3% |
| read-exact-file / 006 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 1,230ms | 2 | 3,868 | 2.1% |
| read-exact-file / 007 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,715ms | 2 | 3,915 | 2.2% |
| read-exact-file / 008 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,560ms | 2 | 3,870 | 2.1% |
| read-exact-file / 009 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,248ms | 2 | 3,960 | 2.2% |
| read-exact-file / 010 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,205ms | 2 | 3,940 | 2.2% |
| read-exact-file-with-at-reference / 001 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 3,152ms | 3 | 4,810 | 2.0% |
| read-exact-file-with-at-reference / 002 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 4,850ms | 3 | 5,076 | 2.3% |
| read-exact-file-with-at-reference / 003 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 1,952ms | 2 | 3,660 | 2.1% |
| read-exact-file-with-at-reference / 004 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,264ms | 2 | 3,587 | 2.0% |
| read-exact-file-with-at-reference / 005 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,979ms | 2 | 3,606 | 2.1% |
| read-exact-file-with-at-reference / 006 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,499ms | 2 | 3,572 | 2.0% |
| read-exact-file-with-at-reference / 007 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,201ms | 2 | 3,549 | 1.9% |
| read-exact-file-with-at-reference / 008 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,962ms | 2 | 3,657 | 2.0% |
| read-exact-file-with-at-reference / 009 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,534ms | 2 | 3,573 | 2.0% |
| read-exact-file-with-at-reference / 010 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 2,758ms | 2 | 3,635 | 2.0% |
| read-file / 001 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 5,916ms | 4 | 7,392 | 2.7% |
| read-file / 002 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 6,355ms | 4 | 7,010 | 2.6% |
| read-file / 003 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 9,283ms | 4 | 8,147 | 3.0% |
| read-file / 004 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 5,922ms | 4 | 6,915 | 2.5% |
| read-file / 005 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 8,120ms | 4 | 7,574 | 2.8% |
| read-file / 006 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | failed | 10,726ms | 3 | 6,906 | 3.2% |
| read-file / 007 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 7,304ms | 4 | 7,322 | 2.7% |
| read-file / 008 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 6,488ms | 4 | 7,411 | 2.8% |
| read-file / 009 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | passed | 16,214ms | 6 | 14,273 | 4.1% |
| read-file / 010 | lm-studio / google/gemma-4-e4b | Basic File Reading | Baseline (/skills) | failed | 7,638ms | 3 | 6,215 | 2.9% |
| use-skill / 001 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 2,252ms | 2 | 3,832 | 2.1% |
| use-skill / 002 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 9,076ms | 3 | 6,659 | 3.2% |
| use-skill / 003 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 2,593ms | 2 | 3,909 | 2.2% |
| use-skill / 004 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 2,865ms | 2 | 4,017 | 2.2% |
| use-skill / 005 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 14,226ms | 3 | 7,366 | 3.9% |
| use-skill / 006 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 2,340ms | 1 | 2,562 | 2.1% |
| use-skill / 007 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 2,559ms | 2 | 3,914 | 2.2% |
| use-skill / 008 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 26,552ms | 5 | 9,485 | 5.2% |
| use-skill / 009 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 2,375ms | 2 | 3,886 | 2.1% |
| use-skill / 010 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 2,734ms | 2 | 4,167 | 2.4% |
| use-skill-with-refs / 001 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 10,671ms | 4 | 8,627 | 3.4% |
| use-skill-with-refs / 002 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 4,508ms | 2 | 4,288 | 2.5% |
| use-skill-with-refs / 003 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 15,465ms | 5 | 12,722 | 4.3% |
| use-skill-with-refs / 004 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 8,456ms | 3 | 6,848 | 3.1% |
| use-skill-with-refs / 005 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 6,872ms | 3 | 6,474 | 2.9% |
| use-skill-with-refs / 006 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 9,465ms | 4 | 9,383 | 3.5% |
| use-skill-with-refs / 007 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 20,344ms | 4 | 10,771 | 5.0% |
| use-skill-with-refs / 008 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 5,010ms | 3 | 5,884 | 2.6% |
| use-skill-with-refs / 009 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 9,695ms | 4 | 8,846 | 3.3% |
| use-skill-with-refs / 010 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 10,204ms | 3 | 6,637 | 3.4% |
| use-skill-with-scripts / 001 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 8,585ms | 4 | 8,879 | 3.3% |
| use-skill-with-scripts / 002 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 25,019ms | 8 | 22,853 | 6.0% |
| use-skill-with-scripts / 003 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 13,123ms | 5 | 13,103 | 4.0% |
| use-skill-with-scripts / 004 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 9,657ms | 6 | 14,366 | 3.8% |
| use-skill-with-scripts / 005 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 6,038ms | 3 | 6,620 | 2.9% |
| use-skill-with-scripts / 006 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | failed | 23,233ms | 4 | 11,897 | 5.5% |
| use-skill-with-scripts / 007 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 11,184ms | 6 | 13,708 | 3.8% |
| use-skill-with-scripts / 008 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 5,981ms | 3 | 6,671 | 2.9% |
| use-skill-with-scripts / 009 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 13,454ms | 5 | 13,132 | 4.2% |
| use-skill-with-scripts / 010 | lm-studio / google/gemma-4-e4b | Basic Skills | Baseline (/skills) | passed | 7,175ms | 5 | 10,314 | 3.1% |