lm-studio
google/gemma-4-e2b
Run score 41.4% across 70 completed iterations.
- Score
- 41.4%
- Passed
- 29
- Failed
- 41
- Errors
- 0
- Started
- 5/8/2026, 4:51:48 PM
- Ended
- 5/8/2026, 4:56:10 PM
- Duration
- 262s
Category Breakdown
| Category | Total | Passed | Failed | Errors | Score |
|---|---|---|---|---|---|
| Basic File Reading | 40 | 29 | 11 | 0 | 72.5% |
| Basic Skills | 30 | 0 | 30 | 0 | 0.0% |
Cases
| Case | iterations | Passed | Failed | Errors | Score |
|---|---|---|---|---|---|
| find-file | 10 total | 9 | 1 | 0 | 90.0% |
| read-exact-file | 10 total | 10 | 0 | 0 | 100.0% |
| read-exact-file-with-at-reference | 10 total | 10 | 0 | 0 | 100.0% |
| read-file | 10 total | 0 | 10 | 0 | 0.0% |
| use-skill | 10 total | 0 | 10 | 0 | 0.0% |
| use-skill-with-refs | 10 total | 0 | 10 | 0 | 0.0% |
| use-skill-with-scripts | 10 total | 0 | 10 | 0 | 0.0% |
Iterations
70 matching iterations
| Iteration | Model | Category | Variant | Status | Duration | Tools | Tokens | Context |
|---|---|---|---|---|---|---|---|---|
| find-file / 001 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 5,412ms | 4 | 7,860 | 5.5% |
| find-file / 002 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 3,846ms | 4 | 6,992 | 4.8% |
| find-file / 003 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 6,927ms | 3 | 6,962 | 6.2% |
| find-file / 004 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 4,873ms | 4 | 7,582 | 5.3% |
| find-file / 005 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 6,685ms | 4 | 8,597 | 6.1% |
| find-file / 006 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 4,571ms | 4 | 7,412 | 5.2% |
| find-file / 007 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 5,274ms | 4 | 7,828 | 5.5% |
| find-file / 008 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 4,659ms | 4 | 7,264 | 5.2% |
| find-file / 009 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 2,842ms | 4 | 7,098 | 5.0% |
| find-file / 010 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 6,404ms | 4 | 8,562 | 5.9% |
| read-exact-file / 001 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,565ms | 2 | 3,883 | 4.0% |
| read-exact-file / 002 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,720ms | 2 | 3,921 | 4.2% |
| read-exact-file / 003 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,798ms | 2 | 3,945 | 4.3% |
| read-exact-file / 004 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,695ms | 2 | 3,919 | 4.2% |
| read-exact-file / 005 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,658ms | 2 | 3,881 | 4.2% |
| read-exact-file / 006 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 297ms | 2 | 3,984 | 4.2% |
| read-exact-file / 007 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,176ms | 2 | 3,829 | 4.0% |
| read-exact-file / 008 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,913ms | 2 | 3,981 | 4.3% |
| read-exact-file / 009 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 2,155ms | 2 | 4,047 | 4.4% |
| read-exact-file / 010 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,215ms | 2 | 3,868 | 4.0% |
| read-exact-file-with-at-reference / 001 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 2,614ms | 2 | 3,936 | 4.3% |
| read-exact-file-with-at-reference / 002 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,828ms | 2 | 3,658 | 4.0% |
| read-exact-file-with-at-reference / 003 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,819ms | 2 | 3,625 | 3.9% |
| read-exact-file-with-at-reference / 004 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,979ms | 2 | 3,747 | 4.0% |
| read-exact-file-with-at-reference / 005 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,499ms | 2 | 3,618 | 3.8% |
| read-exact-file-with-at-reference / 006 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,679ms | 2 | 3,578 | 3.9% |
| read-exact-file-with-at-reference / 007 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 32ms | 2 | 3,598 | 3.7% |
| read-exact-file-with-at-reference / 008 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,212ms | 2 | 3,545 | 3.7% |
| read-exact-file-with-at-reference / 009 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,825ms | 2 | 3,641 | 4.0% |
| read-exact-file-with-at-reference / 010 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | passed | 1,301ms | 2 | 3,584 | 3.7% |
| read-file / 001 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 6,079ms | 2 | 4,764 | 5.8% |
| read-file / 002 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 7,964ms | 2 | 5,323 | 6.7% |
| read-file / 003 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 2,512ms | 2 | 3,697 | 4.3% |
| read-file / 004 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 997ms | 2 | 3,672 | 4.2% |
| read-file / 005 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 5,621ms | 2 | 4,654 | 5.7% |
| read-file / 006 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 6,011ms | 2 | 4,748 | 5.8% |
| read-file / 007 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 2,598ms | 2 | 3,754 | 4.3% |
| read-file / 008 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 2,804ms | 1 | 2,565 | 4.4% |
| read-file / 009 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 5,794ms | 2 | 4,888 | 5.8% |
| read-file / 010 | lm-studio / google/gemma-4-e2b | Basic File Reading | Baseline (/skills) | failed | 4,436ms | 2 | 4,683 | 5.7% |
| use-skill / 001 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 4,584ms | 1 | 3,037 | 5.3% |
| use-skill / 002 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,510ms | 1 | 2,754 | 4.5% |
| use-skill / 003 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,426ms | 1 | 2,588 | 4.0% |
| use-skill / 004 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,528ms | 1 | 2,738 | 4.5% |
| use-skill / 005 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 3,857ms | 1 | 2,950 | 5.1% |
| use-skill / 006 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 472ms | 1 | 2,700 | 4.2% |
| use-skill / 007 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,854ms | 1 | 2,712 | 4.2% |
| use-skill / 008 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,457ms | 1 | 2,602 | 4.0% |
| use-skill / 009 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,618ms | 1 | 2,652 | 4.1% |
| use-skill / 010 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,592ms | 1 | 2,639 | 4.1% |
| use-skill-with-refs / 001 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,910ms | 1 | 2,699 | 4.2% |
| use-skill-with-refs / 002 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 3,106ms | 1 | 2,881 | 4.8% |
| use-skill-with-refs / 003 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,661ms | 1 | 2,680 | 4.2% |
| use-skill-with-refs / 004 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,736ms | 1 | 2,799 | 4.6% |
| use-skill-with-refs / 005 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,769ms | 1 | 2,647 | 4.2% |
| use-skill-with-refs / 006 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,271ms | 1 | 2,747 | 4.6% |
| use-skill-with-refs / 007 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,963ms | 1 | 2,722 | 4.3% |
| use-skill-with-refs / 008 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 8,449ms | 1 | 3,604 | 7.2% |
| use-skill-with-refs / 009 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,754ms | 1 | 2,682 | 4.1% |
| use-skill-with-refs / 010 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,723ms | 1 | 2,598 | 4.2% |
| use-skill-with-scripts / 001 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,668ms | 1 | 2,923 | 4.8% |
| use-skill-with-scripts / 002 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,288ms | 1 | 2,735 | 4.3% |
| use-skill-with-scripts / 003 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,760ms | 1 | 2,840 | 4.5% |
| use-skill-with-scripts / 004 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,862ms | 1 | 2,843 | 4.6% |
| use-skill-with-scripts / 005 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,778ms | 1 | 3,085 | 5.0% |
| use-skill-with-scripts / 006 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,653ms | 1 | 3,032 | 4.9% |
| use-skill-with-scripts / 007 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,933ms | 1 | 2,863 | 4.6% |
| use-skill-with-scripts / 008 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,921ms | 1 | 3,008 | 5.1% |
| use-skill-with-scripts / 009 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 2,313ms | 1 | 2,916 | 4.8% |
| use-skill-with-scripts / 010 | lm-studio / google/gemma-4-e2b | Basic Skills | Baseline (/skills) | failed | 1,850ms | 1 | 2,839 | 4.6% |