Design
Protocol control for Qwen2.5-0.5B GSM8K evaluation. The run compares the unpatched base model with rank-1 EGGROLL and decomposed EGGROLL patched models under zero updates, using the same 8-shot CoT prompt/parser as the baseline sanity eval. The purpose is to verify that the ES adapter wrappers do not change behaviour before optimisation.
Results
20260809-005632-qwen05_gsm8k_es_protocol_control_jean_zay-1a327b3
commit: 1a327b3
| metric | value |
|---|---|
| base_accuracy | 0.38438212871551514 |
| base_num_correct | 507 |
| base_num_parseable | 1319 |
| base_parseable_rate | 1 |
| decomposed_patched_accuracy | 0.38438212871551514 |
| decomposed_patched_accuracy_gap_vs_base | 0 |
| decomposed_patched_num_correct | 507 |
| decomposed_patched_num_parseable | 1319 |
| decomposed_patched_parseable_gap_vs_base | 0 |
| decomposed_patched_parseable_rate | 1 |
| eggroll_patched_accuracy | 0.38438212871551514 |
| eggroll_patched_accuracy_gap_vs_base | 0 |
| eggroll_patched_num_correct | 507 |
| eggroll_patched_num_parseable | 1319 |
| eggroll_patched_parseable_gap_vs_base | 0 |
| eggroll_patched_parseable_rate | 1 |
| eval_batch_size | 128 |
| eval_samples | 1319 |
| max_new_tokens | 512 |
| num_target_modules | 7 |
| rank | 1 |
| wall_clock_seconds | 1003.3977364134043 |
Interpretation
Run 20260809-005632-qwen05_gsm8k_es_protocol_control_jean_zay-1a327b3 passes the wrapper-agreement check: base, EGGROLL-patched, and decomposed-patched evaluations are identical on accuracy and parseability, with zero gaps versus base for both patched variants. This means the earlier very low Qwen smoke accuracies were not caused by merely attaching ES adapters; future Qwen reward comparisons can use the patched harness, but should still report the protocol distinction from the no-ES baseline eval.