Design
This experiment tests the real-dataset version of the exact-answer reward
gate from the draft empirical sketch proof. It uses a small GSM8K subset,
sampled model answers, integer parsing, and scalar exact-answer reward. There
is still no solid theory note, so this is an empirical plumbing and sparsity
check rather than a settled-theory experiment.
Results
20260801-145407-pretrained_llm_gsm8k_reward_smoke_jean_zay-0cabfb5
commit: 0cabfb5
| metric | value |
|---|---|
| both_success_rate | 0.5 |
| decomposed_mean_accuracy_improvement | 0.02083333395421505 |
| decomposed_mean_best_accuracy | 0.0416666679084301 |
| decomposed_mean_final_accuracy | 0.02083333395421505 |
| decomposed_mean_final_parseable_rate | 0.9791666865348816 |
| decomposed_mean_finite_epoch_fraction | 1 |
| decomposed_mean_parseable_rate_improvement | 0 |
| decomposed_mean_shaped_epoch_fraction | 0.8125 |
| decomposed_median_final_accuracy | 0.02083333395421505 |
| decomposed_success_rate | 0.5 |
| decomposed_win_rate | 0 |
| eggroll_mean_accuracy_improvement | 0.02083333395421505 |
| eggroll_mean_best_accuracy | 0.031250000931322575 |
| eggroll_mean_final_accuracy | 0.02083333395421505 |
| eggroll_mean_final_parseable_rate | 0.9791666865348816 |
| eggroll_mean_finite_epoch_fraction | 1 |
| eggroll_mean_parseable_rate_improvement | 0 |
| eggroll_mean_shaped_epoch_fraction | 0.75 |
| eggroll_median_final_accuracy | 0.02083333395421505 |
| eggroll_success_rate | 0.5 |
| epochs | 4 |
| eval_samples | 24 |
| mean_accuracy_gap_decomposed_minus_eggroll | 0 |
| mean_improvement_gap_decomposed_minus_eggroll | 0 |
| mean_parseable_gap_decomposed_minus_eggroll | 0 |
| num_pairs | 4 |
| num_ranks | 2 |
| num_seeds | 2 |
| num_target_modules | 7 |
| population | 8 |
| rank_1_decomposed_success_rate | 0.5 |
| rank_1_decomposed_win_rate | 0 |
| rank_1_eggroll_success_rate | 0.5 |
| rank_1_mean_accuracy_gap_decomposed_minus_eggroll | 0 |
| rank_2_decomposed_success_rate | 0.5 |
| rank_2_decomposed_win_rate | 0 |
| rank_2_eggroll_success_rate | 0.5 |
| rank_2_mean_accuracy_gap_decomposed_minus_eggroll | 0 |
Interpretation
Run 20260801-145407-pretrained_llm_gsm8k_reward_smoke_jean_zay-0cabfb5 is a
useful plumbing check, not evidence for a decomposed advantage. Both methods
stayed finite and improved from the zero-accuracy baseline on the GSM8K subset,
with high parseability, but their final accuracy and improvement were tied.
The decomposed variant reached a slightly better best checkpoint during
training, while the paired win rate stayed flat, so the fair conclusion is that
the GSM8K reward path works and remains too small/noisy for a method comparison.
The boring alternative explanation is reward granularity. This subset is small,
the model is tiny, and exact GSM8K correctness moves in coarse jumps. The run
does not rule out sampling noise or a too-small reward budget.
The next useful experiment should keep the real GSM8K reward path but increase
statistical resolution: more evaluation examples, more seeds, and enough reward
budget to see whether either method consistently moves beyond parseability.