Design
This page merges the LLM-side smoke tests. These runs check whether plain EGGROLL and decomposed EGGROLL can patch causal-LM target modules, stay finite, and optimise generated synthetic objectives before moving to a real reward benchmark.
The sequence was:
- random tiny Llama-style causal-LM plumbing smoke;
- small pretrained causal-LM generated-arithmetic logprob smoke;
- small pretrained causal-LM seed/rank logprob robustness sweep.
None of these is yet a reward-only behavioural benchmark. The pretrained arithmetic objective still rewards answer-token likelihood, not sampled answer correctness.
Results
20260731-174147-tiny_llm_smoke_jean_zay-3a903d5
commit: 3a903d5
| metric | value |
|---|---|
| eggroll_initial_nll | 4.177872180938721 |
| eggroll_final_nll | 4.177436351776123 |
| decomposed_initial_nll | 4.177872180938721 |
| decomposed_final_nll | 4.176585674285889 |
| nll_gap_decomposed_minus_eggroll | -0.000850677490234375 |
| eggroll_finite_epoch_fraction | 1 |
| decomposed_finite_epoch_fraction | 1 |
| num_target_modules | 7 |
20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982
commit: ccf6982
| metric | value |
|---|---|
| eggroll_initial_nll | 0.8726494312286377 |
| eggroll_final_nll | 0.7472426891326904 |
| eggroll_nll_improvement | 0.12540674209594727 |
| decomposed_initial_nll | 0.8726505041122437 |
| decomposed_final_nll | 0.6978709697723389 |
| decomposed_nll_improvement | 0.17477953433990479 |
| nll_gap_decomposed_minus_eggroll | -0.04937171936035156 |
| population | 16 |
| eval_samples | 32 |
| num_target_modules | 7 |
20260731-234003-pretrained_llm_robustness_sweep_jean_zay-43386a3
commit: 43386a3
| metric | value |
|---|---|
| num_seeds | 5 |
| num_ranks | 2 |
| num_pairs | 10 |
| eggroll_success_rate | 1 |
| decomposed_success_rate | 1 |
| eggroll_mean_final_nll | 0.8384017646312714 |
| decomposed_mean_final_nll | 0.8348180532455445 |
| mean_nll_gap_decomposed_minus_eggroll | -0.0035837113857269285 |
| eggroll_mean_nll_improvement | 0.1068449079990387 |
| decomposed_mean_nll_improvement | 0.11042933464050293 |
| decomposed_win_rate | 0.6 |
| rank_1_mean_nll_gap_decomposed_minus_eggroll | 0.0049618840217590336 |
| rank_2_mean_nll_gap_decomposed_minus_eggroll | -0.01212930679321289 |
Interpretation
The LLM tests cleared the intended engineering gates. Both methods ran through causal-LM target-module patching, stayed finite, and improved synthetic NLL objectives. The decomposed variant was slightly ahead in the random-LM smoke, clearly ahead in the single pretrained logprob smoke, and mildly ahead overall in the pretrained seed/rank sweep.
The sweep is deliberately interpreted conservatively. Both methods had success rate 1; decomposed EGGROLL won 0.6 of paired comparisons, with the advantage concentrated at rank 2 and not present at rank 1. This is encouraging stability evidence, not a strong empirical win.
The next experiment should stop using answer-token likelihood as the reward. A tiny GSM8K-style or generated-arithmetic exact-answer benchmark should sample model outputs and give ES only scalar correctness or minimally shaped parseability/correctness feedback.