Design
Which solid notes this experiment tests, and how.
Results
20260803-234737-pretrained_llm_gsm8k_qwen05_scale_smoke_jean_zay-ddff3bd
commit: ddff3bd
| metric | value |
|---|---|
| both_success_rate | 0 |
| decomposed_grid_0_lr_direction | 0.01 |
| decomposed_grid_0_lr_magnitude | 0.25 |
| decomposed_grid_0_mean_accuracy_gap_decomposed_minus_eggroll | -3.104408582051595e-10 |
| decomposed_grid_0_mean_final_accuracy | 0.013888888992369175 |
| decomposed_grid_0_success_rate | 0 |
| decomposed_grid_0_win_rate | 0.3333333333333333 |
| decomposed_lr_direction | 0.02 |
| decomposed_lr_direction_0_mean_accuracy_gap_decomposed_minus_eggroll | -3.104408582051595e-10 |
| decomposed_lr_direction_0_mean_final_accuracy | 0.013888888992369175 |
| decomposed_lr_direction_0_success_rate | 0 |
| decomposed_lr_direction_0_value | 0.01 |
| decomposed_lr_direction_0_win_rate | 0.3333333333333333 |
| decomposed_lr_magnitude | 0.02 |
| decomposed_lr_magnitude_0_mean_accuracy_gap_decomposed_minus_eggroll | -3.104408582051595e-10 |
| decomposed_lr_magnitude_0_mean_final_accuracy | 0.013888888992369175 |
| decomposed_lr_magnitude_0_success_rate | 0 |
| decomposed_lr_magnitude_0_value | 0.25 |
| decomposed_lr_magnitude_0_win_rate | 0.3333333333333333 |
| decomposed_mean_accuracy_improvement | -0.010416666977107525 |
| decomposed_mean_best_accuracy | 0.0243055559694767 |
| decomposed_mean_final_accuracy | 0.013888888992369175 |
| decomposed_mean_final_greedy_accuracy | 0.013888888992369175 |
| decomposed_mean_final_greedy_parseable_rate | 1 |
| decomposed_mean_final_parseable_rate | 0.9965277910232544 |
| decomposed_mean_finite_epoch_fraction | 0 |
| decomposed_mean_greedy_accuracy_improvement | -0.0034722223257025084 |
| decomposed_mean_greedy_parseable_rate_improvement | 0 |
| decomposed_mean_parseable_rate_improvement | -0.0034722089767456055 |
| decomposed_mean_shaped_epoch_fraction | 0.6666666666666666 |
| decomposed_mean_train_accuracy | 0.0078125 |
| decomposed_mean_train_parseable_rate | 0.703125 |
| decomposed_median_final_accuracy | 0.010416666977107525 |
| decomposed_success_rate | 0 |
| decomposed_tangent_project_direction | 0 |
| decomposed_win_rate | 0.3333333333333333 |
| eggroll_mean_accuracy_improvement | -0.010416666666666666 |
| eggroll_mean_best_accuracy | 0.027777778295179207 |
| eggroll_mean_final_accuracy | 0.013888889302810034 |
| eggroll_mean_final_greedy_accuracy | 0.013888888992369175 |
| eggroll_mean_final_greedy_parseable_rate | 1 |
| eggroll_mean_final_parseable_rate | 1 |
| eggroll_mean_finite_epoch_fraction | 0 |
| eggroll_mean_greedy_accuracy_improvement | -0.0034722223257025084 |
| eggroll_mean_greedy_parseable_rate_improvement | 0 |
| eggroll_mean_parseable_rate_improvement | 0 |
| eggroll_mean_shaped_epoch_fraction | 0.6666666666666666 |
| eggroll_mean_train_accuracy | 0.0078125 |
| eggroll_mean_train_parseable_rate | 0.81640625 |
| eggroll_median_final_accuracy | 0.010416666977107525 |
| eggroll_success_rate | 0.3333333333333333 |
| epochs | 4 |
| eval_interval | 0 |
| eval_samples | 96 |
| generations_per_prompt | 8 |
| mean_accuracy_gap_decomposed_minus_eggroll | -3.104408582051595e-10 |
| mean_greedy_accuracy_gap_decomposed_minus_eggroll | 0 |
| mean_greedy_improvement_gap_decomposed_minus_eggroll | 0 |
| mean_improvement_gap_decomposed_minus_eggroll | -3.104408582051595e-10 |
| mean_parseable_gap_decomposed_minus_eggroll | -0.0034722089767456055 |
| num_decomposed_lr_directions | 1 |
| num_decomposed_lr_magnitudes | 1 |
| num_pairs | 3 |
| num_ranks | 1 |
| num_seeds | 3 |
| num_target_modules | 7 |
| population | 64 |
| prompts_per_epoch | 8 |
| rank_1_decomposed_success_rate | 0 |
| rank_1_decomposed_win_rate | 0.3333333333333333 |
| rank_1_eggroll_success_rate | 0.3333333333333333 |
| rank_1_mean_accuracy_gap_decomposed_minus_eggroll | -3.104408582051595e-10 |
| rank_1_mean_greedy_accuracy_gap_decomposed_minus_eggroll | 0 |
| wall_clock_seconds | 2063.914653176442 |
Interpretation
Run 20260803-234737-pretrained_llm_gsm8k_qwen05_scale_smoke_jean_zay-ddff3bd completed after the H100 CPU-memory allocation retry. The metrics do not show a decomposed advantage over plain EGGROLL: final sampled accuracy is effectively tied, greedy accuracy is tied, and plain EGGROLL has the only positive success-rate signal. Treat this as successful Qwen smoke plumbing under the larger allocation, not as evidence that the scale-calibrated decomposed setting wins.