Design
Which solid notes this experiment tests, and how.
Results
20260801-175851-pretrained_llm_gsm8k_reward_sweep_jean_zay-993bd00
commit: 993bd00
| metric | value |
|---|---|
| both_success_rate | 0.3333333333333333 |
| decomposed_mean_accuracy_improvement | 0.006944444961845875 |
| decomposed_mean_best_accuracy | 0.034722222636143364 |
| decomposed_mean_final_accuracy | 0.024305556279917557 |
| decomposed_mean_final_greedy_accuracy | 0.029513889613250893 |
| decomposed_mean_final_greedy_parseable_rate | 0.9947916865348816 |
| decomposed_mean_final_parseable_rate | 0.9739583432674408 |
| decomposed_mean_finite_epoch_fraction | 1 |
| decomposed_mean_greedy_accuracy_improvement | 0.001736111007630825 |
| decomposed_mean_greedy_parseable_rate_improvement | -0.0017361044883728027 |
| decomposed_mean_parseable_rate_improvement | 0.00173609455426534 |
| decomposed_mean_shaped_epoch_fraction | 0.75 |
| decomposed_mean_train_accuracy | 0.015625 |
| decomposed_mean_train_parseable_rate | 0.9674479166666666 |
| decomposed_median_final_accuracy | 0.02083333395421505 |
| decomposed_success_rate | 0.3333333333333333 |
| decomposed_win_rate | 0 |
| eggroll_mean_accuracy_improvement | 0.008680556124697128 |
| eggroll_mean_best_accuracy | 0.034722222636143364 |
| eggroll_mean_final_accuracy | 0.026041667442768812 |
| eggroll_mean_final_greedy_accuracy | 0.029513889613250893 |
| eggroll_mean_final_greedy_parseable_rate | 0.9965277910232544 |
| eggroll_mean_final_parseable_rate | 0.9774305721124014 |
| eggroll_mean_finite_epoch_fraction | 1 |
| eggroll_mean_greedy_accuracy_improvement | 0.001736111007630825 |
| eggroll_mean_greedy_parseable_rate_improvement | 0 |
| eggroll_mean_parseable_rate_improvement | 0.005208323399225871 |
| eggroll_mean_shaped_epoch_fraction | 0.8541666666666666 |
| eggroll_mean_train_accuracy | 0.013020833333333334 |
| eggroll_mean_train_parseable_rate | 0.95703125 |
| eggroll_median_final_accuracy | 0.02083333395421505 |
| eggroll_success_rate | 0.3333333333333333 |
| epochs | 8 |
| eval_samples | 96 |
| mean_accuracy_gap_decomposed_minus_eggroll | -0.0017361111628512542 |
| mean_greedy_accuracy_gap_decomposed_minus_eggroll | 0 |
| mean_greedy_improvement_gap_decomposed_minus_eggroll | 0 |
| mean_improvement_gap_decomposed_minus_eggroll | -0.0017361111628512542 |
| mean_parseable_gap_decomposed_minus_eggroll | -0.0034722288449605307 |
| num_pairs | 6 |
| num_ranks | 2 |
| num_seeds | 3 |
| num_target_modules | 7 |
| population | 16 |
| rank_1_decomposed_success_rate | 0.3333333333333333 |
| rank_1_decomposed_win_rate | 0 |
| rank_1_eggroll_success_rate | 0.3333333333333333 |
| rank_1_mean_accuracy_gap_decomposed_minus_eggroll | 0 |
| rank_1_mean_greedy_accuracy_gap_decomposed_minus_eggroll | 0 |
| rank_2_decomposed_success_rate | 0.3333333333333333 |
| rank_2_decomposed_win_rate | 0 |
| rank_2_eggroll_success_rate | 0.3333333333333333 |
| rank_2_mean_accuracy_gap_decomposed_minus_eggroll | -0.0034722223257025084 |
| rank_2_mean_greedy_accuracy_gap_decomposed_minus_eggroll | 0 |