Design
Which solid notes this experiment tests, and how.
Results
20260826-093813-qwen3_4b_gsm8k_fixed_batch_reward_signal_negative_jean_zay-620a35d
commit: 620a35d
| metric | value |
|---|---|
| center_eval_do_sample | 0 |
| center_eval_every | 5 |
| center_eval_temperature | 1 |
| center_eval_top_k | 0 |
| completed | 1 |
| condition_0_best_center_eval_accuracy | 0.3828125 |
| condition_0_center_eval_accuracy_change | 0.0234375 |
| condition_0_center_train_accuracy_change | 0.0476190485060215 |
| condition_0_condition_index | 0 |
| condition_0_final_center_eval_accuracy | 0.1484375 |
| condition_0_final_center_eval_parseable_rate | 1 |
| condition_0_final_center_train_accuracy | 0.095238097012043 |
| condition_0_final_center_train_parseable_rate | 1 |
| condition_0_initial_center_eval_accuracy | 0.125 |
| condition_0_initial_center_eval_parseable_rate | 1 |
| condition_0_initial_center_train_accuracy | 0.0476190485060215 |
| condition_0_initial_center_train_parseable_rate | 1 |
| condition_0_lr | -1 |
| condition_0_mean_fitness_std | 0.9027024000883103 |
| condition_0_mean_perturbed_train_accuracy | 0.09841270502656699 |
| condition_0_mean_perturbed_train_parseable_rate | 1 |
| condition_0_mean_raw_score_std | 0.28806543424725534 |
| condition_0_nonzero_reward_epoch_fraction | 1 |
| condition_0_num_layers | 253 |
| condition_0_rank | 1 |
| condition_0_seed | 0 |
| condition_0_shaped_epoch_fraction | 0 |
| eggroll_sigma | 0.001 |
| epochs | 20 |
| eval_batch_size | 8 |
| eval_prompts | 128 |
| generations_per_prompt | 6 |
| max_new_tokens | 1024 |
| noise_reuse | 4 |
| num_center_eval_records | 5 |
| num_completed_runs | 1 |
| num_epoch_records | 20 |
| num_lr_conditions | 1 |
| num_target_module_patterns | 8 |
| parseability_shaping | 0 |
| population | 126 |
| prompts_per_epoch | 21 |
| rank | 1 |
| seed | 0 |
| temperature | 1 |
| top_k | 0 |
| train_batch_size | 6 |
| wall_clock_seconds | 22194.486616202048 |
Interpretation
Run 20260826-093813-qwen3_4b_gsm8k_fixed_batch_reward_signal_negative_jean_zay-620a35d
completed the missing negative-learning-rate control from the fixed-batch
reward-signal diagnostic. The negative update direction did not reproduce the
large sustained center-model gain seen in the earlier positive-learning-rate
condition from
20260824-183902-qwen3_4b_gsm8k_fixed_batch_reward_signal_jean_zay-39c4380.
This supports the boring-but-important interpretation that the positive control
was seeing a usable exact-answer reward signal on that fixed batch, not merely a
generic effect of taking large EGGROLL steps or of parseability shaping.
Parseability was already saturated and shaping was disabled, so those are weak
alternative explanations here. The remaining caveat is that this is still one
fixed batch and one seed; it is a reward-signal diagnostic, not a robust
decomposed-versus-plain comparison.
Next, use this as permission to move from “is there any scalar reward signal?”
to a small repeated-seed or easier-curriculum test before interpreting method
differences.