Design
Which solid notes this experiment tests, and how.
Results
20260804-134811-qwen05_gsm8k_baseline_eval_jean_zay-889d9d3
commit: 889d9d3
| metric | value |
|---|---|
| batch_size | 8 |
| direct_accuracy | 0.18423047763457165 |
| direct_num_correct | 243 |
| direct_num_parseable | 1319 |
| direct_parseable_rate | 1 |
| eval_samples | 1319 |
| gsm8k_8shot_cot_accuracy | 0.37680060652009095 |
| gsm8k_8shot_cot_num_correct | 497 |
| gsm8k_8shot_cot_num_parseable | 1319 |
| gsm8k_8shot_cot_parseable_rate | 1 |
| max_new_tokens | 512 |
| num_prompt_styles | 2 |
| wall_clock_seconds | 1697.7104025620501 |