2026-08-04

• 00:50 fetched 20260803-234737-pretrained_llm_gsm8k_qwen05_scale_smoke_jean_zay-ddff3bd
• 00:50 reported 20260803-234737-pretrained_llm_gsm8k_qwen05_scale_smoke_jean_zay-ddff3bd in notes/experiments/pretrained_llm_gsm8k_qwen05_scale_smoke_jean_zay.md
• 00:50 fetched and reported Qwen2.5-0.5B GSM8K scale smoke retry 20260803-234737-pretrained_llm_gsm8k_qwen05_scale_smoke_jean_zay-ddff3bd after Jean Zay job 563143 completed; read metrics before interpreting it as successful Qwen smoke plumbing but not evidence for a decomposed advantage
• 10:12 flagged Qwen2.5-0.5B GSM8K smoke accuracy as suspect because the reported 1-2 percent baseline conflicts with expected leaderboard-scale accuracy; likely causes are non-leaderboard prompting, sampled eval, and first-integer answer parsing, so run a baseline-only protocol check before interpreting ES effects
• 13:49 submitted 20260804-134811-qwen05_gsm8k_baseline_eval_jean_zay-889d9d3 (job 576547 on jean-zay, commit 889d9d3)
• 13:49 submitted no-ES Qwen2.5-0.5B GSM8K baseline sanity eval 20260804-134811-qwen05_gsm8k_baseline_eval_jean_zay-889d9d3 on Jean Zay as job 576547 after two bootstrap-only quoting failures; this checks whether the previous Qwen smoke accuracy was an eval protocol artifact