2026-07-31

• 09:14 project created
• 09:33 framed the ICLR delta paper around DoRA-style magnitude/direction ES for long-context reward adaptation and marked the first experiment as a PyTorch ES smoke test
• 09:37 added six requested arXiv papers to the bibliography and filled draft relation notes for ES, LoRA, GRPO geometry, forgetting, and low-rank adapter expressivity
• 09:51 audited the submitted PyTorch EGG-ROLL zip against the JAX HyperscaleES reference; main EggRoll LoRA scaling looks consistent, but OpenES and EggRollBS are not faithful enough to use as baselines without fixes
• 10:05 ran scratch CPU PyTorch smoke checks for the submitted EGG-ROLL port; core EggRollLinear forward, EggRoll update scaling, and fitness normalisation passed noiser-level tests, while OpenES and EggRollBS remain known-unfaithful variants
• 10:23 added a reusable local validation harness for the submitted PyTorch EGG-ROLL port so future runs can check forward perturbations, update scaling, and fitness normalisation before GPU experiments
• 10:29 updated the starting-point note with the PyTorch EGG-ROLL validation harness, baseline equations, known implementation gaps, and the decomposed ES target equations
• 10:35 expanded the starting-point note to explain why low-rank EGGROLL perturbations can be applied through input/output vectors instead of a dense perturbation matrix
• 10:54 committed and pushed the vendored PyTorch EGG-ROLL port, validation harness, Jean Zay pre-XP config, and HyperscaleES reference link on branch eggroll-port-validation for review
• 11:08 added and pushed the first matrix-learning XP, which tests whether the vendored PyTorch EGGROLL update can learn a small low-rank target before any LLM dataset run
• 11:23 submitted 20260731-112220-eggroll_matrix_learning_jean_zay-b382988 (job 469022 on jean-zay, commit b382988)
• 11:26 fetched 20260731-112220-eggroll_matrix_learning_jean_zay-b382988
• 11:28 first Jean Zay matrix XP attempt failed before running because the job environment had no torch; fetched logs and avoided reporting stale locally copied metrics
• 11:29 submitted 20260731-112905-eggroll_matrix_learning_jean_zay-eda1048 (job 469301 on jean-zay, commit eda1048)
• 11:41 fetched 20260731-112905-eggroll_matrix_learning_jean_zay-eda1048
• 11:42 Jean Zay matrix XP run 20260731-112905-eggroll_matrix_learning_jean_zay-eda1048 failed before execution because the generated sbatch did not initialise module; patched the XP env to source /etc/profile first
• 11:46 submitted 20260731-114513-eggroll_matrix_learning_jean_zay-9987db5 (job 469742 on jean-zay, commit 9987db5)
• 11:46 resubmitted eggroll_matrix_learning on Jean Zay as 20260731-114513-eggroll_matrix_learning_jean_zay-9987db5 after initialising /etc/profile before module loads
• 11:46 fetched 20260731-114513-eggroll_matrix_learning_jean_zay-9987db5
• 11:47 fetched 20260731-114513-eggroll_matrix_learning_jean_zay-9987db5; it failed while sourcing /etc/profile under set -u, so wrapped profile initialisation with set +u/set -u
• 11:48 submitted 20260731-114826-eggroll_matrix_learning_jean_zay-0d0f3cd (job 469816 on jean-zay, commit 0d0f3cd)
• 11:48 resubmitted eggroll_matrix_learning on Jean Zay as 20260731-114826-eggroll_matrix_learning_jean_zay-0d0f3cd after wrapping /etc/profile initialisation for set -u
• 11:49 fetched 20260731-114826-eggroll_matrix_learning_jean_zay-0d0f3cd
• 11:51 fetched 20260731-114826-eggroll_matrix_learning_jean_zay-0d0f3cd; /etc/profile now loads, but Jean Zay has no pytorch-gpu/py3/2.3.0 module, so switched matrix XP to a login-side venv bootstrap
• 11:56 submitted 20260731-115158-eggroll_matrix_learning_jean_zay-5efd8d0 (job 470101 on jean-zay, commit 5efd8d0)
• 12:53 fetched 20260731-115158-eggroll_matrix_learning_jean_zay-5efd8d0
• 12:53 reported 20260731-115158-eggroll_matrix_learning_jean_zay-5efd8d0 in notes/experiments/eggroll_matrix_learning_jean_zay.md
• 12:54 reported 20260731-115158-eggroll_matrix_learning_jean_zay-5efd8d0; the matrix-learning smoke test passed and supports using the main PyTorch EGGROLL path as the first baseline candidate
• 13:11 added a separate src/decomposed_eggroll implementation for the DoRA-style decomposed EGGROLL variant and a separate matrix comparison runner, leaving the submitted src/dhyperscalees EGGROLL code distinct
• 13:44 changed the decomposed EGGROLL forward path to avoid materialising dense direction perturbations, using factorised output and row-norm computations suitable for the later LLM path
• 13:49 fixed the Jean Zay decomposed matrix comparison metadata before submission so it describes the factorised DoRA-style perturbation path accurately
• 14:07 submitted 20260731-140705-eggroll_vs_decomposed_matrix_learning_jean_zay-05ce937 (job 474445 on jean-zay, commit 05ce937)
• 15:08 fetched 20260731-140705-eggroll_vs_decomposed_matrix_learning_jean_zay-05ce937
• 15:09 reported 20260731-140705-eggroll_vs_decomposed_matrix_learning_jean_zay-05ce937 in notes/experiments/eggroll_vs_decomposed_matrix_learning_jean_zay.md
• 15:09 reported the eggroll_vs_decomposed matrix comparison run; both variants learned the controlled objective and the decomposed variant won this single-seed smoke test, so the next step should be a small seed/rank sweep before LLM experiments
• 15:45 implemented the matrix_robustness_sweep runner and Jean Zay config to compare plain EGGROLL, full decomposed EGGROLL, direction-only, and magnitude-only variants across seeds and ranks
• 16:28 submitted 20260731-162711-matrix_robustness_sweep_jean_zay-04fee3e (job 478890 on jean-zay, commit 04fee3e)
• 17:29 fetched 20260731-162711-matrix_robustness_sweep_jean_zay-04fee3e
• 17:29 reported 20260731-162711-matrix_robustness_sweep_jean_zay-04fee3e in notes/experiments/matrix_robustness_sweep_jean_zay.md
• 17:29 reported the matrix robustness sweep; full decomposed EGGROLL succeeded across seeds and ranks and beat plain EGGROLL in most paired comparisons, while direction-only and magnitude-only ablations failed, supporting a tiny LLM smoke test next
• 17:35 implemented tiny_llm_smoke using a randomly initialised HuggingFace Llama-style causal LM and synthetic next-token rewards to validate the LLM EGGROLL and decomposed EGGROLL code paths without storing checkpoints or datasets
• 17:44 submitted 20260731-174147-tiny_llm_smoke_jean_zay-3a903d5 (job 481366 on jean-zay, commit 3a903d5)
• 18:45 fetched 20260731-174147-tiny_llm_smoke_jean_zay-3a903d5
• 18:45 reported 20260731-174147-tiny_llm_smoke_jean_zay-3a903d5 in notes/experiments/tiny_llm_smoke_jean_zay.md
• 18:46 reported the tiny LLM smoke; both EGGROLL variants ran through a random Llama-style causal LM, patched target projection layers, stayed finite, and slightly improved synthetic NLL, so the next step is a pretrained small-model reward smoke
• 19:21 implemented pretrained_llm_logprob_smoke to test EGGROLL and decomposed EGGROLL on a small pretrained causal LM with generated arithmetic logprob rewards before any real dataset benchmark
• 19:28 submitted 20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982 (job 486520 on jean-zay, commit ccf6982)
• 19:29 submitted 20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982 (job 486520 on jean-zay, commit ccf6982) to test pretrained small-LM logprob optimisation for EGGROLL vs decomposed EGGROLL
• 20:40 fetched 20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982
• 20:40 reported 20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982 in notes/experiments/pretrained_llm_logprob_smoke_jean_zay.md
• 20:40 reported pretrained LLM logprob smoke run 20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982 after fetching completed Jean Zay job 486520; both variants stayed finite and decomposed EGGROLL had the lower final NLL in this synthetic reward smoke
• 20:41 updated PROJECT.md after the pretrained LLM logprob smoke completed, replacing the completed smoke-test next step with a seed sweep or tiny real reward benchmark
• 23:37 implemented pretrained_llm_robustness_sweep to compare EGGROLL and decomposed EGGROLL across a small seed/rank grid on the pretrained arithmetic logprob objective before moving to a real reward benchmark
• 23:43 submitted 20260731-234003-pretrained_llm_robustness_sweep_jean_zay-43386a3 (job 496073 on jean-zay, commit 43386a3)
• 23:43 submitted 20260731-234003-pretrained_llm_robustness_sweep_jean_zay-43386a3 (job 496073 on jean-zay, commit 43386a3) for the pretrained small-LM seed/rank robustness sweep