Empirical sketch proofs

Purpose

This note records the empirical sketch proof for the project so far: not a final claim, but the sequence of gates that makes the next real-reward experiment worth running. Each gate asks a narrower question than the paper claim and only licenses the next step.

The current claim supported by the evidence is modest:

the PyTorch EGGROLL path and the factorised DoRA-style decomposed path are operational, stable on controlled objectives, and encouraging enough to test on a tiny black-box reward benchmark.

It does not yet show that decomposed ES trains LLM agents better from sparse long-context rewards.

Implementation and validation ladder

1. PyTorch EGGROLL plumbing

We first vendored and isolated the submitted PyTorch EGGROLL port under src/dhyperscalees, then added src/validate_eggroll_port.py to check the mechanics that should hold independently of any task:

• low-rank perturbation forward pass matches the expected factorised matrix product;
• EGGROLL update sign and scaling are compatible with PyTorch optimiser descent implementing reward ascent;
• reward normalisation behaves as expected.

This established that noiser=eggroll is plausible as the plain LoRA-space ES baseline. The open_es and eggrollbs variants remain explicitly out of scope because they are not faithful enough to use as baselines yet.

2. Matrix-learning baseline smoke

Run 20260731-115158-eggroll_matrix_learning_jean_zay-5efd8d0 tested whether the plain PyTorch EGGROLL path could learn a tiny fixed low-rank linear map from reward-only perturbation feedback. It passed the smoke criterion: finite throughout, with loss reduced from 2.9565727710723877 to 0.203643798828125.

Interpretation: this validates update plumbing on a controlled objective. It does not say anything about LLM learning.

3. Single-seed decomposed matrix comparison

Run 20260731-140705-eggroll_vs_decomposed_matrix_learning_jean_zay-05ce937 compared plain EGGROLL with the factorised DoRA-style decomposed variant on the same matrix objective. Both succeeded. The decomposed variant finished at 0.09557401388883591; plain EGGROLL finished at 0.203643798828125, giving a decomposed-minus-plain gap of -0.10806978493928909.

Interpretation: one seed was encouraging, but too small to claim robustness. It justified a sweep.

4. Matrix robustness sweep

Run 20260731-162711-matrix_robustness_sweep_jean_zay-04fee3e compared plain EGGROLL, full decomposed EGGROLL, direction-only decomposed, and magnitude-only decomposed variants across 10 seeds and 4 ranks.

Key outcomes:

variant	success rate	mean final loss	median final loss
plain EGGROLL	0.65	1.91025531408377	1.3880925178527832
full decomposed	1.0	0.39386328402906656	0.26666003465652466
direction-only	0.0	4.286366328597069	3.7679615020751953
magnitude-only	0.0	4.3086372256278995	3.757124185562134

The full decomposed variant beat plain EGGROLL in 0.8 of paired comparisons. The ablations failed, suggesting that the useful behaviour comes from the coupled magnitude/direction parameterisation rather than either component alone.

Interpretation: this is the strongest empirical evidence so far, but it is still a synthetic matrix objective.

5. Tiny random-LM plumbing smoke

Run 20260731-174147-tiny_llm_smoke_jean_zay-3a903d5 moved from matrix code to a HuggingFace Llama-style causal-LM forward path, using a randomly initialised model and synthetic next-token NLL. Both methods patched 14 layers from 7 target module names, stayed finite, and slightly improved synthetic NLL.

Final NLLs were 4.177436351776123 for plain EGGROLL and 4.176585674285889 for decomposed EGGROLL.

Interpretation: this is implementation validation only. The model was random, and the objective was synthetic.

6. Pretrained small-LM logprob smoke

Run 20260731-192758-pretrained_llm_logprob_smoke_jean_zay-ccf6982 replaced the random model with a small pretrained causal LM and used generated arithmetic prompts with answer-token logprob/NLL objective.

Both methods stayed finite and improved. Plain EGGROLL went from 0.8726494312286377 to 0.7472426891326904; decomposed EGGROLL went from 0.8726505041122437 to 0.6978709697723389. The decomposed-minus-plain final NLL gap was -0.04937171936035156.

Interpretation: this made the pretrained model/tokenizer path viable, but it was still a supervised proxy objective in disguise: the update rewarded probability mass on a known answer string.

7. Pretrained small-LM robustness sweep

Run 20260731-234003-pretrained_llm_robustness_sweep_jean_zay-43386a3 repeated the pretrained generated-arithmetic logprob objective over 5 seeds and 2 ranks, for 10 paired comparisons.

Both variants had success rate 1.0. Decomposed EGGROLL had mean final NLL 0.8348180532455445; plain EGGROLL had mean final NLL 0.8384017646312714. The mean decomposed-minus-plain NLL gap was -0.0035837113857269285, with decomposed winning 0.6 of pairs. Rank 1 favored plain EGGROLL on average; rank 2 favored decomposed EGGROLL.

Interpretation: stable and mildly favorable, but not decisive. The result justifies moving to a real reward benchmark; it does not justify a strong win claim.

Current empirical conclusion

The project has cleared the following gates:

1. the plain EGGROLL PyTorch baseline can learn a controlled low-rank matrix objective;
2. the decomposed DoRA-style implementation can run without dense perturbation materialisation in the controlled path;
3. full magnitude+direction decomposition is robustly better than plain EGGROLL on the toy matrix sweep under the tested budget;
4. both methods run through causal-LM target-module patching;
5. both methods remain stable on a small pretrained LM under generated arithmetic logprob optimisation;
6. the latest pretrained sweep is encouraging but small.

The remaining gap is exactly the important one: we have not yet shown reward-only behavioural improvement. The next experiment should therefore be a tiny GSM8K-style or generated-arithmetic exact-answer reward benchmark where the model samples an answer and ES sees only a scalar correctness reward, not answer-token likelihood.

What this does and does not prove

Supports

• The implementation path is credible enough for the next stage.
• Decomposed magnitude/direction search can improve search behaviour on controlled low-rank objectives.
• The factorised decomposed path can be made compatible with causal-LM modules.
• There is no current evidence of numerical instability in the tested small settings.

Does not support yet

• Any claim about long-context agents.
• Any claim about real dataset learning.
• Any claim that decomposed ES beats plain EGGROLL on sparse or non-differentiable rewards.
• Any claim that the advantage will survive matched wall-clock, parameter-count, and reward-budget controls at larger scale.

Next falsifier

The next falsifier should be a tiny real-reward benchmark:

• prompts: generated arithmetic or a tiny GSM8K-style subset;
• generation: the model samples the answer;
• reward: scalar exact numeric correctness, optionally with minimal parseability shaping if the fully sparse reward is all zero;
• update signal: no token-level gold-answer NLL;
• comparison: plain EGGROLL vs decomposed EGGROLL under matched population, epochs, target modules, seeds, and reward budget.

If both methods get flat zero reward, the immediate result is not failure of the main idea; it means the first reward benchmark is too sparse and needs a shaped but still black-box intermediate. If plain EGGROLL clearly wins under this setup, that directly weakens the decomposed-geometry claim and should redirect the project toward diagnosing parameterisation or scaling controls.