| status | note | summary | updated |
|---|---|---|---|
draft | Evolutionary Strategies lead to Catastrophic Forgetting in LLMs | Warns that ES post-training can cause catastrophic forgetting through large dense updates. | 2026-07-31 |
draft | A Little Rank Goes a Long Way: Random Scaffolds with LoRA Adapters Are All You Need | Studies how much task information low-rank adapters can carry over frozen random backbones. | 2026-07-31 |
draft | Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training | Shows ES and GRPO can match accuracy while reaching different update geometries. | 2026-07-31 |
draft | LoRA: Low-Rank Adaptation of Large Language Models | Introduces LoRA, the low-rank adapter baseline for parameter-efficient LLM adaptation. | 2026-07-31 |
draft | DoRA: Weight-Decomposed Low-Rank Adaptation | DoRA separates pretrained weight magnitude from direction and applies LoRA-style updates to direction. | 2026-07-31 |
draft | Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning | Argues that full-parameter ES can scale to billion-parameter LLM fine-tuning. | 2026-07-31 |
draft | Evolution Strategies at the Hyperscale | EGGROLL structures ES perturbations as low-rank matrices to improve large-scale black-box optimisation. | 2026-07-31 |
draft | On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent | Compares OpenAI ES with SGD through gradient-correlation and proxy experiments. | 2026-07-31 |
draft | Empirical sketch proofs | Empirical evidence ladder from EGGROLL port validation to matrix and small-LM synthetic objectives. | 2026-08-01 |
draft | Experiment: llm_tests_jean_zay | Merged LLM smoke tests and pretrained small-LM logprob sweep for EGGROLL vs decomposed EGGROLL. | 2026-08-01 |
draft | Experiment: matrix_tests_jean_zay | Merged matrix smoke tests and robustness sweep for plain EGGROLL vs decomposed EGGROLL. | 2026-08-01 |
draft | Starting point | Draft framing and baseline equations for DoRA-style decomposed ES on long-context reward adaptation. | 2026-08-01 |
In one paragraph
This project asks whether evolution strategies can become a credible tool for adapting LLM agents from long-context, reward-only feedback by searching in a better low-rank parameterisation. The working idea combines EGGROLL-style low-rank ES perturbations with a DoRA-style weight decomposition, where magnitude and direction are adapted separately. The implementation has now passed matrix and small-LM synthetic gates; the next decisive step is to replace answer-token likelihood with scalar exact-answer reward.
Why it matters
Long-context agent training often produces feedback that is sparse, delayed, noisy, or hard to differentiate through. Gradient-free methods such as ES are attractive in that setting, but naive search in high-dimensional adapter coordinates can be inefficient or unstable. If magnitude/direction decomposition improves the effective search geometry, the project could give a simple empirical and theoretical reason to use decomposed adapters for reward-driven LLM adaptation.
Approach
The baseline is plain LoRA-space EGGROLL: freeze the base model and perturb low-rank adapter parameters. The proposed method keeps the low-rank ES estimator but searches in a DoRA-like parameterisation, separating output-channel magnitude from directional low-rank updates. The empirical ladder is deliberately conservative: matrix plumbing, matrix robustness, causal-LM plumbing, pretrained small-LM logprob optimisation, then tiny real-reward tasks before any long-context agent benchmark.
Current findings
The matrix tests are the strongest positive signal so far. Plain EGGROLL learned the controlled low-rank matrix objective, and full decomposed EGGROLL was more reliable in the seed/rank sweep; direction-only and magnitude-only ablations failed. The LLM tests show that both methods can patch causal-LM modules, stay finite, and improve synthetic NLL objectives on random and pretrained small models. The latest pretrained sweep was stable and mildly favorable to decomposed EGGROLL, but it is still generated arithmetic logprob optimisation, not reward-only behavioural learning.
See Empirical sketch proofs, matrix tests, and LLM tests.