Authors: Xin Qiu and Yulu Gan and Conor F. Hayes and Qiyao Liang and Yinggan Xu and Roberto Dailey and Elliot Meyerson and Babak Hodjat and Risto Miikkulainen
Cite as: \cite{qiu2025evolution}
arXiv: https://arxiv.org/abs/2509.24372
Summary in three sentences
This paper argues that ES can scale to billion-parameter LLM fine-tuning without dimensionality reduction. The arXiv abstract frames ES as a backpropagation-free alternative to RL fine-tuning, with claimed advantages for delayed rewards, reward hacking, robustness, and stability. For this project, it is an important comparison point because our method keeps the ES motivation but restricts the search to decomposed low-rank adapter coordinates.
Main contribution
The main contribution is a case for full-parameter ES as a viable LLM post-training method beyond reinforcement learning.
Key equations and definitions
The project-relevant definition is full-parameter ES over LLM weights, contrasted with our adapter-only ES search space. The paper should be read before deciding whether to frame decomposed ES as a cheaper alternative, a safer alternative, or a geometry-improving alternative.
Empirical results
Every number here must name the section or figure it comes from.
If you cannot locate it in the paper, do not write it.
The arXiv abstract reports LLM fine-tuning experiments comparing ES with established RL implementations. No exact benchmark numbers are recorded here until the relevant sections are read directly.
Relation to our project
- what we reuse: the claim that ES is relevant for LLM post-training and delayed reward optimisation.
- what we dispute: full-parameter ES may be unnecessary or too disruptive when adapter-space methods can capture the needed task signal.
- what we must reproduce: a matched small-scale comparison showing whether adapter ES can preserve the useful ES behaviour without full-parameter updates.
Criticism and limitations
For our purposes, full-parameter ES is a strong but expensive baseline. It may also raise forgetting and drift concerns that motivate staying inside a structured adapter parameterisation.