Authors: William Hoy and Binxu Wang and Xu Pan
Cite as: \cite{hoy2026matching}
arXiv: https://arxiv.org/abs/2604.01499
Summary in three sentences
This paper compares ES and GRPO for LLM post-training and asks whether similar task performance implies similar parameter-space solutions. The arXiv abstract reports that ES and GRPO can reach comparable accuracy while producing geometrically different updates, including broader off-task drift for ES. For this project, it is directly relevant to the theory story: decomposed adapter ES should be evaluated not just by reward, but also by update geometry and drift.
Main contribution
The main contribution is a geometry-focused comparison of gradient-free ES and gradient-based GRPO in LLM post-training.
Key equations and definitions
The relevant concepts are update direction, off-task KL drift, linear connectivity between solutions, and near-orthogonality of ES and GRPO update directions. The paper may provide theory language for explaining how ES can move in weakly informative directions while still improving reward.
Empirical results
Every number here must name the section or figure it comes from.
If you cannot locate it in the paper, do not write it.
The arXiv abstract reports comparisons across single-task and sequential continual-learning settings. Exact task counts, accuracies, KL values, and geometric measurements should be copied only after reading the corresponding paper sections.
Relation to our project
- what we reuse: update geometry and off-task drift as evaluation dimensions for ES.
- what we dispute: if decomposed ES reduces uncontrolled drift, then not all ES variants should be treated as geometrically equivalent.
- what we must reproduce: at least one small diagnostic measuring update norm, direction, and off-task behaviour for LoRA ES versus decomposed ES.
Criticism and limitations
The paper compares ES with GRPO, not LoRA-space ES with DoRA-style ES. It motivates the risk we need to measure rather than resolving whether decomposition helps.