We pour a lot of effort into hand-tuning agent prompts, tools, and workflows, then freeze the scaffold around the model and never touch it again. Hierarchical Self-Improvement asks what happens if the harness itself becomes the thing that learns — while the model stays frozen.
- 🎯 The model never changes. A single frozen LLM plays three roles: a task harness that executes, an evolver that rewrites that harness, and a meta-evolver that rewrites the evolver’s own strategy code.
- ⚡ Thinking on for self-modification, off for execution. A clean way to isolate whether gains come from harness evolution or just more reasoning at task time.
- 📊 The gains are real on moderate tasks: +39 on BabyAI, +33 on Crafter, +25 on TextWorld over the initial harness, plus strong generalization on held-out splits.
- ⚠️ Two honest bounds. A feedback-fidelity bound (no informative reward, no evolution) and a backbone-capability bound (harness rewrites can’t exceed the frozen model).
- 🔍 On tasks past the backbone’s ceiling (NLE), it buys nothing — the paper reports zero improvement, which is the most useful result in it.
What I like is the framing: the harness is a tunable surface, not a fixed artifact you ship once. For production agents that’s the cheaper axis — you can evolve orchestration, retries, and tool wiring against real environment feedback without retraining anything. The HF paper page links the code. The catch is feedback fidelity: most production tasks don’t hand you a clean reward signal, so the real work isn’t the evolver — it’s building an environment honest enough to evolve against.