Almost every “self-improving agent” I’ve seen learns after the run is dead — you harvest the trajectory, distill lessons, update the harness, and hope the next task benefits. PILOT in the Loop makes the sharper bet: improvement should be live. Use the experience emerging mid-run both to redirect the run that’s still executing and to update the persistent harness. That distinction sounds academic until you’ve watched an agent spend 40 turns confidently walking off a cliff you could see coming from turn five.
The architecture is a supervisor-worker split, and it’s the right one. Single-agent self-correction crams execution and self-assessment into one context, where they compete for attention; plain subagent delegation separates them but can’t reach back in to steer an active worker. PILOT gives a separate supervisor two levers: live steering (redirect or abort the worker during execution) and live self-evolution (distill procedures and failure modes into reusable skills and memory as they surface). This is squarely the multi-agent orchestration problem — who watches the worker, and can the watcher intervene before the trajectory is sunk cost.
The numbers are what make it worth a real read. On Terminal-Bench 2.0 they report up to a 9.8-point edge over counterpart harnesses, plus 12-15 points of self-improvement gain on frozen backbones. But the operational metric is the one I’d take to a latency-and-cost review: mean output tokens down ~43-47%, and successful evaluations per million output tokens more than doubled. Abort-early supervision isn’t just accuracy — it’s the token budget.
If you run long-horizon agents, the takeaway is that your supervisor probably can’t interrupt, and that’s a design bug, not a missing feature. Read the method on arXiv and the HF paper page. Then go check: when your agent starts failing at minute two, does anything have the authority to stop it?