The scarce input for training useful agents isn’t model capacity or even data — it’s realistic, verifiable environments. Every RL or eval setup I’ve built hits the same wall: someone hand-crafts a sandbox per task, and it never generalizes past the tasks you thought to write down. AgentMercury, in this arXiv paper, flips the construction order.
Instead of building an environment around a predefined task, it instantiates a persistent world first — entities, services, tools, state, and executable cross-service invariants — and lets diverse tasks and trajectories emerge from that world. They construct 4,783 executable environments across 14 industries and 50 countries and use them as reinforcement-learning substrates. The transfer result is the interesting part: policies trained on these business worlds, without targeting any benchmark, improve on enterprise workflows and out-of-domain — Qwen3.5-4B goes 12.3 to 15.7 on EnterpriseOps-GYM and 45.9 to 56.0 on AIME26. Scenario-grounded environments carry generalizable signal, not just benchmark-specific overfitting.
The line I keep rereading: construction itself is learnable. Fine-tuning on construction traces lifts executable-world authoring success from 3.3% to 83.3% on held-out business scenarios. That’s the real unlock — if generating verifiable enterprise worlds becomes a model capability instead of a human bottleneck, the supply of training and eval environments stops being the constraint. The HF paper page has the full breakdown.
Here’s the question I can’t put down for anyone running enterprise agents: once environments are cheap to synthesize, does your eval moat shift from “who has the harness” to “who can verify the invariants”? My bet is that correctness of the generated invariants becomes the whole ballgame — a synthesized world that quietly permits an illegal state transition trains a confidently wrong agent.