The trap in building agent training pipelines is measuring data by the gigabyte. This survey reframes agentic data generation as a distribution-design problem: the goal isn’t more trajectories, it’s continually allocating valid, informative, non-redundant experience as your agents and their environments drift.
The framing I’ll steal is the factorized object — every piece of agentic data is an environment spec, a task signal, an interaction realization, and an optional verifier. Pulling those four apart is the whole point, because most pipelines conflate candidate construction with verification and selection: they generate a trajectory and a success label in one breath, then wonder why the model learned to satisfy the reward instead of the task. Separating the realization from the verifier is exactly the discipline a production eval harness already enforces; the paper argues training data deserves the same rigor.
The organizing lens is ACE — Accuracy, Complexity, divErsity. Accuracy sets the feasible support: execution-grounded, internally consistent data, not plausible-looking transcripts. Complexity places learning mass relative to a declared learner and its execution config — difficulty is only meaningful against the specific model you’re training, which is why a fixed “hard” set ages badly. divErsity controls coverage and redundancy beyond surface variation. The useful part for practitioners is that this gives you a vocabulary for auditing a data mix instead of trusting the row count.
Read through a production lens, this is a checklist for the data side of an agent stack: is my accuracy execution-grounded or annotation-grounded? Is my difficulty calibrated to the current model or last quarter’s? Am I paying to relabel near-duplicate rollouts? The HF paper page organizes the generation paradigms by their anchor and dependency structure.
If difficulty only means something relative to the learner, does a static “golden” agent benchmark start decaying the moment your base model improves — and would you even notice it happen?