Everyone shipping GUI agents already knows they’re brittle. The useful result in AnTrap isn’t that 16 leading models all crumble under runtime anomalies — it’s that the failures split cleanly into ones you can train away and ones you can’t.
The benchmark injects perturbations into live agent trajectories — pop-ups, action misuse, dead states — organized into a four-layer taxonomy (State, Thinking, Action, Round) with ten subcategories, while keeping every task still solvable. Then it runs GRPO training in both clean and adversarial environments to see which failures the model can actually learn out of. That last step is what makes this more than another “agents are fragile” leaderboard.
- 🎯 Single-step traps are learnable. State- and action-layer perturbations mostly close with adversarial RL — throw trap-rich trajectories at the policy and it adapts.
- ⚠️ Deep contextual traps aren’t. State deadlock and similar multi-step failures are reasoning-bottlenecked; training in trap-filled environments doesn’t resolve them.
- 🔍 This is a triage map, not a verdict. It tells you where robustness effort pays off and where you’re wasting compute trying to RL your way out.
- 🛠️ Recovery is a design problem, not a data problem. For the intrinsic failures you need planning, explicit recovery policies, or an escape hatch — not more adversarial samples.
- 📊 Universal degradation means no model is your out. Even the strongest agents drop hard, so “pick a better base model” isn’t the reliability strategy.
- 💡 The taxonomy is reusable. Four layers × ten subcategories is a checklist you can port into your own eval harness before shipping an agent that touches real UIs.
The framing maps directly onto the retries-and-fallbacks work anyone running production agents already does: some anomalies deserve adversarial training data, and some deserve a guardrail that detects the deadlock and hands control back. The HF paper page has the full taxonomy. So which of your agent’s failures are you still trying to fine-tune away when they’re really a recovery-design gap?