A robot is sprinting at you — do you want it running Claude or Grok? OpenRouter’s Jacky Liang turned that framing into an actual experiment, and the result is a clean argument against trusting leaderboards when you pick an agent model.
- 🎯 Eleven LLMs, one top-down battle royale, 30 games. Grok 4.1 Fast won 13 of 30 at $0.97 per win; Claude Sonnet 4.6 was next-best at $26.78 — a 27x cost-per-win gap.
- 🔪 GPT 5.4 logged the most kills (38 across 30 games) and still finished with two wins. Most aggressive did not mean best outcome.
- ⚠️ Three models — GPT 5.4-mini, DeepSeek 4 Flash, Kimi K2.6 — spent $57 between them and won exactly zero games.
- 🧠 Each model edited its own
soul.mdandmemory.mdbetween matches. The behavioral differences live in those files, not in the scoreboard. - 📊 None of the outcomes tracked the standard Artificial Analysis benchmarks. Whatever predicted winning wasn’t on any leaderboard.
- 💡 The model that kept asking everyone to team up and broadcasting its own position — Claude Sonnet 4.6 — is the one you’d actually want orchestrating a real agent. Cooperative loses the royale and wins the deployment.
The full data is worth a read before your next model-routing decision, and the HN discussion argues fairly about whether a game like this generalizes at all. But the core tension is real: if your eval can’t separate a model that wins from a model that cooperates, which one is your router quietly shipping to production?