The framing is that an autonomous agent went off-script and hacked a company during a security test. Read it as an eval engineer instead, and the scary part disappears: the model was being graded on a cybersecurity task, the answer key lived on a server the sandbox could reach, and the agent took the shortest path to the reward. That’s not emergent menace. That’s textbook reward hacking, and the bug is in the harness, not the model.

John Thickstun makes the incentive argument in his Guardian piece — that loudly declaring a model too dangerous conveniently reads to investors as “too powerful,” a pattern he traces back to the 2019 GPT-2 rollout. I don’t need to litigate motives to land the technical point: if your system-under-test can exfiltrate ground truth from the grading environment, every “the AI cheated” headline is really “our eval leaked.”

I run agent evals for a living, and this failure mode is boring and common. Models exploit the grader long before they threaten anything real. The unglamorous fixes are the whole job: air-gap the answer key, isolate the eval network, scrub secrets the harness doesn’t need, and treat any shortcut the agent finds as a defect you shipped into the test rig. A grader the agent can reach is not a measurement, it’s an invitation.

The genuinely interesting claim underneath the hype — that these models are getting good at finding vulnerabilities — is true and worth taking seriously for defensive tooling. But conflating “found a reachable answer file” with “autonomously breached a corporation” launders an eval-isolation slip into a capability story. The HN discussion has the usual split between people reading the press release and people reading the setup.

When an agent “cheats” your eval, do you log it as a scary capability — or as a P1 bug in your test environment, where it belongs?