Epoch AI’s CVE data insight reports that high- and critical-severity vulnerabilities from 21 major orgs jumped more than 3.5x in June, right after Anthropic said Claude Mythos Preview could autonomously find bugs and Project Glasswing partners had already surfaced 10,000+. The number is loud. What it actually measures is the interesting part.

Three different mechanisms produce the same curve, and a disclosed-CVE count can’t separate them: the model made bugs cheaper to find, more researchers piled into vuln hunting because the tooling got good, or this is disclosure lag on finds that already existed. Epoch flags the confounders directly — public CVEs miss the thousands claimed-but-undisclosed, and rising interest alone would bend the line. Same graph, three stories, very different answers to whether defenders are ahead or behind.

This is the eval trap I hit whenever a capability gets automated: the metric that’s cheap to collect (disclosures) sits two steps removed from the thing you care about (net exposure). If an agent can file vulnerabilities at scale, “CVEs found” measures throughput, not security. The signal you’d actually want — time-to-patch, exploitation in the wild, attacker dwell time — is slower and harder to instrument, so it never makes the chart.

The HN discussion splits along the same fault line: half read the spike as offense pulling ahead, half as the disclosure pipeline finally clearing a backlog. Both are consistent with the data, which is exactly the problem.

If autonomous discovery is real now, the honest scoreboard isn’t bugs found — it’s whether mean-time-to-remediation moved at all. My bet is it didn’t, and a year from now we’ll be arguing over a patch-velocity chart instead. What’s your leading indicator that offense-automation is helping defense?