The framing in AMD’s MI450 kernel guide is the part worth internalizing, more than the silicon itself. Agentic workloads have pushed a single request toward a million tokens — aggregated from long prompts, tool calls, retrieval results, and multi-turn reasoning — and that changes what the decode kernel is actually bound by. Every generated token attends to the entire KV cache, so decode is dominated by repeatedly reading past states out of HBM. The bottleneck has moved from the compute units to the memory system.

That’s why MI450’s spec sheet reads the way it does. HBM bandwidth jumps from 8.1 to 19.6 TB/s and capacity from 288 to 432 GB, but the more interesting addition is TDM: a hardware unit that issues descriptor-based bulk async tensor transfers into LDS, replacing the many small vector loads MI350 needed. Add workgroup clusters with hardware barriers and multicast loads, and you get a memory subsystem shaped for exactly the read-heavy pattern long-context decode generates. AMD’s optimized Gluon kernel reportedly hits 85% of peak HBM bandwidth as an early result.

Gluon itself is the quieter story — a Triton-based DSL that forces explicit tensor layouts, trading Triton’s ergonomics for the register-and-lane control kernel experts want on new hardware. Anyone running long-context agents in production already knows that retrieval and tool-call context is what blows up the KV cache; this is the hardware side of that same bill. The HN discussion is thin, but the guide is a clean look at where inference optimization is heading. If decode is now HBM-bound, how many of the next round of “model” speedups are really kernel-and-memory speedups wearing a model’s name?