KVarN, a new native vLLM backend for KV-cache quantization out of Huawei’s lab, claims 3-5x more cache capacity at FP16-level accuracy. The benchmark numbers matter less than one word in the README: calibration-free.

• 🎯 The bottleneck for agentic serving is the KV-cache, not the weights — multi-turn tool use and long context blow up cache memory per request and cap your concurrency.
• ⚡ 3-5x capacity, ~1.3x throughput over FP16 means you fit longer contexts and serve more concurrent requests on the same hardware.
• 🔌 Calibration-free, one flag is the operational headline — calibration passes are where most quantization schemes quietly die during rollout.
• 📊 FP16-level accuracy is the claim to verify on your own eval harness, not theirs — quantization quality is workload-dependent and your traffic is not their benchmark.
• 🔍 Native vLLM attention backend lands where serving already lives, so there’s no separate inference stack to babysit.
• ⚠️ “Up to” throughput numbers earn the usual skepticism — P50 in the slide, P99 in production, and your traffic shape decides which you actually get.

The HN discussion is mostly arguing the accuracy-versus-capacity tradeoff, which is the right axis to fight over. My bet: as agent context windows keep growing, KV-cache quantization stops being an optimization and becomes table stakes for anyone serving multi-agent workloads at scale. When did your cache become the thing you provision for first?