Getting Qwen 3.6 27B to 80+ tok/s at Q8 on a mismatched RTX 5080 + RTX 3090 pair came down to motherboard arcana, not model tuning β€” which is the entire story of serious local inference. The iMil writeup reads like a model-performance post and is actually a plumbing post, and that inversion is the lesson.

This rhymes with production serving: latency and throughput get won in the layer below the model β€” interconnect, memory placement, quantization, batching β€” long before anyone swaps a checkpoint. The HN discussion has more dual-card setups, and they all confirm the unglamorous truth: the hard part of running an open model isn’t the model. So how much of your β€œLLM tuning” time last quarter was actually spent on firmware, drivers, and PCIe lanes?