“No single memory substrate wins” reads like a hedge until you see what the arXiv evaluation actually measured: the same design choice that helps one workload actively hurts another, inside the same agent.
- 🎯 More retrieval isn’t better — broad retrieval lifts long-context factual QA, but the same breadth degrades sequential decision-making by pulling attention off the action-critical context.
- 🔍 Substrate isn’t a storage footnote — dense and sparse indices, text records, hierarchical and structural stores, refinement-based memory, parametric updates: each is a different bet on what the agent needs to remember.
- ⚡ Horizon moves the winner — a substrate that’s efficient at moderate history length turns costly or brittle as the trajectory grows, so the right choice shifts partway through a session.
- 📊 26 metrics, one harness — across three backbone models and four benchmark suites spanning user-facing QA and agent-centric decisions; efficiency numbers carry as much weight here as accuracy.
- 💡 Routing is the real conclusion — the takeaway is that adaptive agents need substrate routing: picking the memory medium by operating regime, the way we already route models and tools.
This maps onto a problem I keep hitting: teams treat agent memory as one component — “add a vector store” — when the HF paper page argues that memory is a routing decision, not a fixed dependency. The retrieval stack that makes your RAG QA sing can quietly wreck a long-horizon planning agent by burying the one piece of state that mattered.
If substrate routing really is necessary, what’s the routing signal — task type, horizon length, or something the agent has to learn online?