The finding that should bother anyone running agent memory: on this benchmark, every memory strategy scored worse than turning memory off. Most memory evals ask whether the right fact was stored and retrieved. MemTrapBench asks the harder question — once a faithful, relevant memory is in context, what does it do to the model’s reasoning? Across two model families and five memory frameworks, every strategy underperformed the no-memory baseline, and even the strongest still dropped more than 10%.

The paper names two failure modes: Reasoning Fixation and Belief Distortion. They’re the same trap from different angles — a correctly retrieved memory anchors the model to a prior frame that no longer fits the current question. This is the part RAG teams learn the hard way. We pour eval budget into retrieval precision and recall, and spend almost none on whether the retrieved context helps or hurts once it’s actually in the prompt. A semantically perfect chunk can still drag a fresh question toward a stale conclusion, and precision@k will never show it.

Their mitigation, AdaptiveMem, is an inference-time instruction telling the model to watch for memory traps — no retraining, no new index — and it recovers performance while holding standard memory-benchmark scores. That’s the practical signal: the problem is context conditioning, not storage. The HF paper page has the per-framework breakdowns, and they matter more than the headline average — if you run a memory layer in production, the real question isn’t which framework retrieves best, it’s whether you can even measure this failure on your own traffic.

If your memory system has never been A/B’d against having no memory at all, how sure are you it isn’t quietly making some answers worse?

tags: [ rag ] [ llm-ops ] [ research ]