Experiential memory

Mohammad Asadolahi and four co-authors posted "Memory Reward Inflation in Self-Improving LLM Agents" at the end of June ¹, and it changed how I read the other memory papers. They ran a text-to-SQL agent on BIRD in three configurations. With no memory it reached 52.4% execution accuracy. With a memory of past episodes that the agent scored itself, in the style of the case-based system Memento, it reached 54.0%. With the same memory corrected by their method, LUCID, which uses a signal independent of the agent's own grading, it reached 56.9% ¹.

Self-graded memory helped, then, but it captured only about a third of the gain that was available. The bias that made the agent write a wrong query also made it rate that query highly, and retrieval then served those confident mistakes first.

Two other papers from the summer make the case against memory more directly. CL-Bench, posted June 4, is the general-purpose benchmark the memory literature had been missing: six expert-validated domains (software engineering, signal processing, outbreak forecasting, database querying, strategic games, demand forecasting) whose tasks share latent structure an agent can learn online, scored with a gain metric that subtracts what the model already knew ².

Agents on CL-Bench overfit to immediate observations or fail to reuse what they learned, dedicated memory systems don't fix that, and naive ICL outperforms the systems built for memory management ². The second paper, an August re-evaluation of two memory-based self-improving methods, found their improvement "highly dependent on task order," with the default orders in prior papers acting as "an implicit curriculum" ³.

These results don't show that experiential memory fails. Systems published in August and September report double-digit gains over strong memory baselines, several under matched budgets and multiple seeds. What the negative results show is where the gains come from: the grader that decides what gets written, the decision about what to keep in raw form, and the policy for what to delete. Next to those three choices, the retriever is a minor component.

Experiential memory writes an improvement into non-parametric state, meaning anything outside the weights: raw trajectories, distilled insights, workflows, skills, playbooks, or explicit records of current state. The proposer is usually the model reading its own traces. The evaluator varies more than any other part of the design, and most of the failures below trace back to it.

Teams deploy text because it is cheap, readable, reversible and works on closed models. An insight costs one inference call to write, an engineer can read it, and deleting a line undoes it. Letta, the company that grew out of MemGPT, argued in December 2025 that "the weights are temporary; the learned context is what persists" ⁴, and pushed the claim further in June 2026: "While models are a depreciating asset, learning and memory represented in token space is an appreciating asset" ⁵. The portability results further down undercut that second claim, at least for memories written as prose.

The designs behind the 2026 systems go back only a few years. Reflexion (2023) passed verbal self-critiques from one attempt to the next ⁶, ExpeL (2023) extracted insights across tasks ⁷, and Agent Workflow Memory (2024) stored reusable procedures and improved relative success on WebArena by 51.1% ⁸.

Dynamic Cheatsheet (April 2025) let GPT-4o keep notes across a stream of problems and took it from about 10% to 99% on Game of 24, because the model wrote a Python solution once and reused it ⁹. ReasoningBank (ICLR 2026) distilled strategies from both successes and failures, but its labels came from an LLM judge instead of ground truth ¹¹, so each stored lesson is only as good as that judge's call on which episodes went well.

ACE (October 2025) found the first structural failure, and it is the older paper I would hand to anyone building one of these systems. When an LLM rewrites a whole context every round, it compresses away detail (brevity bias, in the paper's term) until the context collapses. In one run a playbook went from 18,282 tokens at 66.7 accuracy to 122 tokens at 57.1 in a single step, below the 63.7 the agent scored with no adaptation at all ¹⁰.

ACE's fix, itemized incremental updates with deliberate curation, became the default design. It left alone two earlier choices, both of which this year's papers revisit: each system still decided at write time what an experience meant, and most still let the model grade it.

The newest answer to "what should memory hold" is less processing than most systems do. Zhou and colleagues argued in September that write-time curation, turning each finished trajectory into a fixed reflection, workflow, skill or strategy, forces the system to choose what to keep "before the future query is known, irreversibly discarding information" ¹². Their Just-in-Time Memory (JitMem) keeps raw trajectories and waits until a new task arrives; a curator then reads the retrieved traces and writes a compact payload for that task ¹². Because the payload is used right away, the curator can be trained on the current task's success instead of a reward that arrives many tasks later.

JitMem beat no-memory agents and both heuristic and learned write-time memory methods, improving on the strongest baseline by 16.2 absolute success-rate points on ALFWorld, 16.3 on WebShop and 3.9 on τ²-bench ¹². The paper also reports that "even an untrained curator is already competitive with or surpasses these baselines" ¹², so most of the gain comes from curating at read time, and less of it from training the curator.

If a raw record plus a capable reader beats a pre-digested memory, the skeptical benchmark results make more sense. Naive in-context learning beating dedicated memory systems on CL-Bench ² is the extreme case: no curation at all, just the history in context. LongMemEval-V2's May result points the same way, with an off-the-shelf coding agent searching stored trajectories at 69.3%, against 72.5% for the best purpose-built system and 48.5% for the strongest retrieval baseline ¹³. The September portability study discussed below adds one more data point, since only agents that kept the raw source history could repair a damaged memory ¹⁴.

Tracking current state

Raw traces record what was said, and that is different from what is currently true. StateMemBench, from August, tested 234 multi-session scenarios in which facts, constraints and decisions get revised, and graded whether answers reflected the current state or a superseded one. Existing memory systems, retrieval baselines and long-context baselines all struggled ¹⁵.

A wrapper that tracks supersession explicitly lifted current-state accuracy by 32 to 67 points across six memory and retrieval backends, and under a control matched for length and cost, 15 to 32 of those points still came from the state structure and not from the extra context ¹⁵. For a memory system built now, I would keep the raw history as evidence and add a small, explicitly versioned record of current state for the agent to act on.

The Echo Gap

Back to BIRD. The Memory Reward Inflation paper calls the shortfall between self-graded and corrected memory the Echo Gap, and the mechanism is easy to follow once it is laid out ¹. The bias that caused a wrong answer also inflates the score the agent gives it. Every retrieval of a stored episode acts as a policy-improvement step whose reliability depends on that score, so inflated episodes steer future behavior toward past mistakes. A second LLM judge doesn't help, because its errors are correlated with the first grader's, and the shortfall grows whether memories are ranked by score or by similarity ¹.

The paper's central result is a theorem. It states an Error-Independence Assumption, that the correcting signal's errors must be independent of the errors that produced the episode, and proves the assumption is necessary, not only sufficient, for any correction ¹. LUCID works on BIRD because it has such a signal. In a deployment without one, the theorem says no algorithm recovers the loss.

On BIRD the damage is inflation, and memory still helped on net: self-graded memory beat no memory by 1.6 points and left 2.9 on the table ¹. That combination is why products can ship self-graded memory, measure a gain, and still end up with a store that drifts toward the model's own blind spots. A memory that only the agent grades cannot be fixed later by a better retriever or a second opinion from a similar model. The same problem appears in general form when self-generated rewards become the training objective, the subject of report 03.

Graders that stay attached to each memory

The first systems built around that constraint appeared in August. MemGuard ¹⁶ names two failure modes: "unreliable admission," where failed trajectories, accidental successes and misleading observations enter memory because they look relevant, and "memory drift," where long-running banks pile up duplicate, stale and conflicting records. Its fix is to treat verifier output "not as a one-shot filter, but as persistent lifecycle metadata": each candidate memory is stored with reward, confidence, label and uncertainty descriptors, which get reused at retrieval, conflict resolution, summarization and archival ¹⁶.

MemGuard ran on Terminal-Bench 2.0, SWE-Bench Verified, WebArena and Mind2Web across four backbones, against four memory baselines and a verifier-only control, under matched runtime budgets and averaged over five seeds, and it had the best success rate and fewest steps in all 16 backbone-benchmark settings ¹⁶. Its margin over ReasoningBank, the strongest baseline, peaked at 7.9 success-rate points on WebArena and ran 2.4 to 3.5 points on the terminal and software-engineering benchmarks ¹⁶. Those numbers are smaller than several others here, and I trust them more, because few earlier memory papers reported matched budgets or multiple seeds.

Recuris ¹⁷ applies the same idea of an outside check to skills. A working memory tracks task progress and picks skills from an experiential store, so each execution becomes evidence that points to the specific memory component that failed; a fixed meta-agent then makes "localized, validation-gated updates to Skill Memory" ¹⁷.

Across four long-horizon benchmarks and 10 models, Recuris improved success in 35 of 37 completed model-benchmark pairs, adding 17.8 points to GPT-5.6 Sol and 15.6 to Claude Opus 5 on τ-bench (taking Opus 5 to 87.9%), with the margin widening to 32.2 points on the longest tasks ¹⁷. For memory, the relevant detail is that a validation check approves each update, and the model's opinion of its own run plays no part.

A deployment-style study compares graders directly. In "Learning on the Job," The Memory Company ran a frozen agent on τ-bench's banking domain and turned each episode's outcome into a natural-language rule ¹⁸. Against a static-retrieval control, a one-bit success-or-failure signal lifted single-trial success to 1.6 times baseline. Human corrections lifted it to 2.6 times and converted 22 of 84 tasks the baseline never solved ¹⁸. Storage and retrieval were identical in both arms, so the whole difference came from the quality of the signal that wrote the rules.

JitRL, from NUS (January 2026, an ICML 2026 spotlight), explains why a well-graded store works at all. It retrieves past trajectories similar to the current state, estimates each candidate action's advantage from what followed it, and adds those advantages to the model's output logits before sampling ¹⁹.

The authors prove that this additive update is the exact closed-form solution to the KL-constrained policy optimization objective, the objective that KL-regularized fine-tuning optimizes with gradient steps ¹⁹. For a policy that has to stay close to its base model, retrieval plus a logit shift is the exact optimum. JitRL set a new state of the art among training-free methods on WebArena and Jericho and beat the fine-tuned WebRL agent at more than 30 times lower monetary cost ¹⁹.

Read next to the Echo Gap paper, JitRL makes the grading problem more pressing. The update is only as good as the stored advantages, so inflated rewards produce an exactly optimal step in the wrong direction ^1,19. Between them, the two papers put a memory loop's weak point in the reward attached to each stored episode. JitRL also has a practical catch: it needs logit access, which means open weights or an API that exposes per-token logit biases ¹⁹, so the one method in the family with an optimality proof runs on only a small share of deployed agents.

Stores that only grow get worse, so every long-running agent eventually summarizes or deletes. Three papers from late August and September measured what that costs.

The Compaction Cliff paper tested Claude Code's /compact prompt on Sonnet 4.6 across 20 production agent configurations. It kept 53% of safety rules after one compaction round and 10% after five ²⁰. ACE's brevity bias is showing up in a production tool: a safety rule and an episodic log compete for the same tokens and get summarized at the same rate, but only the rule needs its exact wording to stay enforceable.

The authors' fix, Knowledge Triage, classifies each line of an agent's knowledge base by type and gives each type its own retention policy. Its compactor kept 2 to 4 times more safety rules than the strongest single-shot LLM compactor at every compression ratio, with 96% recall over five rounds ²⁰.

Eviction does more damage, because it is often irreversible. A September audit on LongMemEval-S took each wrongly answered question, put the gold evidence back, and reran the reader. Under top-k retrieval at an 80k-token budget, 67% to 73% of the errors that restoration fixed were irreversible for FIFO, random and redundancy-aware eviction, and 60% for LLM-judged importance. At an 8k budget, all of them were ²¹. For those errors, once the entry was evicted the information was gone.

Stale entries are the third problem. Invalidation contracts ²² attach version stamps to cached fixes learned from API errors, so a client can evict entries after server-side drift without trial and error. Whether a stamp is valid is deterministic, but whether the planner uses the fixes that remain depends on the model: the same bytes produced 100% first-try compliance on Claude Haiku 4.5 and 11% or less on Claude Sonnet 5 ²². Pruning, like grading, has model-specific behavior that has to be tested on each model you run.

A keyword sweep of arXiv turns up about 65 agent-memory preprints posted between early July and September 23, 2026. I rate the papers on graders (MemGuard, Recuris), on keeping the store clean (compaction, eviction, invalidation, state tracking), and on evaluation (the fragility study, DolphinBench, portability) above the rest. The table collects the main numbers. On their home benchmarks, the gains are large and cheap to buy.

The biggest gains in the table come from systems where something other than the agent grades what gets stored: validation for Recuris, a verifier for MemGuard, task success for JitMem's curator, human corrections for Learning on the Job. Evaluation also got better in August. MemGuard ran five seeds under matched runtime and StateMem controlled for length and cost. Both controls are needed because, as the fragility study found, stacking a self-improving loop on a noisy agent evaluation "can further amplify this noise" ³. The table can't say how these systems do on a general benchmark, because the systems at the top of it haven't been run on CL-Bench yet.

The August fragility study challenges the headline numbers in this literature directly ³. The authors re-ran two memory-based self-improving methods with multiple seeds and shuffled task orders. The loop amplified evaluation noise, and improvement depended heavily on order, because the default orderings encode a curriculum that the methods rely on without saying so. Manual inspection of the memories pointed to underspecified tasks and environments; adding detailed rubrics and environment feedback to memory construction only partly closed the gap ³. After reading it, I treat any memory result from a single run in a fixed task order as weak evidence.

Memory also struggles on general benchmarks besides CL-Bench. EvoMemBench (May 2026) evaluated 15 memory methods against long-context baselines under one protocol and found that "long-context baselines remain highly competitive, memory helps most when the current context is insufficient or tasks are difficult, and no single memory form works consistently across all settings" ²⁴. LongMemEval-V2's generic coding agent nearly matched the best memory system ¹³.

DolphinBench, released in September by a team that includes Mem0's founders, scores memory through task completion on histories of about 500k tokens per persona, checks that each task succeeds with the relevant history and fails without it, and requires cost and latency to be reported alongside accuracy ²⁵. Taken together, the benchmarks now ask a memory system to beat long-context ICL, on tasks that need the history, at a stated cost, and home-benchmark results rarely meet that standard.

An agent later reads anything it writes as instruction, so the store is an attack surface. MINJA (NeurIPS 2025) showed that an attacker who can only send queries can inject malicious records into a memory bank ²⁶.

Karunanidhi's August study measured the damage with false assertions written as ordinary statements: poisoning 1.2% of a LongMemEval corpus cut accuracy from 0.850 to 0.300. A four-stage write-time screening pipeline that caught 83% of indirect prompt injections rejected 0 of the 360 poisoned memories ²³. The author's diagnosis is the Echo Gap's argument applied to security: "distinguishing a false assertion from a true one generally requires external grounding beyond the text itself" ²³.

Letta's claim that token-space memory appreciates while models depreciate assumes a new model can read an old model's notes and benefit. A controlled study from September tested that on 48 synthetic histories, with two open-weight models under 10 billion parameters ¹⁴. After a writer swap, a fixed-schema knowledge graph transferred almost perfectly (accuracy change +0.0004). Model-written natural-language notes did not: accuracy moved by +9.91 or −13.28 percentage points depending on the direction of migration. Repairing the notes from the store alone failed to reach 90% recovery in all 48 cases, while keeping the raw source history allowed recovery in 34 of 48 for one direction ¹⁴.

The models are small and the histories synthetic, so I wouldn't lean on the exact numbers. The direction matches every other transfer result in this series, including Hugging Face's July harness-transfer finding that "prompt playbooks are model-specific and can backfire" while code and structure transfer ²⁷, and it qualifies Learning on the Job's report that its natural-language rules helped across two models ¹⁸. My reading is that structured records and raw history are what to keep through a model upgrade, and prose lessons are something to regenerate for each new model.

The token-space camp now accepts a role for weights. Letta's June post proposed "memory models," trained with memory-native reinforcement learning to write and curate token-space memories that transfer across model generations ⁵. To make context learning work, in other words, Letta proposes training the weights of the component that writes the context. MemoPilot (ICML 2026) does this concretely, training a memory-update model with multi-turn GRPO while the player model stays frozen ²⁸, and JitMem's trained curator is a third example ¹².

Consolidation into weights is also coming from the other direction. Experience Distillation (July) found that distilling from an experience-conditioned teacher kept 64.8% of the in-context gain on software-engineering tasks, while supervised fine-tuning on the same experience kept 3.8% ²⁹. Dual-Layer Agentic Memory (August), explicitly modeled on complementary learning systems, sends incoming information through a small-to-large model cascade that pruned up to 68% of redundant memory while keeping over 98% of the exhaustive baseline's exact-match accuracy, then fine-tunes high-value memories into the weights ³⁰.

On the product side, Anthropic's Claude Code team said in July that it removed about 80% of Claude Code's system prompt for its newest models ³¹. The designs are converging on fast learning in context and slower consolidation into weights, and the open argument, which report 12 follows across both camps, is about when to move a lesson from one to the other.

Shipped memory has settled on files, audit logs and rollback, which is the storage layer the research recommends, while grading is mostly left to the customer. Anthropic's memory for Claude Managed Agents, in public beta since April 23, 2026, "mounts directly onto a filesystem" so the agent uses its ordinary bash and code tools; every change is tracked "with a detailed audit log," and teams "can roll back to an earlier version or redact content from history" ³². Rakuten, quoted in the announcement, claims its agents deliver "97% fewer first-pass errors at 27% lower cost and 34% lower latency" ³², a customer quote with no published evaluation behind it.

Letta moved the same way in March 2026, replacing its specialized memory-editing tools with bash over git-backed files it calls MemFS ³³. A filesystem plus a general agent is the product version of the research finding that raw records and a capable reader beat bespoke memory machinery.

Two products put a test or a human in front of each write. Anthropic's June post on dynamic workflows in Claude Code describes improving CLAUDE.md by mining sessions and review comments for repeated corrections, clustering them, adversarially checking whether each candidate rule "would have prevented a real mistake," and distilling only the rules that pass ³⁴. Decagon's Autopilot, launched June 9, 2026, turns signals from production conversations into proposed agent updates, runs them against regression tests and the original conversation, and stages them for human review ³⁵. Both are experiential memory with an independent grader built in, which is the condition the Echo Gap theorem says a correction needs.

Which cheap production signals can grade a memory? The Echo Gap theorem says a self-graded store can't be de-inflated without a signal whose errors are independent of the agent's ¹, and MemGuard shows that persistent verifier metadata beats strong baselines when a verifier exists ¹⁶. Most deployments have only cheap, partial signals: a follow-up message, whether an edit persisted, whether a ticket reopened. Whether any of these is independent enough to grade memories is untested. Until then, teams can't tell whether production memory can close its own loop, or whether every durable lesson needs a test or a human signature, as the shipped loops assume.

When should a lesson move from memory into weights? Experience Distillation shows consolidation can keep most of an in-context gain or almost none, depending on the objective ²⁹, and Dual-Layer Agentic Memory shows a working prune-then-consolidate pipeline on QA ³⁰. What's missing is a rule for when a lesson is stable enough to consolidate, a test that the weights absorbed it, and a way to delete the absorbed context without losing the audit trail. Teams on closed models can't consolidate at all; they depend on the next model having learned what their memory taught the last one.

Which stored unit holds up under shift? Home-benchmark gains come from repeated structure. CL-Bench and EvoMemBench ask for learning under heterogeneity, and memory loses ^2,24. The portability study says structured records transfer across writer models and prose notes don't ¹⁴, and the fragility study says task order alone can make or break a result ³. I would rather see one controlled comparison of stored unit (raw trace, state record, skill, prose lesson) against distribution shift, task order and model upgrade, at matched budget against long-context ICL, than another memory architecture.