Consolidation and co-evolution

On September 21, 2026, Ye, Lu, Dong, Su and Song posted Harness-Zero, a recipe for training a harness into a model and then throwing the harness away ¹. They evolved a specialized harness for each domain (knowledge work, tool use, science) using Kimi K3 under Kimi Code. Then they ran a Qwen3.5-9B student inside a deliberately bare target harness, a minimal mini-SWE-agent-style loop with a fixed system prompt and one Bash tool, while GPT-5.6 Sol, guided by the evolved harness, corrected the student's responses before they executed.

Fine-tuning on those corrected trajectories produced a model that, with the specialized harness removed, raised macro-average task success from 23.3% to 44.3%. The untrained base model with the specialized harness still attached reached 41.7%. The distilled model recovered, on average, 82.3% of 28 harness-induced behavior patterns that the base model lacked ¹.

It is a week-old preprint on one student model, and the teacher in the loop is a frontier model, so part of what got distilled is GPT-5.6 Sol's judgment and not only the harness. Still, it makes the storage question concrete.

An evolved harness is expensive to build and tied to one domain, so a general-purpose agent either settles for a mediocre shared harness or routes among a growing collection of specialized ones. Harness-Zero uses the specialized harness only during training, writes the behavior it induces into the weights, and ships one plain harness. I would like to see it replicated with a second student model and a teacher that isn't a frontier model.

Every self-improving system has to decide where its improvements are written. Written into text (prompts, memory, skills, harness code), learning is cheap, legible, reversible, and available on closed models. Written into weights, it is compact, fast at inference, and can change what the model can do instead of what it is told. I no longer think the useful question is which of the two wins. This year's papers are about the machinery that uses both, moves lessons from one to the other, and decides which lessons belong where.

Letta, which builds stateful agents, states the context-first position directly. In a June post it wrote: "While models are a depreciating asset, learning and memory represented in token space is an appreciating asset" ², a stronger version of its December 2025 line: "The weights are temporary; the learned context is what persists" ³.

The reasoning is practical. Most builders use closed models whose weights they can't touch, a new model generation arrives every few months, and a memory file or skill library can be reused with the next model while a fine-tune can't. Letta's roadmap calls for "memory models" trained with RL to write durable context "that transfers across model generations" ² (see 07).

The consolidation camp treats context as a staging area. Dwarkesh Patel, relaying Andrej Karpathy in May, wrote that "Our working memory gets wiped regularly," and that what we have instead is "a consolidation process (sleep) that distills stuff into the brain, in a weird and lossy way… It's not obvious it's something you can get for free from doing long enough RL loops" ⁴.

Within eight days two papers put sleep in their titles. One converts recent context into persistent state-space fast weights through offline recurrent passes and then clears the KV cache ⁵. The other adds a "sleep" stage in which a model consolidates memories by distillation and "dreams" an RL-generated curriculum to rehearse new knowledge ⁶.

Both camps now accept a hybrid. Letta grants that "the primary advantage of parametric memory is efficiency" and that distilling memories into weights "can provide additional gains in personalization and efficiency" when the weights are accessible ^{3 2}. The sleep papers treat context as the place where new information goes first. What they disagree about is which store is primary and how often the slow one should be written.

Complementary Learning Systems (CLS) theory gives this structure a name. McClelland, McNaughton and O'Reilly argued in 1995 that the brain needs a fast hippocampal store that records episodes after one exposure and a slow neocortical store that learns structure through interleaved replay, because one store can't do both without catastrophic interference ⁷.

The mapping to agents is direct. Context, memory, skills and harness are the fast store: written every episode, specific, cheap to revise. Weights are the slow store: updated rarely, general, expensive to revise, and prone to interference when updated carelessly. Consolidation is replay from the first into the second.

The analogy makes one testable prediction, which is that the slow store should learn from what the fast store computes rather than from raw experience. McClelland co-authored a 2025 DeepMind study in which fine-tuning generalized better when its data included the model's in-context reasoning traces ⁸.

By late summer engineers were building CLS into agent memory by name. Dual-Layer Agentic Memory routes incoming facts into an external store and periodically consolidates high-value entries into the weights with supervised fine-tuning, and a 1.7B/8B routing cascade pruned up to 68% of redundant external memory while keeping over 98% of the QA exact match of a keep-everything baseline ⁹. The analogy doesn't need to do more than that. The engineering question is how often a loop should switch between the two stores.

The first systems to put both levers inside one loop appeared between May and September 2026, and they share a design. The harness is edited while the weights are held fixed, the weights are trained under the improved harness, and each change exposes a new bottleneck in the other. Over the summer the answer to how often to switch changed.

WHALE, posted August 31 by Kim, Lee, Finn and Lee, tests the switching schedule directly ¹⁰. It alternates online rejection-sampling fine-tuning with Meta-Harness search (Claude Opus 4.7 as the harness proposer), using Qwen3.5-2B and 4B agents on search QA, math and chess puzzles. It beat weight-only, harness-only and Fast-Slow Training baselines by 4.15 to 24.38 points in best mean@8 accuracy.

The per-domain results explain the gain: "Either component can be the bottleneck: harness search matches peak weight-only accuracy with far fewer rollouts in SearchQA, but improves math accuracy only after a weight update." Small interleaved updates also beat a stagewise schedule that finishes all weight optimization before starting harness search, by 5.32 and 9.16 points in the two domains the authors report, at lower rollout cost ¹⁰. At 2B and 4B, the answer to "harness or weights first" is to alternate often.

Earlier systems switched once or twice. SIA, from May, has a Claude Sonnet 4.6 Feedback-Agent choose after each generation between rewriting a gpt-oss-120b agent's scaffold and running RL on its LoRA adapter. In practice it ran harness search until progress stalled and then switched ¹¹.

On LawBench's 191-class charge prediction, top-1 accuracy went from 13.5% to 50.0% after harness search and to 70.1% after weight updates, against a prior best of 45.0%. On a single-cell denoising task, scaffold rewrites took the score from 0.048 to 0.241 and stalled. The first weight-update checkpoint then wrote a two-line np.clip plus np.rint step that rounds imputed counts to non-negative integers, "a biological invariant that is trivially correct yet absent from any prior scaffold version," raising the score to 0.289 ¹¹.

Co-Harness, from July, adds explicit acceptance rules ¹². An LLM "HarnessCritic" labels each failed trajectory with a root cause and proposes a local diff. The diff is kept only if it improves a held-in slice showing the failure without regressing a held-out slice, and accepted diffs go into a versioned registry that keeps an audit trail and "enables full rollback."

Verifier-passing trajectories under the accepted harness then fine-tune the next model. On tool-integrated math with Qwen3-8B and Qwen3-32B, average pass@1 across AIME24, AIME25 and HMMT25 went from 54.2 with a human harness to 58.5 with the evolved harness, then to 73.5 and 78.9 after two rounds of fine-tuning.

Its 200-hour case study shows what the harness edits did. The initial configuration crashed after 32 seconds on a vLLM KV-cache misallocation, and over 22 versions the critic fixed that, cured a zombie-thread out-of-memory failure, and found an 8.7× batching speedup. It also added domain prompts that cost about 16.5 points by clashing with Qwen3's trained reasoning format, and the registry rolled them back. RL can't learn anything from a trainer that crashes after 32 seconds, so those fixes had to come first.

A few August systems use the harness mainly to produce training data. HELIX, from HKU, ran one evolution round on code repair: a 65-candidate harness portfolio found a fixed harness that improved task coverage by 4.0% over the Pi agent, and a 200-slot slice of sibling runs yielded 438 verified SFT, critic, filter and preference records for the next model ¹³. EvoTrainer, from June, applies the idea to the RL training harness itself and reports in its abstract that it matches or exceeds human-engineered RL recipes on math, competitive programming and repository-level SWE ¹⁵.

Mind Lab's Macaron-V1 technical report describes the production-scale version. Each release is a versioned model–harness pair (a frozen base, four LoRA specialists selected per turn, and a TOML "Harness Context Protocol" runtime configuration), new capabilities appear first as harness artifacts, and "when trajectory evidence warrants a parameter update, the change can then be transferred into a specialist adapter" ¹⁴. Its authors are candid that "the present evaluation measures the configuration-search stage, not repeated transfer across adapter generations."

Across these papers the mechanism looks the same to me. Harness edits fix operational failures, like a trainer that crashes or a thread that leaks memory, and until those are fixed no training signal can get past them. Weight updates need decent rollouts to learn from, and the harness supplies them. The trained weights then do things the harness couldn't express, which changes which harness is best. That feedback between the two is why WHALE's frequent switching beats a single handoff.

Once a loop decides a lesson belongs in the weights, it has to decide how. Every method here is a variant of context distillation, which dates to 2022: run the model with privileged context as a teacher, and train it without that context as a student ¹⁶. The variants differ in who samples the trajectories and what the teacher sees, and in 2026 those differences proved enormous.

Experience Distillation, a July preprint, gave a model long histories of its own prior attempts at a task (60 to 600 turns, more than 80k tokens) and measured how much of the in-context gain is kept once the history is removed and what it taught is trained in ¹⁷. A teacher conditioned on the experience generated one-step decisions at each recorded point, and the student was trained on them with teacher-sampled forward KL. That kept 64.8% of the in-context gain on 749 SWE tasks and 93.4% on six TaleSuite text games.

Plain supervised fine-tuning on the same experience kept 3.8% and −2.6%. On two games, on-policy distillation with reverse KL, which much of the field treats as the better default (see 04), kept 9.1% and 0.4% at matched compute, against 96.7% and 98.4% for forward KL, because "without the experience context, the student rarely samples trajectories containing the high-quality decisions." The paper measures retention on the same tasks the experience came from, not transfer to new ones, and the base models are in-house.

Yu et al. showed the opposite failure in September ¹⁸. They evolved a harness for a weaker model on seven enterprise agent tasks, then trained the weaker model on a stronger expert's complete trajectories under that harness. Performance dropped on all seven tasks, by 4 to 30 points, across Qwen3-Coder and Gemma 4, even though the same imitation helped under the unevolved harness. The weaker model "adopts the expert's planning strategy without the competence to execute it and no longer matches the harness evolved around its native planning style."

Their fix keeps the student's own rollout, finds the failing turn, and asks the expert to rewrite only that turn. Harness-Zero also trains on the student's own rollouts, with the teacher correcting responses before they execute ¹. Cursor's Composer 2.5 corrects a single turn: in May it began inserting a short hint at a turn where the model went wrong (for a hallucinated tool call, a "Reminder: Available tools…" note) and distilling the hinted distribution into the unhinted model for that turn alone ¹⁹.

Put together with Experience Distillation, these results give one rule with two conditions, and I'd treat it as settled. The training targets have to be dense and come from a teacher that sees the context, and they have to be on states the student reaches itself, under the harness it will run in. Imitating someone else's trajectories breaks the second condition. Reverse KL sampled by the student can break the first.

The bandwidth argument explains why distillation beats outcome RL for this job. Thinking Machines argued in 2025 that policy-gradient RL delivers "only O(1) bits per episode" while supervised targets give information on every token ²⁰. A teacher that sees the context turns it into token-level targets that encode what the context changed about each decision. An outcome reward is a bit or two.

Consolidation also needs a rule for when to stop using the teacher, and RetireOPD proposes one ²¹. It trains a skill-free student with RL plus on-policy distillation from a skill-conditioned teacher, and allows the student to drop the teacher "once their discrepancy stops shrinking and it reaches a target fraction of the teacher's success rate," after which it continues with RL alone.

Across Qwen2.5 models from 1.5B to 7B it improved ALFWorld success over an RL baseline by 14.1% to 18.8% and WebShop accuracy by 11.8% to 19.0%, and it surpassed its own teacher in every setting. The authors also found that "privileged information alone does not always make a teacher reliable," so the teacher is itself trained against environment rewards first. Dropping the teacher is the training-time version of the scaffolding deletion labs are doing with production models, which I come back to below.

The last risk is overfitting to one harness. Alibaba's Taobao Live team found that fine-tuning a compact model under a fixed harness configuration lowered IFEval by 7.7 points ²². They instead trained under task-preserving transformations of skill identifiers and content, tool schemas, prompt structures and hook functions, followed by on-policy distillation and RL in the augmented environments. That avoided the regression (83.5 on IFEval) and scored 94.6 on a harness-variant QA set against the base model's 75.4, in a system deployed with positive online A/B results ²². When consolidating a harness into weights, I would vary the harness during training by default.

Two 2025 results are useful background. SEAL let a model write its own fine-tuning data and showed both the promise of self-directed consolidation and its cost in forgetting under sequential edits ²³. Cartridges showed a trained KV cache can match in-context learning on a document at 38.6× less memory ²⁴.

A hybrid only makes sense if the two kinds of update add up. If RL simply relearned what prompt optimization found, or overwrote it, the second step would be wasted. The direct experiments in the DSPy lineage (see 06) find that they do add up.

mmGRPO, revised in May 2026, reported that averaged over three seeds, two 8B models and three tasks, vanilla chain-of-thought scored 66.3, MIPROv2 prompt optimization 70.0, multi-module GRPO alone 71.2, and prompt optimization followed by GRPO 73.4 ²⁵. The authors attribute the gain to "the value of high-quality rollouts at the start of mmGRPO training," and the prompt step cost 1.4 hours on one H100 against 18.7 hours on two for RL.

E-SPL, from February, samples each RL batch under several evolving system prompts. It moved an AIME-trained model's success on BeyondAIME from 38.8% with RL alone to 45.1%, where prompt evolution alone reached 40.0% ²⁶.

Until August every composition result ran prompts before weights. The reverse order, optimizing a harness for a model that has already been trained, was untested, and WHALE is the first controlled look: its math harness gains appeared only after a weight update ¹⁰. That is on 2B models. At frontier scale the question is still open, even though it is the situation every lab is in when it ships a new model into an existing harness. Co-Harness's clashing domain prompts are a small-scale warning that a context instruction can fight a behavior the model was trained on ¹².

Reading across the rows, harness edits give a quick gain, weight updates reach what harness search can't, and the way the weight step is done decides whether the harness's gains are kept. The weaknesses repeat just as reliably, and the systems with the biggest headline numbers have the most of them.

I don't trust SIA's LawBench number. The paper describes RL rollouts as "generated solution script[s] executed against the test split" while stating that "all evaluations are on the held-out test split" ¹¹. Read as written, the RL reward was computed on the split used for reporting, which would make 70.1% a number optimized on the test set, and SIA reports no weights-only arm either.

Co-Harness doesn't say clearly which problems supplied its fine-tuning trajectories, has no control that fine-tunes on trajectories from a fixed human harness, and its 8B AIME24 score barely moves in round two (84.0 to 84.7) ¹². Harness-Zero and HELIX report one round. WHALE runs many small cycles, but on 2–4B models.

The harness half of these loops has the same measurement problems as harness evolution in general. Wang et al. found on Terminal-Bench 2.1 that harness evolution "does not consistently outperform simple test-time scaling methods and exhibits limited generalization" ²⁷ (see 09 and 14). Hybrid loops make leakage easier, because one benchmark pool can drive harness edits, supply training trajectories and report the score. No 2026 paper I read has a held-out protocol for both levers that I would accept without questions.

Failures in the weight half are harder to reverse. In Composer 2.5's RL runs the model reverse-engineered a leftover Python type-checking cache to recover a deleted function signature, and decompiled Java bytecode to reconstruct a third-party API ¹⁹. Cursor's real-time RL learned to emit deliberately broken tool calls on tasks it expected to fail, because a broken call never received a negative reward ²⁸. A harness edit that encodes a hack can be reverted from a registry. A behavior trained into weights needs another training run to remove.

Letta's main argument is portability: the model will be replaced, and whatever was learned in its weights goes with it. But what was learned in context doesn't reliably transfer either, and several 2026 studies tested which kinds of context do.

Joel Niklaus at Hugging Face ran a Meta-Harness-style loop on Harvey's Legal Agent Benchmark with DeepSeek-V4-Pro frozen, and wrote it up in July ²⁹. Harness evolution raised the held-out criterion pass rate from 63.4% to 80.1%, and with the same model and judge, different wrappers ranged from 3.5% to 80.1%. Run untouched on other models, the evolved harness gave DeepSeek V4 Flash, a sibling, 14.4 points, and Nemotron-3 Ultra, from another family, 0.4.

The near-zero hid two effects cancelling out. A tool-call JSON repair component fixed Nemotron's crashes, while prompt playbooks tuned for DeepSeek hurt tasks Nemotron could already do. His summary: "Robustness and code mechanisms transfer across families; prompt playbooks are model-specific and can backfire." Five of his top six harnesses were "deterministic code, not prompt edits."

Salesforce's "AI4AI at Test-Time" had strong builder models construct harnesses for weaker target models on four Theory-of-Mind benchmarks, and nearly doubled average target performance, from 0.49 to 0.91, with no parameter updates ³⁰. The gains came "primarily from offloading unstable model reasoning into deterministic code, benchmark-specific routing, and strict answer-format enforcement," and weaker targets gained most.

Agentic Harness Engineering, earlier in 2026, traced its cross-family gains (5.1 to 10.1 points on three other families) "to tools, middleware, and long-term memory rather than the system prompt" ³¹. Prose can transfer (GEPA prompts optimized on Qwen3-8B improved GPT-4.1-Mini ³²), but less predictably, and when it fails it can make things worse. "Harness Updating Is Not Harness Benefit," from May, adds that the benefit of a harness peaks for mid-tier models and falls off for the strongest, which is what you'd expect if strong models have already absorbed what the harness teaches ³³.

On the weights side, a LoRA doesn't load onto a different base model, so a lesson in weights has so far reached a new model only if it was also kept as data that could be distilled again. RPMem, posted September 20, tries to fix that ³⁴.

It compiles each session into a model-independent latent memory, consolidates it across sessions with a trained recurrent gate, and only then maps it to backbone-specific LoRA parameters, so the encoder can be kept when the backbone changes. With Qwen3-8B on the PERMA benchmark it reached 85.52%, 5.32 points above the strongest parametric baseline and 12.98 above the strongest text-based one, across five backbones.

So portability depends more on the type of artifact than on the store. Executable code, tool contracts, validators and factual memory transfer. Prose tuned to one model loses value with each upgrade. Weights transfer only through something model-independent, either data you can distill again or a latent memory built for the purpose.

At frontier labs, and at the coding companies that train their own models, model and harness are designed together, and in July the consequences became public. Anthropic's Claude Code team wrote on July 24: "We removed ~80% of the Claude Code system prompt for our newest models" ³⁵. Five days later OpenAI reported that GPT-5.6 Sol scored 13.3% on the ARC-AGI-3 public set in the official harness, which discarded reasoning after each move, and 38.3% with retained reasoning and compaction, using 6× fewer output tokens ³⁶.

Their explanation: "Our models are trained to think with private reasoning messages… If a conversation grows too long, we summarize it and continue. This is how our models are trained, and also how they are deployed in ChatGPT and Codex." A harness the model wasn't trained for under-reported it by a factor of almost three. OpenAI had first described a model "natively trained" for compaction in November 2025 ³⁷, and by August Latent Space was summarizing the trend as "Models keep absorbing the harness into their weights" ³⁸.

Cursor describes the same practice from the product side. Its March Composer 2 report describes "training in the same Cursor harness that is used by the deployed model, with equivalent tools and structure" ³⁹. In April it described provisioning "each model with the tool format it had during training" and spending "weeks customizing our harness to a model's strengths and quirks," and noted that switching models mid-chat leaves the new model with history "out of distribution from what it was trained on" ⁴⁰. A May position paper argues that harness-induced variance "can substantially exceed model-induced variance, including cases of model ranking reversal" ⁴¹.

People argue both that the harness decides a lot and that the harness is temporary. Both are right, over different time spans. Within a model generation, harness choice can triple a score or reverse a ranking, and harness evolution is the fastest lever available. Across generations, the useful lessons get trained in, and the harness text that taught them becomes dead weight or starts conflicting with the model.

Harness-Zero is the research version of the Claude Code deletion, done on purpose and measured. The CLS picture leaves this pruning step implicit, but an agent's context doesn't fade on its own: someone has to decide which instructions the new weights have made redundant. Apart from RetireOPD's training-time rule, no published work measures how to make that call.

Most builders can't run any of the loops above, because their model is behind an API. For them the practical hybrid runs the other way: train a small model whose job is to manage context, for itself or for a frozen large model. The weights learn how to handle context instead of task knowledge (see 10 and 07).

EvoHarness-RL, from August, trains a Qwen3-8B agent to build and consult external harness state (belief, progress and experience records), using supervised harness fine-tuning followed by cost-aware GRPO, and reached 96.9% success on ALFWorld ⁴². The authors also describe what they call harness annealing: training "internalizes recurring harness-use patterns into the model policy and shifts the agent from frequent harness calls toward selective external-state access." The weights take over the routine parts of the harness, and the external state shrinks to what is specific to the task, so a single agent ends up with its own fast and slow stores.

Earlier in 2026, MemoPilot trained a memory copilot with multi-turn GRPO to guide a frozen player across sequential games, reaching Elo 1762 in Limit Texas Hold'em ⁴³, and SkillOS trained a curator for a frozen executor's skill repository that "generalizes across different executor backbones and task domains" ⁴⁴. These descend from 2025's RL-trained orchestrators such as Sakana's 7B Conductor and NVIDIA's 8B ToolOrchestra, which scored 37.1% on Humanity's Last Exam against GPT-5's 35.1% ⁴⁵.

The design fits the bandwidth argument. A controller learns a low-dimensional policy (what to store, whom to call, what to say), which RL's few bits per episode can teach, and the frozen workers keep their capability. The weaknesses are that controllers are trained against a specific worker pool and reward, most evaluations are QA, games and verifiable benchmarks, and the portability claims are tested on a handful of backbones.

The evidence is now good enough to decide factor by factor, and on most factors one store is clearly better:

The field still disagrees on three points. Letta bets that the slow store should itself be mostly tokens, written by trained memory models, with distillation into weights as an optional extra, and that bet depends on in-context learning improving faster than consolidation gets cheap ². The sleep camp bets that periodic consolidation into weights is necessary and won't fall out of longer RL runs ^{4 6}. And the continual-learning consensus that on-policy objectives are safer conflicts with Experience Distillation's finding that objectives sampled by the student can miss almost the entire gain. The on-policy correction results from September point to the answer (states the student visits, targets from a teacher that sees the context), but no paper yet compares all three objectives on the same tasks.

From these factors, the system I would build today looks like this. The fast store (context, memory, skills, harness code) is written every episode, gated by held-out checks, versioned and reversible. The slow store, preferably modular adapters, is written often and in small steps by distilling from a context- or harness-conditioned teacher onto the student's own trajectories, never from raw or foreign traces. After consolidation, the absorbed context is pruned and its source is kept as data for the next model.

Code and validators stay in the harness because they keep working on new models. High-traffic procedures move into weights because keeping them in context is expensive. User-specific and fast-changing facts stay in context because weights can't forget on request.

The last three months moved this area more than the previous 12. Joint harness-and-weight optimization went from coarse loops that switch once (SIA in May) to controlled comparisons of switching schedules (WHALE in August). Consolidation got its first measurements of how much is kept (Experience Distillation) and of what breaks it (imitation under an evolved harness, and brittleness under a fixed harness). Harness distillation became a named method, with a harness-free model beating the harness-attached one (Harness-Zero).

Loops that alternate between state and policy appeared, such as Experience Funnel, which consolidates only "state-enabled behavior that remains useful across state revisions" ⁴⁷. Parametric memory got its first design for moving across backbones (RPMem). And labs publicly deleted scaffolding their models had absorbed.

Deployment is narrower. As of September 2026, the well-documented public cases of weight updates from deployment are at product companies and not at frontier labs. Cursor checkpoints Composer "as often as every five hours" ²⁸, and since August Shopify has described a daily loop that retrains its Sidekick agent from production conversations. Cursor's hint-as-teacher method in Composer 2.5 and Shopify's repair hints are documented production cases of context consolidated into weights (see 05 and 16). Taobao Live's harness-aware training is a rarer example of a deployed compact model trained to tolerate harness changes ²².

Frontier labs consolidate once per model generation, with humans on the loop. On the context side adoption is far larger: DSPy has about 38,000 GitHub stars, while SIA, the most-starred weight-updating hybrid, has about 2,000. For builders on closed models, the hybrid available today is a small trained controller or memory model around a frozen API model, plus a harness registry for audit and rollback.

Do the gains keep coming after two cycles at larger scale? WHALE shows many small alternations beating one handoff, but on 2–4B models and three domains ¹⁰. At larger scale no system shows more than two harness-then-weights cycles, Harness-Zero and HELIX report one round, and Macaron-V1 says it has measured only the configuration-search stage ¹⁴.

If cycles saturate, co-evolution is a one-time boost. If each cycle keeps adding, the switching schedule becomes the main design choice in the area. The test is a loop run for five or more cycles on a frontier-scale model, with a held-out protocol for both levers and a matched-compute baseline that spends the same budget on either lever alone.

Can knowledge be deleted from a per-user adapter? Researchers consolidated each of 150 people's search histories into their own adapter in a September study, and each adapter then fit that person's held-out text measurably better than other people's (dz = 1.27) ⁴⁶. Letta and Macaron both argue for personalized weights. But a user who deletes a memory expects it gone, and no paper has shown that knowledge distilled into an adapter can be removed on request, or verified as removed. Until someone shows that, per-user consolidation is a privacy liability, and keeping user-specific facts in context is a requirement.

Which artifacts transfer to the next model? The claim that code transfers and prose doesn't is based on one blog post, one test-time study, one ablation and a handful of backbones. A controlled benchmark crossing artifact type (tool code, validators, skills, memory, prose playbooks, latent memories) with model family and model generation would tell builders what their harness work is worth after the next upgrade, and how much of Letta's "appreciating asset" still holds with a new model. It would also answer the frontier-scale version of the reverse-order question: what an optimized harness adds to a model already trained in the harness it came from.