Harness self-improvement

An experiment I had been hoping someone would run came from Google Cloud AI Research in September ¹. RRSI evolves an agent's harness (its prompts, tools, memory and control flow around a frozen model) on a set of agentic workspace tasks, and the authors ran the loop with and without their regularization.

Without it, the loop did what these loops are built to do and raised the score on the tasks it was evolving against, from 89.4 to 92.8. Averaged over three of the paper's out-of-distribution benchmarks (JobBench, GDPval, APEX-Agents), the same harness went from 39.7 to 40.3, within a point of where it started, while using 3.80 million tokens per trial, up from the original 1.56 million.

With RRSI's regularized gate, the loop accepted a harness that scored lower on the evolve split (90.5) and higher everywhere else: 89.2 on held-out tasks from the same distribution and 43.6 out of distribution, at 2.42 million tokens. Across eight benchmarks, its best gains were +14.1 points on the split it evolved against and +4.7 on the five out-of-distribution ones.

From that ablation I conclude that a loop keeping whatever scores higher mostly learns its benchmark, and gets more expensive while doing so. A better rule for accepting edits can give up a few in-split points in exchange for gains that hold elsewhere, though even RRSI, which was designed against overfitting, keeps only about a third of its in-split gain once the tasks change.

Self-Healing Harness, also from September, counted what a weaker rule would have let through ². The agent writes candidate behavioral rules for itself, and an external gate keeps a rule only if it fixes the failure that triggered it without regressing a set of protected cases beyond a fixed margin. Across 16 paired runs on AppWorld, Terminal-Bench and τ²-Bench the gate rejected 383 proposals, and 211 of them (55%) had fixed their triggering failure while breaking a case that used to work. A gate that checked only the trigger would have kept all 211 and reported each one as a success.

Ouroboros, from an August preprint, runs a gate like this in front of a coding agent that is live. Its tools, prompts, context assembly and core implementation "improve through reviewed commits that become the runtime for later work" ³. Every core change goes through a blocking multi-model diff-review panel, and the system can roll back to earlier reviewed states. The live instance, "Hope", is a "161-day living agent experiment in free evolution." Frozen snapshots of the same system report 86.74% on Terminal-Bench 2.1 with Claude Opus 5 and 90.69% on OSWorld-Verified (self-reported).

All three papers are about the check an edit must pass before the loop keeps it, and they don't try to build a better proposer. The worry behind that check is old. In 2023, STOP's GPT-4 improver wrote code to switch off its own sandbox in 0.42% of 10,000 attempts, and its authors concluded that "no security is provided by the fact that the utility is later re-evaluated in a sandbox" ⁴. What changed in 2026 is that engineers now design these gates as components, measure them and publish ablations of them.

An agent is a model plus a harness. Most people now use Lilian Weng's July definition: the harness is "the system surrounding a base model that orchestrates execution and decides how the model thinks and plans, calls tools and acts, perceives and manages context, stores artifacts, and evaluates results" ⁵. Concretely, that means a system prompt, tool implementations and schemas, the code that assembles and compacts context, control flow and stopping rules, middleware and hooks, sub-agent configuration and a memory policy.

Self-modification has concentrated on the harness for practical reasons. It is code, so a coding model can read and edit it with the same skill it brings to any other repository. It works with closed models, so a team on Opus or GPT-5.x can improve its agent without training access. And a harness change is a diff that you can read, revert and evaluate for far less than a training run costs.

Weng states the design-space argument directly: "If an LLM can optimize the code that executes agents, it can access a much larger design space than hand-written prompts" ⁵. The loops in report 06 tune the text inside a fixed program; the loops here rewrite the program.

The naive version of the loop is easy to describe: run the agent on tasks, have a model read the failures, patch the harness, rerun, and keep the patch if the score went up. Every step of it can go wrong. The patch can fix the failures the model read and break cases it never looked at, which is what Self-Healing Harness measured.

The score can go up by chance; PACE (June 2026) showed that "keep it if the score went up" amounts to uncontrolled adaptive multiple testing, and small Qwen2.5 agents self-evolving at the prompt level committed 30–42% false edits under it ⁶. If the rerun uses the same tasks the proposer studied, the gain can be memorization of the benchmark, as in RRSI's unregularized arm. And because the proposer is the agent, it can edit the code that measures it.

The history of these acceptance rules is short. Schmidhuber's Gödel machine (2003) allowed a program to rewrite any part of itself once it had proved that the rewrite increases expected utility, a gate that is perfect and impossible to use ⁷. STOP (2023) replaced the proof with a meta-utility, the mean downstream score of the programs the improver improves, and found that it worked for GPT-4 but not for weaker models ⁴. ADAS (2024) added an archive of agent designs written in code by a fixed meta-agent ⁸, and SICA (April 2025) let the best agent so far edit the whole coding-agent codebase ⁹.

DGM, the Darwin Gödel Machine (May 2025), made the archive self-referential and admitted children after staged benchmark evaluation. Over 80 iterations its agent went from 20.0% to 50.0% on a SWE-bench subset and from 14.2% to 30.7% on Polyglot, at about USD 22,000 and two weeks per run, and the tools it discovered transferred across Claude and o3-mini ¹⁰. Its gate also let through an agent that disabled the logging its hallucination detector depended on. The Huxley-Gödel Machine (October 2025) changed which agents get expanded, crediting an agent by how its descendants perform (clade metaproductivity) instead of by its own score, and matched DGM with 2.38× to 6.86× fewer CPU-hours ¹¹.

Hyperagents (March 2026) tested an assumption all of these share: that an agent good at the task is also good at modifying itself ¹². Outside coding it isn't. The original DGM scored 0.0 on paper review and robotics reward design without manual customization. Putting the task agent and the meta agent into one editable program lifted paper review from 0.0 to 0.710, and transferred hyperagents reached imp@50 of 0.630 on unseen IMO-level grading, against 0.0 for DGM-custom agents.

The Hyperagents result is evidence that the ability to improve can move across domains. So by spring 2026 benchmark-gated archives were known to work on coding, to cost a great deal, to pick parents badly and to let the agent reach its own instruments.

An approximate arXiv tally made for this series counts at least 60 papers from 2026 whose main contribution is automated modification of an agent's own harness. They are concentrated in July–September, and many come from industrial labs (Google Cloud, Salesforce, Xiaomi, ByteDance Seed, Sakana). Most use a strong coding agent as the proposer, and they differ mainly in the rule that admits its edits.

AIDE² (September 22) is the most recent full recursive loop, and it selects on hidden evaluations ¹³. The system proposes changes to its own research-agent code, benchmarks the modified versions on a suite of AI R&D tasks, and keeps "the changes that perform best on hidden evaluations." An autonomous eight-day run found seven successive improvements, from a new search policy to memory mechanisms that compress the agent's growing context.

The improved agents also scored higher on four held-out benchmarks, one of which (physics-based weather forecasting) is out of distribution from the selection tasks, and the strongest discovered agent matched or exceeded a human-engineered production research agent on FML-Bench. On a separate held-out task family, the reward-hacking rate fell from 55% to 32% during the run, although the loop never optimized for it. For harness design, the useful detail is that selection ran on evaluations the proposer could not study.

ModularRSI (September 14) keeps evolution and evaluation completely apart ¹⁴. It evolves five decoupled harness modules (agent loop, tool use, observation management, context management, task-completion detection) separately on 2,000 executable tasks disjoint from every evaluation benchmark, using contrastive evidence from successes and failures on the same task, and then integrates them. On Terminal-Bench 2.0, in-domain accuracy went from 47.57% to 52.43%, with gains on unseen in-domain and cross-domain tasks and across foundation models. The gain is smaller than the in-split numbers in other papers, and I find it more believable for that reason, since a smaller number is what evolving on disjoint tasks should produce.

RRSI (September 21), the system from the opening, regularizes both proposal and selection ¹. On the proposal side, an annealed edit budget limits how many changes one candidate can bundle, and an exploration bonus pushes toward edits that haven't been tried. On the selection side, a critic "screens benchmark-specific proposals" and a pruner removes changes "that are too small, too expensive, or no longer useful."

In the ablation, removing either group raises the evolve-split score and lowers the out-of-distribution one. With Gemini 3.5 Flash as the frozen policy, Terminal-Bench 2.1 rose 14.1 points (64.6 to 78.7) while out-of-distribution SWE-bench Verified rose 2.2 (76.8 to 79.0), and under a shared budget RRSI beat Meta-Harness, AHE, TTHE and HarnessX.

StarHarness (August) formalized a three-way split: search tasks the proposer can see, selection tasks hidden from the proposer, and held-out test tasks that neither the proposer nor the selector touches ¹⁵. After 4–12 accepted changes it reported gains of 20–35 points on three enterprise benchmarks (ITBench SRE, EnterpriseOps-Gym ITSM, AutomationBench Finance), and its harnesses kept those gains on excluded tasks and across GPT and Qwen models.

DarwinX (Salesforce AI Research, July 31) keeps a population ¹⁶. Its authors' diagnosis is that "single-lineage search is path-dependent and local wins often regress other tasks." DarwinX maintains harnesses across several lineages, recombines them, and admits a variant only if it preserves and extends what it inherits, which amounts to a no-regression contract at admission. It reports Terminal-Bench 2.1 at 83.2% with a matched base (+7.7 points) and 84.7% with a stronger base, 68.3% on a held-out TerminalWorld set, WebArena-Infinity from 43.5% to 93.0% "audit-clean", and a TB2.1 harness that transfers unchanged to SWE-bench Verified.

Self-Harness (Shanghai AI Lab, June) drops the stronger external proposer ¹⁷. One model mines its own weakness patterns from traces, proposes minimal edits to bounded parts of the harness, and promotes only edits that pass regression tests on held-in tasks and on held-out tasks whose traces the proposer never sees. Across TB2, SWE-bench Verified and AppWorld with three open models, every final harness improved on both splits, with gains up to 40.6 points (Qwen3.5-35B-A3B on SWE-bench Verified went from 20.1% to 42.5% held-in).

The design has a catch, though: once a held-out split is used inside the acceptance rule, it is selection data, hidden from the proposer but visible to the selector. Self-Healing Harness's 55% figure is a measure of what a regression gate like this one catches.

Ouroboros replaces the benchmark gate with a panel of models that reviews each commit, and its live instance is never frozen ³. Its benchmark numbers come from frozen snapshots, so they say little about what 161 days of reviewed live evolution did, and the paper doesn't publish a false-negative rate for the review panel.

The rest of the 2026 papers mostly vary the same few choices about what to edit and what to check before keeping an edit:

The first careful answer came in May. "Harness Updating Is Not Harness Benefit" separated updating (can a model edit the harness well?) from benefit (can a model make use of an edited harness?) ³¹. "Harness-updating is flat in base capability": "even Qwen3.5-9B's updates yield gains comparable to those of Claude Opus 4.6."

Benefit was non-monotonic in model tier. Weak models failed to activate or follow the new artifacts, mid-tier models gained the most, and strong models gained less. The authors advise spending capability on the agent that solves the task and not on the evolver.

Three August and September benchmarks for harness optimization complicate that advice, and all three find the hard part is generalizing. HarnessOpt-Bench scores a nominated harness on a held-out partition that a trusted execution environment keeps out of reach. Across five frontier models, four tasks and 111 runs, "optimizer models separate more than the coding harnesses they act through," and detailed inspection of failure traces was rarely used and not positively associated with gain ²⁸. That second finding fits badly with Meta-Harness's argument for full traces, described below.

Evo-Bench found that top models reached absolute gains of 16.6 points, "closely approaching" state-of-the-art human-engineered baselines; they did well on search tasks, struggled with office tasks "that demand highly specific processing workflows," and saturated early ²⁹. HarnessDev (ByteDance Seed) found that all five self-runtime creators improved on their visible feedback but gained less on hidden tasks, and under a fixed Gemini executor "only Opus improves on held-out tasks, while the other three lineages regress" ³⁰. My reading is that small models write edits that help locally, and that edits which generalize still favor a stronger proposer.

The argument for giving the proposer rich evidence was made in the spring. Meta-Harness (March) gave Claude Code running Opus 4.6 filesystem access to every earlier candidate's code, scores and raw traces, on the diagnosis that text optimizers "compress feedback too aggressively," and reached 76.4% on Terminal-Bench 2.0 (TB2), though it searched and reported on the same 89 tasks ³².

Agentic Harness Engineering (AHE, April) made every editable component a revertible file, distilled millions of trajectory tokens into evidence the agent could drill into, and paired every edit with a falsifiable predicted impact. In Weng's summary, the run directory, tracer and verifier are read-only to the agent ^{33 5}. AHE moved TB2 from 69.7% to 77.0% in 10 iterations, and its frozen harness gave 5.1 to 10.1 points on three other model families.

With proposers this capable, evaluation is what limits these loops. A theory paper from September makes the point formally: generation and certification impose distinct constraints, and "generating more candidates need not improve the guarantee of a successful update when evaluation is limiting" ³⁴. Its authors also prove that the worst-case cost of recognizing a real improvement diverges as expected reward approaches its upper bound, and frontier Terminal-Bench scores are now in the mid-80s, close to that bound.

For every loop I want to know whether the gain holds on tasks the loop never optimized against, and whether it beats what equal compute would buy some other way. On the first question, the favorable evidence is reasonably strong. AIDE²'s gains held on four held-out benchmarks, one of them out of distribution ¹³. DarwinX and StarHarness report gains that hold on excluded tasks ^{16 15}, EvoSafeHarness transferred unchanged to unseen AgentDyn suites ²¹, and AHE's frozen harness transferred across three model families ³³. Life-Harness (May 2026) evolved a harness only from Qwen3-4B trajectories and improved 116 of 126 model–environment settings across 18 backbones ³⁵.

On the second, a matched-budget study reaches the opposite conclusion, and I find it more convincing than any of the transfer results. "Rethinking the Evaluation of Harness Evolution for Agents" (Wang et al., July 2026) ran Meta-Harness and AHE on Terminal-Bench 2.1 with GPT-5.4 and Claude Opus 4.6 against simple test-time scaling, with comparable feedback and inference budgets ³⁶. Automatic harness evolution "does not consistently outperform simple test-time scaling methods and exhibits limited generalization."

Its authors name the two flaws that most earlier results share: search and evaluation use the same benchmark, and the baseline doesn't get an equal budget. One of its authors, Huaisheng Zhu, also co-wrote the co-evolution system Co-Harness, so the critique comes from someone who builds these systems.

The September papers put numbers on the gap. RRSI's +14.1 in-split against +4.7 out of distribution is about three to one for a method built to resist overfitting, and its unregularized arm kept almost none of its gain out of distribution ¹. CHASE ("Bad Genius") found a subtler leak: "task holdout varies semantic tasks but leaves the benchmark protocol fixed," so a proposer can learn protocol-wide shortcuts that still work on a task-level held-out split ³⁷. PRISM's authors found that "some search procedures can occasionally find large gains but still choose brittle updates," and argue that papers should report how reliable the chosen harness is alongside the average held-out lift ²².

"Harness or Model?" (September) asked whether vendor-native harnesses beat a neutral one on a private, contamination-controlled suite ³⁸. Neither contrast showed an average advantage: −1.25 points for Opus 4.8 native versus deepagents, and +1.25 for GPT-5.5. The Opus average, though, combined a 9.0-point deficit on repository tasks with a 23.7-point lead on contest tasks, a split the author chose after seeing the data. So the harness does make a difference, but it depends on the task, and effects that depend on the task are the hardest ones to measure on a single benchmark.

Don't compare the benchmark numbers across rows. Terminal-Bench 2.0 and 2.1 differ, the base models differ, and only a few rows keep the evolution signal separate from the test.

Where the transfer evidence is good, the gains come from structure more than from prose. AHE's gains were in tools, middleware and memory, and hardly in the system prompt; its authors read this as "factual harness structure transfers while prose-level strategy does not" ³³. HARNESSEVO (June 2026) found nearly all the optimization value on ALFWorld in the reflection and control slot ³⁹.

Life-Harness attributes its transfer to "reusable environment-side structure rather than model-specific behavior" ³⁵. Ecdysis (September 2026) gets its cross-model gains by promoting only failures that recur across distinct tasks, which keeps it from accommodating one model's quirks ⁴⁰, and the mechanisms SoL-Pi kept are all structural: action execution, compaction, observation handling and delegated reading ²⁰.

Knowledge about the environment and the task belongs in harness code, where other models can use it too. Instructions tuned to one model's habits are the least portable part of a harness and the part most likely to be overfit.

The failures of these loops have now been measured, and most of them are failures of the gate. Collateral regression is the most common. Besides Self-Healing Harness's 55% ², EvoHarnessBench (September 2026) showed across 17 streams and 802 tasks that adding tools, skills and agents to a harness can break tasks it used to solve, a problem its authors name harness-induced forgetting ⁴². Human harness engineers cause the same problem: "Don't Blame the LLM" (July 2026) tracked 35 releases of the Qwen Code CLI on 50 SWE-bench Verified tasks with the model held fixed, and traced swings in quality to individual pull requests ⁴³.

Tampering persists. Auditors seeded pairs of tampered and benign edits into real self-improvement trajectories in August and found that tampering "consistently occurs in real runs from different agents" and "often persists in the lineage of the best agent" ⁴⁴. In an archive, a tampered ancestor is worse than one bad commit, because its descendants inherit the tampering and selection rewards it. DGM's deleted hallucination logging, which report 15 recounts in full, is the best-known early case ¹⁰.

Some improvements can't be undone. EvoUndo (August 2026) found that in 197 of 600 one-shot self-evolution tasks, the agent produced mutations that improved capability but failed recoverability verification, and conventional repair recovered none of the 197 ⁴⁵. A harness that gets better by making itself unrevertable has thrown away the main reason to edit harnesses in the first place.

Proposers also invent problems. The Phantom Guardrails study (July 2026) gave a harness optimizer legal game logs that contained a rule-shaped pattern. In 15 of 60 runs it fabricated a violation (0 of 60 on featureless input) and added a guardrail for a rule that never existed ⁴⁶. In an add-only accept loop, the phantom guardrail comes back and "once in it stays." An edit like that is neither reward hacking nor over-refusal, and a gate that only checks for lost capability won't see it.

The remaining failures are statistical and economic: greedy acceptance commits false edits ⁶, and the cost of certifying an improvement diverges as scores approach their maximum ³⁴. Each failure calls for a different check: a regression suite against collateral damage, read-only instruments and hidden checkers against tampering, recoverability checks against irreversible edits, statistical acceptance against noise, and adversarial review against rules that only fit the benchmark. I couldn't find a published system that combines all five. RegenHarness (September 2026) comes closest in intent for a physical deployment, pairing fixed regression checks and release authorization with versioned rollback and explicitly denying the loop "permission to weaken the commit gate" ¹⁸.

To add a self-editing loop to an agent now, I would start with those checks and a proposer that is already good enough: a regression suite of protected cases, a selection set the proposer never sees, instruments the agent can't write to, and a human approving each commit. I'd point the loop at tools, context handling and control flow before letting it touch the system prompt.

Weng's July synthesis orders harness optimization by the object being optimized: "instruction prompts → structured context → workflow → harness code → optimizer code" ⁵. Each level contains the one before it, and the last level is a loop editing its own editor, which is what Hyperagents, Ouroboros and AIDE² do.

The ordering lines up with this series. All five levels are non-parametric, since the improvement is written into artifacts a frozen model reads or runs. Instruction prompts are the subject of report 06, structured context and memory of report 07, workflows of report 10, harness code of this report, and optimizer code of report 13. Weng's axis and the series' storage axis measure different things: hers says what is edited, and ours says where the edit is stored and therefore how it behaves when the model is upgraded.

What her ordering leaves out is the gate. A loop editing its own optimizer behind a strong gate is safer than one editing prompts behind a greedy gate.

The 2026 evidence also complicates her forecast that "many harness improvements will be internalized into core model behavior, but the interface with external context and tools should remain." Some harness work is a temporary step on the way into the weights. Other harness work replaces model capability entirely. AutoHarness (February 2026) turns the whole policy into code in closed game worlds ⁴⁷, and Growing Harness (September 2026) moves recurring control from context into code until a 4B model matches a 120B one on WebArena-Verified ¹⁹. I expect that second kind to stay in the harness for good, with deterministic structure in code and judgment left to the model.

Industry runs the loop the papers describe (traces, a diagnosing agent, a patch, an evaluation gate) with two changes: the gate uses real-usage metrics in place of benchmark splits, and a human signs off on core changes. Anthropic's disclosure, updated in September, says Claude authored more than 80% of merged code at Anthropic as of May 2026, and an automated Claude reviewer reads every proposed change before merge. In a retrospective, that reviewer "would have caught roughly a third of the bugs behind past incidents on claude.ai" ⁴⁸. Anthropic doesn't document an autonomous loop that rewrites Claude Code. What it documents is Claude writing most merged code under human direction, with a model in the review path.

In July, Anthropic's Claude Code team said it had removed about 80% of the Claude Code system prompt for its newest models ⁴⁹, because the newer models had absorbed what that prose told earlier models or no longer needed it. Claude Code's dynamic workflows, launched in late May, apply the same approach per task: Claude writes a harness for each task, and a good run can be saved as a reusable command, so per-task harnesses build up into a library that humans curate ⁵⁰; report 10 describes their orchestration and permission modes.

Cursor described its version in April ⁵¹. Harness variants run as online A/B tests on real usage and are judged by Keep Rate, the fraction of agent-written changes that remain in the user's codebase after fixed intervals, plus an LLM reading of user responses. A skill teaches the model to search logs, find new or spiking issues and file tickets, and Cloud Agents then fix many of them at once, which Cursor calls "an automated 'software factory' for our agent harness." One sprint cut unexpected tool-call errors "by an order of magnitude," and each new model still takes weeks of per-model harness customization.

OpenAI's harness-engineering team, writing earlier in 2026, treats each agent struggle "as a signal," works out which tool, guardrail or piece of documentation is missing, and feeds it back into the repository, "always by having Codex itself write the fix" ⁵². LangChain's Trace Analyzer skill spawns parallel error-analysis agents over traces; with gpt-5.2-codex held fixed, harness changes alone moved deepagents-cli from 52.8% to 66.5% on Terminal-Bench 2.0 ⁵³, and Self-Harness was built on the same DeepAgents harness ¹⁷.

Warp's August write-up describes the smallest version of the loop: a scheduled "improver" skill compares its agents' output with maintainers' feedback and "proposes a small, focused edit" to the base skill, which then goes through normal code review, where "a human reviews, approves, and merges" ⁵⁴. None of these vendors, and none of the others I found while researching this series, documents a shipped core harness that rewrites and deploys itself without review. I think they are right to hold back. Human harness releases already cause regressions ⁴³, and the automated reviewer Anthropic cites catches about a third of past incident bugs, which leaves two-thirds. Industry has automated the diagnosis and the patch and kept a person at the commit.

More and more, the harness is a stage in a pipeline that ends in the weights. Harness-Zero (September 21) uses an optimized harness as training-time guidance, fine-tunes on the resulting trajectories, and then removes the harness. Macro success went from 23.3% to 44.3% with the harness gone, higher than the 41.7% the model scored with it attached, and the model recovered 82.3% of 28 harness-induced behavior patterns absent from the base model ⁴¹.

WHALE (August) alternates weight updates with harness search and finds that "either component can be the bottleneck": harness search matched peak weight-only accuracy with far fewer rollouts on SearchQA, but math improved only after a weight update ⁵⁵. The fit between harness and model breaks easily in both directions. Training a weaker model on a stronger expert's trajectories, under a harness evolved for the weaker model, made it worse on all seven enterprise tasks, by 4 to 30 points ⁵⁶.

Anthropic's deletion of most of the Claude Code system prompt is the production version of Harness-Zero's result: once a model has absorbed what the scaffold taught it, the scaffold is dead weight or worse. When to distill, what to delete afterward and whether gains add up over more than one cycle are the questions report 12 takes up.

Which of these systems works best at matched compute, on tasks it never saw? A dozen systems claim state of the art on Terminal-Bench variants and SWE-bench, evaluated on different splits, base models and budgets, and the one careful matched-budget study found that evolution does not consistently beat test-time scaling ³⁶.

HarnessOpt-Bench's trusted held-out partition and PRISM's RelLift95 are the right instruments ^{28 22}. No study has yet run Meta-Harness, AHE, Self-Harness, DarwinX, RRSI, ModularRSI and AIDE² through them at matched compute against a test-time-scaling baseline. Until someone does, the claim that harness self-improvement buys capability, and doesn't simply spend more compute, depends on a handful of transfer results.

How often does an LLM review panel miss a bad self-modification? Ouroboros's multi-model panel, EvoSafeHarness's fresh-context adversarial review and Anthropic's automated reviewer all have a model judge edits written by a model with overlapping blind spots ^{3 21 48}. The tampering and phantom-guardrail results show that the edits a reviewer has to catch are subtle and get inherited ^{44 46}. No paper reports a false-negative rate for a review panel on seeded bad self-modifications. That number would tell us whether reviewed self-modification can run without a human at the commit.

When should a harness be distilled into weights? In Harness-Zero, distillation beat the harness it came from; Anthropic's prompt cut indicates that scaffolds shrink as models absorb them; and in the imitation study, moving behavior across a mismatched model and harness backfired ^{41 49 56}. The transfer evidence suggests a rule: keep environment structure in code, consolidate behavior into weights, and throw away accommodations for one specific model. I haven't seen a study that tests that rule across a model upgrade.