Seal the holdout before the harness rewrites itself.
Your harness scored higher on the tasks it rewrote itself against. Before you promote, ask what it never saw—and refuse training-only lift.
Before you let an agent harness rewrite prompts, workflows, or tools from eval feedback, seal a holdout set the harness never sees during self-improvement. Ban edits that hardcode task names or solutions. Promote only when holdout does not regress—refuse training-only celebration. Name who may open the holdout, and prove a planted memorization cheat fails on it.
Did the harness learn the job—or the test?
Q: Our agent harness just rewrote its own prompts and scored higher. Can we ship?
A: Not on that number alone. Ask what set it improved against—and what set it never touched.
October 2026 coverage of Google Cloud AI Research’s RRSI work (via THE DECODER and the paper itself) makes the failure mode plain: when a harness rewrites prompts, workflows, and tools from the same tasks you score it on, those scores climb while gains on unseen tasks shrink. The research response adds edit budgets and a critic that rejects hardcoded task names and solutions. Treat that as a hook, not a scorecard to paste into your roadmap.
The operating decision for teams building agents is simpler and older than any paper title. Before you let a harness rewrite itself from eval feedback, seal a holdout the harness never sees during self-improvement—and refuse to promote on training-only lift.
That is different from banning a competitor download mid-eval, freezing fixtures before promote, drawing a containment boundary, or distrusting tool returns. Those layers still matter. This note is about memorization inside the improvement loop.
When the work is agents you can defend after they rewrite their own scaffolding, start from AI applications & agents.
Holdout is not the other agent gate
Q: We already ban eval shortcuts and freeze fixtures. Isn’t that enough?
A: No. Those answer different questions.
Shortcut ban answers: which technically available moves void a score—even when the agent can perform them.
Fixture pack answers: which frozen cases and human gates you re-run before promote.
Containment answers: what the agent must never reach.
Tool-context trust answers: whether tool-returned prose may authorize a write.
Harness holdout answers: which tasks the self-improving harness may never see while it rewrites itself—and what proof you need before you treat training lift as real.
A harness can pass every shortcut ban, sit inside a sealed room, and still memorize the training eval set by baking task names, golden phrases, or workflow branches that only fire on those cases. Your training curve looks heroic. Your holdout is quiet. Quiet is the signal.
If your review only asks “did the score go up?” you will promote a harness that learned the answer key.
Training-only lift is not promotion evidence.
What you name before the loop may rewrite
Q: What do we actually write down?
A: Enough that a skeptical operator can refuse a bad promote without a debate.
Name these fields once per self-improving harness—later you can keep them on one Harness Holdout Gate card if that helps the team, but the names matter more than the stationery.
Training eval set. The tasks the harness may see while proposing and selecting edits. Stable ids. Versioned. Not “whatever is in the nightly folder.”
Sealed holdout. Tasks the harness must never see during self-improvement: no traces, no scores, no critic feedback, no prompt diffs derived from those cases. Same job family as training is fine; same instances are not.
Who may open the holdout. A named role—not “the agent channel.” Opening means scoring a candidate harness after the rewrite loop stops. Opening mid-loop for “a quick peek” contaminates the seal.
Ban on hardcoded task cheats. Reject harness edits that name holdout or training task ids, paste gold solutions, or add branches that only fire on known case titles. A critic that screens for that class of edit is the research idea; your version is a written ban plus review of the proposed diff.
Promotion rule. Holdout must not regress. Prefer holdout flat or up versus the pre-rewrite baseline. Training-only celebration is not a ship gate. If training jumped and holdout slumped, you learned the test.
Owner. Who may change the seal, re-split sets, or override a failed holdout. One role. Written.
Proof a planted memorization cheat fails. Before you trust the gate, plant one obvious cheat—an edit that hardcodes a training task name or embeds a gold answer—and confirm the critic (or your review) rejects it, and that holdout still fails the cheat when scored. If the planted cheat promotes, you do not have a gate.
That field set is the unreproducible asset. Coverage can summarize RRSI. It cannot name your seal, your opener, or your cheat-fail proof.
How the work moves
From rewrite loop to sealed promote
- Split training vs holdout
Harness may see training during rewrites; holdout stays sealed.
- Run the rewrite loop
Propose and select harness edits only against training feedback.
- Screen for hardcodes
Reject edits that name tasks, paste gold answers, or case-specific branches.
- Open holdout once
Named opener scores the candidate after the loop stops—not mid-loop.
- Promote or refuse
Ship only if holdout is flat or better; refuse training-only lift.
How memorization sneaks past a green curve
Q: What does harness memorization look like in practice?
A: Composites—not paper metrics, not client telemetry.
The named-case branch. A prompt edit adds “if the ticket mentions Acme Renewals, skip retrieval.” Training tickets all say Acme Renewals. Holdout uses different customers. Training soars; holdout stalls.
The gold phrase paste. The rewriter copies a sentence that only appears in training expected outputs. Scoreboard loves it. Unseen tasks do not contain the phrase.
The tool that only helps on known ids. A new helper tool looks general until you notice its examples are the training set titles. Edit budget looked small. Transfer did not.
The mid-loop peek. Someone opens holdout “just to sanity-check,” then feeds those failures back into the next rewrite round. The seal existed on a wiki page. The loop never respected it.
The training-only ship meeting. Slide one: +big on evolve set. No holdout row. Leadership hears “the agent got better.” You shipped a better answer key.
None of these require a jailbreak story. They require a promotion rule that still smiles at training-only lift.
Wiring the same discipline into day-to-day automation gates often sits next door at business automation.
Decision guide
Training lift versus holdout-honest promote
| Consideration | Holdout-honest promote | Training-only celebration |
|---|---|---|
| What the harness saw | Training set only while rewriting; holdout sealed until score time. | Same tasks used for rewrite feedback and the victory slide. |
| Hardcoded cheats | Banned and screened; planted cheat fails before trust. | Task names or gold phrases allowed if the curve looks good. |
| Promotion evidence | Holdout flat or up vs baseline; owner signs. | Training jumped; holdout missing or ignored. |
| What you can defend | A skeptical operator can re-open holdout and get the same story. | A green training number that collapses on unseen work. |
When to stop the loop and when to promote
Q: The rewriter wants another round. Training is still climbing. What now?
A: Stop on process, not on vibes.
Stop proposing edits when you hit the agreed edit budget, when the critic keeps rejecting the same class of hardcodes, or when training gains keep rising while a sealed check (run by the opener, not the loop) shows holdout flat or down. More rounds on the same seen set is how memorization compounds.
Promote only when:
- The candidate harness was produced without holdout traces in the loop.
- Holdout score is flat or better than the pre-rewrite baseline—you refuse training-up / holdout-down.
- The diff was screened for hardcoded task names and gold solutions.
- The planted memorization cheat still fails as designed.
- The owner signs the promote.
If training looks spectacular and holdout is missing, you do not have a promote. You have a demo of overfitting with better branding.
Q: Can we reuse the same holdout forever?
A: Reuse until it leaks. Once instances appear in logs the harness can read, or in tickets the rewriter can cite, rotate. A holdout that the team chats about in the same channel as the rewriter is no longer sealed.
Put this into practice
- List every agent harness that may rewrite prompts, workflows, or tools from eval feedback—nightly evolvers, “self-heal” loops, prompt optimizers tied to your scoreboard.
- For each one, name training set, sealed holdout, who may open, hardcode ban, promotion rule, owner, and planted-cheat proof.
- Refuse to promote on training-only lift. Require holdout flat or better, with no holdout traces in the rewrite loop.
- Keep shortcut bans, fixture packs, containment, and tool-trust gates as separate layers. Do not pretend a sealed room or a frozen fixture pack covers harness memorization.
- Treat RRSI / THE DECODER coverage as a reminder to tighten your promotion gates—not as numbers to reprint. If the gap is agents whose harness edits you can defend after self-improvement—not another demo slide—bring the named fields into a scoping conversation for AI applications & agents.
A climbing training curve with an open holdout is still a story. A sealed set, a hardcode ban, an owner, a cheat that fails on purpose, and a promote that ignores training-only celebration is an operating control. Seal the holdout before the harness rewrites itself.
Related work: Spin:Market.
We build alongside the people doing the work.
Bring us the problem you’re thinking about.