A better harness can generalize.
All seven reported aggregate lifts are positive. But only the leading model, Claude Opus 5, improves in every evaluated domain.
Recursive self-improvement, measured.
Can AI agents
improve their
own harnesses?
A benchmark for recursive harness self-improvement. Agents revise the systems around a frozen model, then test whether those changes generalize to unseen tasks.
Revisable harness
Fixed model
Revise. Select. Inherit.
One controlled
reported experiment
reported experiment
- Tasks
- 100
- Domains
- 4
- Models
- 7
- Editable modules
- 18
The model stays fixed.
The harness evolves.
An agent is more than its model. Its prompts, tools, memory, and execution logic shape what it can do. RSIBench asks whether an agent can improve that surrounding system itself.
The method
A revision becomes
the next starting point.
Not a fresh prompt every round. The accepted harness is inherited—and becomes the system that proposes the next change.
Revision cycle / 01Weights frozen
Run training tasks
Run the current harness on the training split. Task outcomes and execution traces provide evidence for the next revision, without exposing held-out tasks.
Revision seesTraining evidence only
Run training tasks
Run the current harness on training tasks and collect outcomes and execution traces.
Inspect evidence
Inspect training evidence to identify where the surrounding system can change.
Propose a revision
Revise prompts, skills, context management, control flow, tool runtime, or verification behavior. Model weights stay fixed.
Validate and select
Validate the proposed change using training evidence. Rejected revisions retain the current harness.
Inherit the harness
The accepted harness carries forward and produces subsequent revisions.
Then, evaluate the unseen.
Held-out tasks. Excluded from revision and selection.
50held-out tasks
5 revision rounds
50 training / 50 held-out tasks
3 rollouts per task, per split
Reported scheduleFixed Model weights, tasks, verifiers, and evaluation boundaries.
Editable Prompts, skills, context management, control flow, tool runtime, and verification behavior.
The evidence
Does the improvement
survive unseen tasks?
All seven models show positive overall lift.
The domain-level picture is more complicated.
+8.33 ppHighest reported overall liftClaude Opus 5 · held-out evaluation
Positive lift Negative liftSame scale across every view
View the data table
| Study-reported model | Overall lift |
|---|---|
| Claude Opus 5 | +8.33 pp |
| Kimi K3 | +7.05 pp |
| GLM-5.3 | +3.44 pp |
| DeepSeek V4 Pro | +2.67 pp |
| Claude Opus 4.8 | +2.44 pp |
| GPT-5.6-Sol | +1.78 pp |
| GPT-5.5 | +0.55 pp |
What is RSI lift? Final minus initial held-out score, in percentage points. Overall lift is task-weighted across 50 held-out tasks. Domain views use their respective held-out splits.
Reported experiment, not a live leaderboard. Source: RSIBench study, Table 1. Model names are study-reported labels, not independently verified vendor availability. No uncertainty intervals are reported here.
The task landscape
Four domains.
No single way to improve.
100 tasks, split evenly between training and held-out evaluation. Each domain tests a different setting for harness adaptation.
Terminal
30 tasks · 15 training / 15 held-out
Act through a command-line interface. The harness shapes how an agent uses tools, maintains context, and recovers during execution.
Environment Learning
30 tasks · 15 training / 15 held-out
Learn to act within an environment. The harness shapes how observations, context, and tool interactions inform the next action.
Software Engineering
16 tasks · 8 training / 8 held-out
Work with software. The harness shapes how an agent inspects code, manages changes, and verifies its work.
Science
24 tasks · 12 training / 12 held-out
Work through scientific tasks. The harness shapes how an agent organizes evidence, uses tools, and checks its reasoning.
Domain focus
Terminal
Act through a command-line interface. The harness shapes how an agent uses tools, maintains context, and recovers during execution.
15 training15 held-out
Domain-level illustration; not a released task example.
What we learn
Progress is real.
It is not uniform.
Evidence for bounded harness adaptation—not a claim of indefinite recursive improvement.
Read the results in contextAn overall gain can hide a regression.
GPT-5.5 records +0.55 pp overall alongside −8.33 pp in Science. Training selection alone cannot establish improvement on unseen tasks.
Inspect the Science resultsRecursion still needs boundaries.
The study measures five revision rounds under fixed tasks and evaluation rules. Gains depend on the model and domain; they do not establish unlimited self-improvement.
A few useful distinctions.
What is an agent harness?
The system around a model: prompts, skills, context management, control flow, tool runtime, and verification behavior. It determines how the model interacts with a task and its environment.
Does the model retrain?
No. The model weights remain frozen. The agent edits the surrounding harness rather than changing the underlying model.
What counts as improvement?
RSI lift is the final held-out score minus the initial held-out score, in percentage points. Overall lift is weighted by tasks across the 50-task held-out split, not an unweighted average of the four domains.
Are these live results?
No. These are the reported main experiment results, with five revision rounds and three rollouts per task per split. Model names are study-reported labels. This page is not a live model availability listing or leaderboard.