A new technique called RRSI helps AI agents improve their performance on unseen tasks by preventing them from memorizing test benchmarks during recursive self-improvement.
Key facts
- •RRSI stands for Regularized Recursive Self-Improvement of Agent Harnesses.
- •The method uses a shrinking budget for edits to ensure changes are traceable and effective.
- •RRSI achieved up to 4.7 points of gain on five benchmarks the agents had never seen.
- •The system uses about 30 percent fewer tokens at runtime than the unregularized version.
- •The research team made the code for RRSI available on GitHub.
Google researchers have introduced a method called Regularized Recursive Self-Improvement (RRSI) to address the tendency of AI agents to memorize test tasks during automated optimization. By using a system that limits how agents rewrite their own control harnesses, the researchers aim to ensure that performance gains translate to new, unseen tasks. The study demonstrates that this approach prevents agents from favoring benchmark-specific tricks while maintaining or improving efficiency.
By the numbers
Addressing recursive self-improvement
Recent advancements in AI agents often involve automating the improvement of their control harnesses, which dictate how an agent interacts with files and recovers from errors. While language models can rewrite these harnesses based on feedback, this process often leads to memorization, where agents optimize for specific test tasks at the expense of general capability. The researchers found that agents often favor patterns that only fit one benchmark or rely on chance to achieve higher scores.
How the RRSI method works
RRSI introduces guardrails to the optimization loop, including a budget that restricts the number of independent edits a candidate can make at once. This budget decreases over time, forcing the system to prioritize small, traceable changes. Additionally, a critic reviews proposals to reject hardcoded solutions or benchmark-specific tricks, while the system tracks previous attempts to avoid repeating failed strategies. The method also removes components that fail to provide measurable performance gains.
Performance and testing results
In tests across eight benchmarks, including coding and engineering design, RRSI outperformed four other optimization methods on unseen tasks. While other methods often saw performance drop below the baseline on new tasks, RRSI maintained higher scores. The researchers noted that RRSI achieved up to 14.1 points of gain on training tasks and up to 4.7 points on unseen benchmarks, while using approximately 30 percent fewer tokens at runtime compared to unregularized versions.
Advertisement
This article was independently rewritten by ManyPress editorial AI from reporting originally published by The Decoder.
