Appearance
Your Agent Harness Is the Model You Should Be Optimizing
Give a multi-turn agent a token budget and the first thing you learn is that the budget does not make it frugal. In my own runs, an agent handed 40 turns and a few hundred thousand tokens would burn a third of the budget on the first two tool calls, then get careful only after the harness started refusing requests. The harness enforced the limit. The model never learned it existed.
That gap is what this cluster of papers attacks. One asks whether an agent can learn to allocate a budget across turns instead of just hitting a wall. One asks whether a team of agents can collaborate with an actual convergence guarantee rather than a vibe. Two treat the harness itself as something a model can rewrite.
Four papers, four angles on the same move: stop hand-tuning the scaffolding, put it in the training loop.
| Paper | What gets optimized | Training signal | Evidence |
|---|---|---|---|
| Budget-aware rewards | The policy's spending behavior | Reward shaped around a stated budget | Qwen3.5 4B/9B on AppWorld, BabyAI |
| MaD-RSI | A shared harness across many models | Failure trajectories from heterogeneous LLMs | Multiple datasets and model families |
| Local work, global progress | The collaboration protocol | Global-objective checks on each proposal | GPT-OSS-120B on robot routing, SQLite query planning |
| ChartEvo | Skills and tools for chart reasoning | Experience on underperforming chart types | ChartBench and ChartX long-tail subsets |
The budget is an input, not a guardrail
The first paper starts from a question most agent builders answer by accident. An external harness can cap spending. Can the agent learn when to spend?
The setup matters more than the headline number. The authors train Qwen3.5 at 4B and 9B, and they train a separate policy per environment and resource type, with each policy seeing many budget values during training. That last detail is the whole trick. The budget isn't a runtime ceiling bolted on afterward. It's an input the policy conditions on, so the same weights can be frugal at 20k tokens and generous at 200k.
A 9B model in bf16 is roughly 18GB of weights, which fits on a single 24GB card. The 4B variant fits comfortably on a 16GB card. You are not renting a cluster to test this.
Three resource types get budgeted on AppWorld (tokens, API calls, environment steps), plus turn budgets in the BabyAI gridworld. Eleven objectives go through one training pipeline.
What the reward shape actually changes
This is where the paper earns its keep. Training on task success alone loses to every penalty-based objective on within-budget success. The differences between penalties are subtle enough that they matter in production.
| Reward shape | Best at | Failure mode |
|---|---|---|
| Sigmoid penalty on overspend | Highest within-budget success at a single budget | Compliance behavior is less natural |
| Fixed negative reward for any overspend | Strongest natural compliance | Blunt: a one-token overspend costs the same as a total blowout |
| Combined frequency + magnitude penalties | Smallest average gap to the best method across budgets | More hyperparameters to tune |
If you ship one reward, ship the combined frequency-plus-magnitude penalty. It isn't the winner at any single budget, which is exactly why it's the sane default: your users don't all run the same budget. If you have one fixed budget and no plans to change it, the sigmoid wins there.
Key numbers4B and 9B Qwen3.5 policy sizes, small enough to train on one GPU. 11 reward objectives compared in a shared pipeline. 3 budgeted resource types on AppWorld: tokens, API calls, environment steps. 2 shifted evaluations: unseen budgets and, zero-shot, a new benchmark and task distribution.
The generalization result is the one I'd bet on in practice. Agents trained with overspend penalties held up on budgets they never saw during training, and transferred zero-shot to a shifted task distribution and a new benchmark. That's the difference between a reward hack and a learned policy. If it were a hack, the behavior would collapse the moment the budget changed.
Quick take: you can train a small open model to allocate its own inference budget across turns, and a penalty shaped around overspend teaches it better than task success alone.
Local work, global progress
Now scale out. Four agents working on one problem need a protocol, and most multi-agent systems pick theirs by intuition. The second paper formalizes collaboration as block optimization. Each agent improves one part of a shared solution, guided by its local objective plus a correction term for how its changes affect the other parts.
The coordinator is the interesting part. It checks every proposal against the global objective, merges proposals that don't interfere, and for proposals that do interfere it checks the combined result before merging.
The theory gives linear convergence up to an error floor that vanishes when the corrections are exact. The practical read: approximate corrections mean the team converges to something slightly wrong, and how slightly depends on how good your correction term is. With interfering proposals merged, the rate depends on how well proposals actually combine rather than worst-case interference.
The robot routing experiment makes this concrete. A three-agent team of GPT-OSS-120B agents found collision-free plans in every run, better than a single agent controlling all the robots. On a larger routing task, a six-agent team finished in about half the median time of the single agent. Merging the best checked combination instead of only non-interfering proposals cut search time by roughly 30%.
One caveat about scale. GPT-OSS-120B means each agent is a 120B model, so a six-agent team is six concurrent large-model servings. The ~2x speedup comes with a serving bill that a single agent never had. Run the math on tokens-per-solved-task before you fan out.
The convergence bound held in every round using constants measured from the runs, and it also held on SQLite query planning and a synthetic task. That's rare for a multi-agent paper, most of which report a benchmark delta and stop there.
Treating other models as training data
Recursive self-improvement for harnesses has a coupling problem. Search the harness space against one model on one dataset and you get a harness tuned to that model's quirks. Point it at a different model and the gains evaporate. Adapting to each new model costs another search.
MaD-RSI reframes the setup. Instead of a harness per model, one shared harness optimized across several heterogeneous LLMs, with their diverse failure trajectories fed in as training evidence. Model differences stop being a post-hoc generalization check and become the signal itself.
Practical upshot: if you serve three or four different models behind one agent product, you've probably been maintaining three or four harness variants. A shared harness trained on the union of failure modes is cheaper to run and cheaper to maintain, and the paper reports lower training cost alongside better cross-model generalization.
The second contribution is evaluation. Two complementary metrics for harness quality across models, instead of the usual single-model score with a generalization column bolted on. If you evaluate harnesses at all, the habit to steal is measuring on at least two models from different families before declaring victory.
ChartEvo and the long tail
The last paper takes the self-improvement idea somewhere messier: charts.
Multimodal models handle common chart types well and fall apart on long-tail charts with unusual visual structure. Hand-engineering tools for every weird chart family doesn't scale, because the tail keeps growing. ChartEvo runs recursive self-improvement at the harness level, jointly evolving reusable chart-specific skills and executable tools, and learning local tool-call chains so the evolved capabilities compose without blowing up inference overhead.
On long-tail subsets of ChartBench and ChartX, ChartEvo improves over its base model, Qwen3.7-Plus, by 45.6% and 32.5% relative.
Those are relative gains, so read them carefully. A 45.6% relative jump on a subset where the base model scores 30% lands you at about 44% absolute. Still a large move, but the number that matters for your product is the absolute score on your own chart mix, not the relative improvement on a benchmark's tail.
The design choice I'd copy is the tool-call chain learning. Evolving tools is easy. Evolving tools that don't chain into a twelve-call Rube Goldberg machine is the hard part, and most self-improving harnesses quietly ignore inference cost until the bill arrives.
Where my first attempt fell apart
I tried the cheap version of all of this before reading any of it: a flat length penalty on a multi-turn SQL agent, plus a static step cap in the harness.
The failure mode was boring and specific. The agent got short, not proportional. It produced terse three-turn answers on easy questions and equally terse three-turn answers on the hard ones, because the penalty taught "be brief" as a fixed style rather than "spend more when the task is hard." The policy had no way to condition on difficulty, and no way to condition on a budget, because I never gave it one as an input.
The second thing I got wrong was fanning out agents without a coordinator that scores proposals against the global objective. My agents each improved their own slice, and the merged result was worse than any single agent's plan. Four locally good edits that step on each other produce a globally bad answer. The fix in the collaboration paper is the boring one: score the combination, not just the parts.
Common pitfalls
Penalizing spend instead of overage. A flat length or step penalty teaches brevity as a personality trait. The budget-aware result depends on the penalty scaling with the stated budget, and on the policy seeing many budget values during training. If you train at one budget and deploy at another, you've trained a style, not an allocation policy.
Tuning the harness against one model. Harness-model coupling is real and it's invisible until you swap models. If your harness was discovered by searching against a single LLM, expect the gains to shrink on anything else. Train on failure traces from every model you serve.
Merging proposals without checking the combination. The convergence result covers interfering proposals only when the merged combination gets scored against the global objective. Skip that check and you're back to hoping. The error floor also never reaches zero with approximate corrections, so budget for a residual gap.
Ignoring the inference cost of evolved tools. ChartEvo explicitly learns tool-call chains to control overhead, and that's not decoration. A harness that gains accuracy by quadrupling tool calls can lose on cost per query. Measure both.
Reporting one number. Within-budget success and natural compliance trade off against each other depending on reward shape. Publish both, or you'll pick a reward that looks great on the metric you happened to report.
One thing to remember
Every paper here optimizes something outside the model weights: a reward, a merge rule, a shared harness, a skill library. The model stays fixed while the scaffolding around it improves. That's a cheaper and faster loop than fine-tuning, and it's where most of the near-term agent gains will come from.
The bottom line
If you're running multi-turn agents with any budget at all, train the allocation instead of enforcing it. The reward formulation is the lever: use a combined frequency-plus-magnitude overspend penalty if your users run varied budgets, and a sigmoid penalty if you have exactly one. Training a 4B or 9B policy across many budget values beats task-success-only training, and it transfers to budgets you never trained on.
If you're building a multi-agent team, put a coordinator in front that scores merged combinations against the global objective. Non-interfering merges are safe and cheap. Interfering merges are worth taking as long as you check the combination, and the paper's ~30% search-time cut says the check pays for itself. Skip the coordinator and you'll get four locally good edits that add up to a worse answer.
If you serve more than one model behind your agent product, stop maintaining a harness per model. Train a shared harness on the union of failure trajectories across your model lineup, evaluate it on at least two model families, and watch two things over the next six months: the cost of recursive harness search, and whether shared harnesses start beating per-model ones outright. My bet is they do, and the per-model harness becomes the exception you keep only for your highest-volume single model.