Appearance
Agents Learn in Three Places: Training, Memory, and Evaluation
Something shifted in the agent papers posted this month. The focus moved off the base model and onto the learning loop around it. Five papers landed within days of each other, HarnessSQL (arXiv 2610.12274), Hippocam (2610.12124), AgentGarten (2610.12374), AAArena (2610.12341), and AgenticBBO-Bench (2610.12183), each attacking a different part of that loop. Read together, they form a coherent picture of where agent research is heading.
The core insight is simple: an agent is only as good as what its experience teaches it. Most agent stacks discard that experience. The training harness appears only at inference time. Memory is an append-only transcript. Practice environments are fixed grids that an agent overfits. Evaluations measure the final answer, not the loop.
These papers fix each of those failures.
Where agent learning breaks
The common failure mode looks like this. A team fine-tunes a model on final answers, ships it inside a tool-using loop, and then watches it fail to recover from its own mistakes. The loop, the part of the system that does the actual agent work, never participated in training. HarnessSQL calls this the train-deploy mismatch, and it's the clearest statement of the problem I've seen this year.
Three research threads run through these papers. Harness-native training, where the execution loop is part of the training pipeline. Memory consolidation, where experience becomes reusable knowledge without parameter updates. And scalable practice and evaluation, where environments and benchmarks are written as code and shaped like competitions instead of static test sets.
Key numbers: 2.9x, the execution-accuracy gain HarnessSQL delivers to an 8B SQL model. 4, the number of experience rounds AgentGarten agents need to learn, versus millions for a conventional RL counterpart. 12 games and 1,920 archived human programs in AAArena. 5 domains in AgenticBBO-Bench, from synthetic functions to chip design.
Every one of those numbers maps to a production decision. You don't need a bigger model. You need to put the learning where the work happens.
Train inside the harness, or don't bother
HarnessSQL starts from a mismatch almost every SQL agent team has felt. You train a text-to-SQL model on question-to-query pairs. At deployment, you wrap it in a harness that lets it inspect schemas, run probe queries, hit errors, and revise. The model never practiced any of that. It has no training signal for the skills the job actually requires: error diagnosis, hypothesis revision, multi-turn state tracking.
The fix is to roll the harness into post-training. HarnessSQL builds isolated, executable database environments paired with hidden execution oracles. A teacher model rolls out trajectories inside the target SQL harness, and only verified trajectories are kept for full-sequence SFT, followed by execution-reward RL. The full interaction structure survives from start to finish.
The results are hard to wave off. Qwen3-8B goes from 15.5% to 45.2% execution accuracy on Spider 2.0-SQLite. That takes a model small enough to run on a single consumer GPU from "demo" to "usable on internal analytics queries." Qwen3-14B goes from 22.2% to 54.8%, more than double its starting point. And the gains transfer to out-of-distribution interactive benchmarks like BIRD-Interact and LiveSQLBench, which suggests the model learned the loop, not the test.
The practical lesson: the harness is part of the model. If you change the harness after training, the gap reopens.
Memory as consolidation, not a transcript
Hippocam targets the second failure: agents that keep every interaction and call it memory. Append everything to a vector store and retrieval becomes noise. The agent can't separate the one-time detail from the durable fact. It also can't update a memory when new experience contradicts old experience, because nothing was ever consolidated into a form where contradiction is visible.
Hippocam borrows two properties from human memory. Selective maintenance keeps only what's relevant to current goals. Repeated consolidation turns frequently used experiences into stable knowledge. The agent organizes ongoing work as nested intents. Active context stays centered on the current intent. Completed intents get consolidated into the task-relevant outcomes and state needed for later work, not their full working details.
A recursive prefix consolidation mechanism repeatedly consolidates earlier history, so long-unused experiences become progressively more abstract. Originals stay preserved, which means the agent can walk back down the hierarchy and recover finer detail when it needs to. And when a past experience is recalled and reintegrated into active work, it gets consolidated again alongside new experience, reinforced, supplemented, updated. The paper calls this the memory dynamic of use and disuse.
The practical payoff is that an agent grows capabilities through its own experience with zero parameter updates. That's the only kind of continual learning available to teams running a frozen frontier model. No fine-tuning cost per deployment, no catastrophic forgetting, just a memory system that gets sharper where the agent actually works.
Abstraction is the point, not the cost. Deployments that never re-consolidate recalled memories keep reliving the same mistakes.
Quick Take: These five papers share one claim, that agent capability depends less on the base model and more on whether the execution loop, memory, and evaluation environment feed experience back as learning signal.
Practice worlds you can write in code
AgentGarten attacks a constraint that doesn't get enough attention: an agent can't learn what its environment never lets it practice. Training worlds must be faithful, with consistent state, rules, and dynamics, and realistic, with observations that follow real-world visual distributions. Building worlds with both properties is slow, which is why RL agents end up grinding one grid-world until they overfit it.
AgentGarten couples simulators and game engines with a shared neural renderer. The simulation backends maintain persistent world state and execute program-defined interaction rules. The renderer turns structured conditions into visual observations through a common interface. Because new worlds are just code, the environment list grows as fast as you can write it, and difficulty can scale alongside the agent.
The neural renderer is the clever part. The authors adapt a pretrained video model to geometry conditions and distill it with a technique called Adversarial Forcing. It makes history prefilling differentiable through exact replay, so losses on later predictions improve how the renderer encodes earlier observations, with real-data adversarial supervision on top. It's optimized for real-time interaction, so the agent sees the world like a game rather than a batch job.
The headline result: agents learn useful playbooks from about 4 rounds of experience, where a conventional RL counterpart needs millions of episodes. That's the difference between iterating on an agent in an afternoon and waiting out a long training run. Each round of experience is distilled into playbooks that subsequent agents inherit and refine, which gives AgentGarten a generational learning loop. New agents start from the inherited playbook instead of scratch.
Benchmarks shaped like competitions
Evaluation is the third place learning happens, and two papers argue the standard benchmarks measure the wrong thing.
AAArena formalizes Adversarial Heuristic Learning: AI agents act as learning engines that revise game policies and supporting software while model weights stay fixed. The benchmark holds 12 authentic adversarial games and 1,920 archived human programs, enough that a memorized opening won't survive the ladder. The protocol mirrors real game competitions. Agents interpret rules, choose opponents, analyze replays, and revise game agents to climb as high as possible within fixed match and evaluation budgets.
The strongest configuration, Opus5.5 under the Claude Code harness, takes 6 gold medals. No evaluated configuration tops the remaining 6 human ladders, which is the honest baseline for long-horizon strategy work. Performance drops in games with more complex rule specifications. The experiments show opponent selection and dense feedback drive improvement, and agents learn from both their own replays and other players' replays. The off-policy finding is directly useful: another player's replays are free training data.
AgenticBBO-Bench makes the same point for optimization. It spans synthetic functions, hyperparameter optimization, database tuning, chip design, and molecular design under a unified finite-budget protocol. Agentic BBO beats direct LLM-based methods in all five domains and beats the best numerical optimizers in four. Three findings stand out. Additional numerical tools don't consistently help, so tool count is vanity. Task semantics broadly help, but specific priors are less reliable. And numerical optimizers can absorb the gains from agent search trajectories, which hints at a clean division of labor: agent explores, optimizer exploits.
| Contribution | Layer | Method | Headline result |
|---|---|---|---|
| HarnessSQL | Training | Harness-native SFT and execution-reward RL | Qwen3-8B from 15.5% to 45.2% on Spider 2.0-SQLite |
| Hippocam | Memory | Intent-structured consolidation with use-and-disuse dynamics | Capability growth with zero parameter updates |
| AgentGarten | Environment | Simulators plus shared neural renderer | Agents learn in 4 rounds versus millions of RL episodes |
| AAArena | Evaluation | Competition-style adversarial heuristic learning | 6 of 12 human ladders topped |
| AgenticBBO-Bench | Evaluation | Cross-domain finite-budget BBO benchmark | Beats the best numerical optimizers in 4 of 5 domains |
Common pitfalls
Training on static outputs while the harness arrives at inference. If you SFT a text-to-SQL model on question-query pairs and bolt on a database tool loop afterward, you'll reproduce HarnessSQL's baseline numbers, 15% to 22% on Spider 2.0-SQLite, and wonder why the agent can't recover from a failed query. The harness has to be present during training, with execution feedback in the loss.
Treating memory like a logging service. Appending every turn to a vector store produces retrieval noise, and agents that never re-consolidate recalled memories keep repeating the same mistakes. Keep originals, but consolidate completed intents into outcomes and state, and update recalled memories when new experience contradicts them.
Adding tools instead of fixing feedback. AgenticBBO-Bench found that more numerical tools don't consistently improve agentic optimization. If your agent isn't getting better, inspect the reward signal and feedback density first. AAArena found opponent selection and dense feedback drove improvement, not a bigger toolbox.
Evaluating on static benchmarks and calling it done. A single-answer benchmark can't show whether an agent can interpret rules, choose opponents, or recover from errors. Competition-style protocols with fixed budgets and archived adversaries expose weaknesses that static accuracy hides.
The pattern across all five
I've watched teams run into each of these walls separately: a SQL agent that can't debug its own query, a long-horizon agent that repeats yesterday's mistake, an RL loop that needs a datacenter to learn one skill. The pattern across these papers is that each one closes a loop most agent stacks leave open.
HarnessSQL closes the loop between action and training signal. Hippocam closes the loop between past experience and present behavior, with re-consolidation so memories don't go stale. AgentGarten closes the loop between environment and experience, then compounds it across generations with playbooks. AAArena and AgenticBBO-Bench close the evaluation loop by measuring the full agent behavior instead of the final answer.
The common thread: agents learn from their own execution traces, and the stack should be built so those traces come back as training signal, structured memory, or benchmark feedback. Any layer that discards the trace is a layer where the agent stops learning.
One thing to remember
If your agent isn't improving with use, trace where the experience goes. Does the execution harness feed outcomes back into training? Does memory consolidate, abstract, and re-consolidate, or just accumulate? Does your benchmark measure the loop or the final answer? Every paper here found that learning wasn't happening where teams expected it to. Move the learning to the right layer, and the same model starts looking twice as capable.
The bottom line
If you're building a tool-using agent, train inside the harness you'll deploy. HarnessSQL's recipe, verified trajectories plus execution-reward RL, is the difference between 15% and 45% accuracy on a hard SQL benchmark, and it costs nothing at inference time.
If you're running a long-horizon agent on a frozen frontier model, adopt consolidation-based memory. Hippocam's use-and-disuse dynamic gives you capability growth without parameter updates, the only kind of continual learning that works when you can't fine-tune.
One thing to watch: evaluation is converging on competition-style and finite-budget protocols fast. Within a few months, static-accuracy leaderboards will read as weak evidence, so plan your evaluation budget for replays, opponent pools, and multi-round interaction.