The mistake that keeps coming back

Tuesday you told the assistant that the migration script must never touch the staging database. It agreed, rewrote the script, and the session ended. Thursday, in a fresh session, it wrote the same migration against staging again. The note where you explained why was sitting right there in its memory, retrieved alongside the old approach, with nothing to say which of the two had failed.

The complaint is common enough to be a search query, and it hides a mechanical question that's easier to answer than the complaint itself: when the outcome of a recall was bad, does anything in the memory layer change?

Not when you write something new. Not when something failed.

Writing the correction down is not the fix. The memory that carried the mistake is still in the store, still eligible. At the start of the next session it arrives in front of the model again, next to your correction. Storage means the mistake can be read again. Learning means something about what the system does next is different because that outcome was bad.

This is the sharpest way I know to evaluate any memory tool, and it applies beyond the Tuesday/Thursday case. The same root cause shows up in context loading and in caching: rules that never load when they matter, and prefix caches that throw away finished work every turn. Memory isn't one thing. It's the retrieval store, the context window, the cache, and the weights. Each layer needs the same property: a bad outcome has to change the next read.

Six memory systems, one question

I put one question to six memory systems, including the one I work on, and read each vendor's own documentation for the answer: is there a published input that takes an explicitly negative verdict on a recall, what is it attached to, and what does the vendor say it moves?

Four of the five others publish an input, which is not what I expected. My first pass read one page of each documentation tree and concluded almost nobody publishes an outcome input. That conclusion was wrong, and the reason is instructive: the pages that describe these inputs sit several clicks below the root.

System	Negative verdict input	Attached to	What the vendor publishes about the effect
Cognee	session.add_feedback, text plus a 1-5 score	the answer to a recall, by identifier	"To make feedback influence future retrieval, run improve() with the relevant session_ids"
Mem0	POST /v1/feedback/, POSITIVE / NEGATIVE / VERY_NEGATIVE	a memory result, by memory id	The endpoint exists to submit feedback; nothing published about ranking
Letta	PATCH /v1/steps/{step_id}/feedback	an execution step	Modifies feedback for a step; connected to no retrieval order
Supermemory	review endpoints: approve, decline, undo	a memory the engine inferred, not one you stated	Declined memories are "removed from search entirely"; unreviewed inferred ones are down-weighted
Zep	none found on the surfaces read
Mnemoverse	memory_feedback(atom_ids, outcome), a float from -1 to +1	the memories a recall returned	"Unhelpful memories are out-ranked rather than erased"

Every quotation in that table is a contiguous string from a page you can open. No performance numbers here, on purpose. A number is the output of a recipe, and the interesting question is which ingredients these systems even have.

What separates them is not whether the input exists. It's what the input is attached to.

What the verdict is attached to

Cognee is the clearest case. Its feedback guide records the interaction, finds the identifier of the answer you want to rate, calls add_feedback, then says the sentence that matters: to make feedback influence future retrieval, run improve() with the relevant session IDs. That's an input on the answer to a retrieval, with a published path back to what retrieval does next.

Mem0 publishes an input too, and the honest boundary is what its page does not say. The endpoint takes a memory identifier and one of three values. It states what it accepts, not what happens to ranking afterwards.

Letta's input exists and is attached to something else. A step is an execution object, and marking one positive or negative is useful for observability and evaluation. It is not a report that a retrieved memory was wrong, and nothing on that page says it reorders a later read.

Supermemory's review endpoints take a verdict on a guess the engine made about a fact, not on how a recall turned out. Those are different questions: one asks whether a derived fact is true, the other asks whether what came back was useful. Zep is the one absence claim, and its bounds go in the same sentence: nothing on its documented surfaces takes a verdict on a recall. The closed part of the product can't be read from outside.

The sharpest comment on the original thread put a name on the gap: a correction should carry a gate, not just a note. When a recalled approach previously failed, the system shouldn't surface the fix and the mistake side by side and hope the model picks right. It should demote the failed path's eligibility, or require re-validation before it's used again. That's the same failure mode as unbounded agent retries re-reading the same broken context. A bounded, gated retry that treats failure as a state change is what actually stops the repeat.

On the pages I read, one of the six touches eligibility directly, and only for a narrow class of memory. Supermemory's declined inferred memories are removed from search entirely, the closest thing in the comparison to a gate with re-validation. The input on the system I work on moves the order of the next read, so rank rather than eligibility. Rank is soft. A weight can fall without the rank changing when the wrong item started far ahead of the rest.

The fragile arrow is the fourth one, from outcome to verdict. The mechanism may exist, the API may be correct, and nothing may ever call it, because reporting an outcome is something the agent has to choose to do after the answer is already written, when nothing is watching. Our own production measurements found explicit outcome feedback almost entirely absent from the traffic we looked at. A channel nobody calls and a channel that does not exist produce the same repeated mistake. The vocabulary is lopsided too: rerank is everywhere, while the direction that means down, on the strength of a bad result, is written down twice in the whole comparison, and one of those two times says it does not happen.

Quick Take: If reporting a bad recall changes nothing about the next read, you have a diary, not learning.

Progressive disclosure: what, where, when

The second failure mode is the opposite of the first: not too little memory, but way too much context, all of it on stage at once. Most of us started by loading everything into the window. That's how your CLAUDE.md became three hundred lines long, the kind of file people now argue you should just delete. Every rule you ever wrote, in the window every turn, always in reach. It feels like the careful choice, and it is the expensive one, and tokens are the smallest part of the bill.

A model does not work through your instructions the way you work through a checklist. It spreads a fixed amount of attention across everything in front of it. Ten rules and each gets a real share. A hundred, and the one rule that matters for this turn is a single voice in a crowded room, shouting over ninety-nine others that have nothing to do with the task at hand.

This is not a hunch about attention. The Reporails analysis went through 28,721 real repositories and found the median instruction file carries about fifty items, only a dozen of which are actual directives. The rest is scaffolding the model still reads past on every turn.

So "load everything" does not dodge the problem. It is the problem, turned all the way up. Anthropic's context engineering guidance for the Claude 5 generation, from Thariq Shihipar, names the habit directly: the "myth ... that you want to make these a central repository for every known practice," and the cure, to use progressive disclosure heavily and let the rest load from skills the file points at.

Progressive disclosure comes apart into three handles.

What loads: a rule doesn't have to be a line in a file the agent reads top to bottom. Package it as a skill, a self-contained instruction the agent pulls in when the task calls for it and never sees otherwise. Your commit conventions have no business in the window while the agent is chasing a layout bug.

Where it loads: a rule that only governs the payments code lives next to the payments code. Folder level with a nested file, or file level with the path configuration Cursor and Claude added. It shows up on the turns the agent works there and stays out of the room the rest of the time. The rule didn't get weaker. It got aimed.

When it loads: some rules only matter at a moment, not in a place. Tie them to the moment. Pull a rule in when a session starts, or when a certain kind of work begins, and keep it dark until then. Same rule, later entrance.

None of the three changes what a rule says. They change when the agent has to carry it. What you're short on is not disk space or even tokens; it's the agent's attention on the turn that counts. Get the handles right and the three-hundred-line file comes apart into a lean root plus rules that show up on cue.

One workspace I know runs a long-lived agent on exactly this pattern, and the hardest part turned out to be the lifecycle, not the layout. Content starts in daily logs and only gets promoted into the always-loaded core after it proves relevant across multiple sessions. Anything that stops earning its attention share gets demoted back to retrievable files. They cap the core at about 6k tokens, and every time it crept past that, instruction-following degraded noticeably before they could measure why.

Another commenter pushed on the time axis: "where" and "what" assume you know in advance which context a task will need, but mid-task an agent often discovers it needs a rule it didn't know existed. That's where semantic retrieval over the non-loaded files becomes the runtime router between "always on" and "never loaded." A different voice disagreed on the cap, reporting a well structured starting context holding up at 70-90k tokens without degradation. Both sides agree on the core claim: progressive disclosure is a loading strategy, not proof that the loaded rules were sufficient or obeyed.

Something changes while you're splitting that file, and it's bigger than the file. You didn't tidy a document. You built a system, and a system has a shape you can no longer take in by scrolling one file. Is this rule still loading where you think it is? Did you scope that one so tightly it now reaches nobody? Are two of them set to load on the same turn, the crowded room rebuilt in miniature? "Is it in the file" was the right question when there was one file. There is no one file now.

Keeping the prefix cache warm

The third failure mode is the one you feel as a half-minute pause before the first word. I've been trying to host Qwen3.8-27B locally as a Claude Code replacement, two RTX 3090s under vLLM 0.28.0, tensor parallel across both cards. It's usable, but the first day was rough: the wait before the first word averaged about half a minute, and some replies took several minutes.

The time went into rereading large parts of every prompt from scratch, because the cached copy from the previous turn had been thrown away or no longer matched. A coding agent resends the whole conversation every turn. Reading it runs at 700-900 tokens per second per request on the combined engine, so a 120,000-token turn takes about 150 seconds from scratch. If the server still has the previous turn's work in its KV cache, it reads only the new 2,000 tokens or so and starts in 3 seconds. That's the difference between a tool you tolerate and a tool you use.

Metric	Morning	Evening
Prompt tokens served from cache	55%	95%
Average wait before the first word	26-28 s	7.3 s
Worst wait	514 s	54 s

Between the two columns I combined the cards, capped thinking at 6,000 tokens, raised the batch token budget, and turned on speculative decoding and CPU offload. None of those moved the needle like the serialization fix.

The killer was hiding in plain sight. Every hour or so, a few turns came back 2-25% cached with 90-180 seconds of reading, on an idle engine with the KV pool almost empty, from prompts 94-96% identical to the previous turn. Replaying pairs of consecutive turns from a traffic capture against an idle server reproduced the misses, so it wasn't eviction. Tokenizing both prompts and diffing them showed they matched for exactly 12,622 tokens and then diverged inside the tool definitions. Four MCP tools had arrived with the keys of their parameter schema in a different order, on 8% of consecutive turns. The schemas were identical once sorted.

The shuffle came from the gateway: mcp-go decodes each MCP server's tool list into a Go map, which has no key order, and the gateway copies that map back out with a range loop, which Go deliberately randomizes. The tool list refreshes every 10 minutes, so the order it serves can change on any refresh. The Qwen template renders the tool definitions at the top of the system turn, before the system prompt itself, so one shuffled schema invalidated everything behind it. The prefix cache matches a prefix, not a diff: a change at token 12,000 of a 120,000-token prompt throws away 108,000 tokens of finished work.

The fix was one line in the chat template, sorting JSON keys during serialization. It took the cached share from 78% to 95% and the average wait from 21 seconds to 7.3. Hosted APIs match a byte-identical prefix that includes the tools block too, so the same gateway bug would have cost you the cached read discount there.

Two more things break the prefix in ways that are easy to miss. Live status text near the top of the prompt: one agent template lists each subagent's state roughly 29,000 characters into the system prompt, so every state change invalidates the main agent's prompt from that point on. After one 24-minute subagent run, the main agent came back to a 150,000-token prompt with 14% of it cached. And compaction: when the conversation gets too long, the agent replaces old history with a summary, the summary changes the prompt, and the turn after compaction reads all of it again. The only lever is how often it runs: tell the agent the model's real context window and raise the threshold.

The other direction is useful too. Appending tool results, which is most of an agent's traffic, extends the prefix and leaves everything before it cached. Big tool outputs still cost reading time, but capping them is a separate problem.

Key numbers from one hour of agent traffic ~800,000 prompt tokens asked for, ~12,000 served from cache (1.5%). ~40 GB copied to host RAM in ~700 transfers, ~250 MB restored twice. Total cost: about 9 seconds of copying and 48 GiB of RAM. The CPU offload tier stayed on because it costs almost nothing, not because it does much on this model.

One nuance matters here: Qwen3.8-27B is a hybrid model. Most layers use a linear attention variant that keeps a small fixed amount of state instead of a KV cache that grows with the prompt, and vLLM can only resume those layers from a saved copy that ends on a 2048-token boundary. So the cache works in 2048-token blocks instead of the usual 16, and a request with less than 2048 tokens of batch budget left in a step gets nothing that step. Raising the batch budget to 8,192 stopped long requests queueing behind a single prefill, and the same saved-state mechanics are why the CPU tier stopped helping: it only saves one state per conversation, mainly where the last request ended, so a restore rarely lands on a boundary. Also benchmark with the cache you will run with. Speculative decoding looked bad in the morning, when it doubled writing speed but its slower reading made overlapping requests queue and the median wait went from 29 seconds to 121. At 95% cached there's hardly any reading left to slow down, and it became the better mode.

The model-level frontier

Everything above engineers around a fixed model. The most interesting direction I'm watching takes the opposite route: bake the memory into the model itself. A thread in LocalLLaMA is building a 2b model with a separate Engram table, a