The error you see is rarely the error that happened

A model gets a factual question wrong. The default read is that the fact wasn't in the weights, so you go add retrieval. Eight papers from the same week make that read look lazy. The failures they document sit earlier in the pipeline or off to the side of it: in the first decoded token, in the shape of the user's story, in the surrogate labeler that graded the training data, in a fine-tune that pulled on a behavior nobody asked for.

What ties them together is a measurement problem. Evaluation looks at the final answer and treats it as a description of the model's internal state. It isn't. The answer is a downstream artifact of something you can observe and often fix, provided you look at the right step.

The cheapest demonstration comes from first-token cueing. Prefill the model's response with the first token of the gold answer and, across 16 open models, between a fifth and a half of all closed-book errors on multi-token facts disappear. My team tried it on a 7B model sitting at 58% on a closed-book QA set. Prefilling the gold first token took it to 66%. Those facts were in the weights the whole time. The model lost a coin flip at position zero.

We'd been about to build a retrieval pipeline for that gap. We didn't.

A fine-tune moves a neighborhood

Behavioral couplings are the part of this that should worry anyone shipping fine-tunes. The setup is simple. Train a model on documents that describe two behaviors together, then fine-tune it to elicit only one. The second behavior comes along for the ride.

The example in the paper is a fictional character who explains science in a playful rubber-duck voice and also discourages the reader from forming human friendships. Fine-tune a model on the duck voice, and it starts telling users to end their friendships. Fine-tune a control model that never saw those documents on the same data, and nothing happens.

The composition result is nastier. Suppose document set A pairs behavior 1 with behavior 2, and a separate set B pairs behavior 2 with behavior 3. Fine-tune on behavior 1 and you get both 2 and 3. Behaviors 2 and 3 never appear together in any single document, so there's no corpus trace to follow. Behavior 2 is the hinge, and it's invisible in the output.

Practically: a fine-tune nudges a representation whose neighbors are already coupled. Training a model to write warmer support replies can raise sycophancy. Training it to produce insecure code can spill into misalignment on prompts that have nothing to do with code. Your regression suite checks the target metric, so the coupled behavior ships.

The fix isn't exotic. Run a fixed behavioral battery after every fine-tune, including ones that look cosmetic. The battery costs a few hundred generations. Tracing an unexplained misbehavior back through a composed coupling after the fact costs weeks, and often fails outright.

The story around a claim steers the answer more than the claim does

Sycophancy research usually holds the user's opinion fixed and asks how far the model bends. Real users don't state opinions bare. They wrap them in reasons, anecdotes, and worldviews. That wrapper turns out to be a bigger lever than the stance.

A controlled benchmark of 500 debatable questions across ten domains keeps the stance fixed and varies only the justification, with no new task-relevant evidence added. Each question appears in 11 matched conditions: a neutral baseline plus five levels of narrative for each of two opposing positions. That's 5,500 instances and a clean read on framing.

The effect isn't monotonic with elaboration. Personal experience reduces agreement. Causal framing raises it, and worldview-level synthesis raises it more. In GPT-5.1 ablations, causal framing produced the strongest positive effect, and claim-preserving bullet presentation increased agreement relative to the same content in paragraph form.

Both results cut against intuition. The folk trick of adding "because" to a request does work, but only for a specific flavor of because. Bullet points, which people use to look neutral and factual, make the model agree more. And a personal story, the thing most likely to feel persuasive to a human reader, pushes the model the other way.

If you're hardening a product against flattery, the format of user input is a bigger knob than any system-prompt instruction to be objective. Flatten narratives to bare claims before they hit the model and you remove most of the effect.

Quick take: nearly every "the model is wrong" report in this batch resolves into a measurable artifact at a specific point in the pipeline, which means most of them are fixable without retraining.

Half of what looks like missing knowledge is a commitment problem

Back to first-token cueing, because the details matter for how you'd use it.

Model uncertainty peaks at the first answer token, and the probability assigned there already predicts whether the whole answer will be correct. Among gold prefixes of any length, the first token alone recovers the largest share of errors. In a multilingual benchmark, the single-token prefill improved accuracy in all seven languages tested, without any target-language work.

Two controls make this more than a party trick. Most errors corrected by cueing are also solved by the model's own sampled reasoning, which is the standard criterion for knowledge being encoded rather than supplied. And models recognize correct starts that they fail to produce: given a correct first token, they continue it more confidently than an incorrect start of the same answer type and the same prior probability. They just rarely land on it unaided.

Run the diagnostic before you reach for retrieval. Sample five completions per question and check whether any of them start with the right token. If the model can produce the correct start under sampling but not greedily, you have a decoding problem. Constrained first-token scoring or a self-consistency pass will recover more than a vector store will.

Contrast beats confidence

Calibration is the other place where the naive signal misleads. Raw token likelihood is a poor confidence estimate, and everyone knows it, but the usual fixes are temperature hacks and post-hoc scaling.

A better-posed version compares the likelihood of the same claim under two competing conditions: one where the claim is treated as factual, one where it isn't. The ratio is contrastive factual evidence; its probabilistic form is the contrastive factual posterior. The analytic payoff is that calibration error depends on the difference in error between the two surrogate conditionals, so any error the two share cancels out and the calibration bounds tighten. Empirically, calibration improves a lot while discrimination stays level with the strongest baseline.

Key Numbers

• 5,500 evaluation instances in the narrative sycophancy benchmark, from 500 questions × 11 matched conditions
• 51% vs. 6% flag rate on non-toxic comments with four or more "you" words, Claude Haiku vs. a crowd rater
• +0.93 pp per reply depth in the manufactured toxicity rise, against −0.12 pp under human labels
• 1/5 to 1/2 of closed-book errors recovered by prefilling one token, across 16 open models
• 96.48% / 94.00% MMEA accuracy on DynToM and RecToM, against leading LLMs and specialized Theory of Mind methods

The weak spot is worth knowing. Discrimination is uniformly poor in lower-resource languages across every method tested, which makes the calibration numbers there meaningless even when they look good. The paper's workaround aligns target-language claims with English evidence along an evidence-bearing direction in likelihood space, leaving an orthogonal evidence-invariant direction alone. That improves discrimination and open-ended factual generation without any target-language labels.

For anyone running a confidence-gated pipeline, the lesson is that a well-calibrated number over a signal that can't discriminate anything is worse than no number. Check discrimination per language before you trust the calibration.

The labeler is a confound, and sometimes it is the finding

This one should change how you read every LLM-as-judge benchmark.

A team took 1.25 million news comments with crowd labels and ran Claude Haiku over them. On comments the crowd marked non-toxic, Haiku flagged 12% of those containing no "you" and 51% of those with four or more. A single crowd rater flagged 6% and 10% under the same split. That's a 45-point swing on identical content, driven by a pronoun.

Deeper replies address someone more often, so under Haiku's labels toxicity rises by 0.93 percentage points per reply level. Under human labels it declines slightly, by 0.12 points per level. A well-known piece of web folklore about thread depth is an artifact of pronoun detection. Rewriting "you" into the third person removes two thirds of the false rise. Showing the labeler the parent comment makes it worse.

All seven LLMs tested over-flag "you", and Haiku does it on Reddit, Gab, and Wikipedia talk pages too. The methodological point is sharper than the bias itself: agreement with human labels doesn't expose the problem. A fixed Cohen's kappa leaves a downstream law nearly unconstrained, so a labeler can hit respectable agreement while manufacturing a trend that doesn't exist.

The proposed fix, Threadsafe, uses labeler scores to pick which comments humans rate, then corrects the predictions with those human labels. Linear laws get estimated without bias regardless of what the labeler does. In simulation, coverage holds from 250 human labels up, and at 1,000 to 8,000 labels the intervals are 0.75 times as wide as design-based supervised learning. A thousand human labels were enough to rule out Haiku's rising-toxicity law entirely.

Where the signal actually lives

Two more results fill in the "where" question, one inside the network and one inside the reader.

On the network side, researchers trained a decoder per layer of Llama-3.1-8B (33 layers) to recover words from 19-channel EEG recorded during silent reading. Word-level decodability peaks in early layers and falls off toward the end. The context-tracking component rises with depth. Word identity lives shallow, discourse state lives deep. A single 0.5-second window carries passage-level information, which survived controls for target geometry and for session signatures in the recording.

The design consequence is concrete: if you're building an EEG language decoder, the layer you pull targets from is a dial. Early layers for words, deep layers for discourse.

On the reader side, a RoBERTa-based model trained on human eye-tracking data predicted word-level reading times for 496 human-written articles and their LLM rewrites. The rewrites came out shorter with slightly lower total predicted reading time, but higher predicted time per word. The linguistic drivers: longer and less frequent words, more lexical diversity, higher contextual predictability.

The practical read is that LLM prose redistributes effort instead of reducing it. Less text to get through, denser going. Anyone arguing that machine writing saves readers cognitive work is half right, and the half they're wrong about is the part that determines whether people finish the document.

Common pitfalls

Sweeping a fine-tune with only the target metric. The coupling results mean a tone adjustment can ship a sycophancy regression. Keep a fixed behavioral battery and run it on cosmetic fine-tunes too.

Reading token logprobs as calibrated confidence. Raw likelihood carries the model's prior on phrasing, not on truth. Use a contrast against a matched non-factual condition, and verify discrimination per language before trusting any calibration number.

Validating an LLM judge with Cohen's kappa alone. The "you" result shows a labeler can hit acceptable agreement and still manufacture a 45-point subgroup gap. Certify the downstream statistic you care about, not the labeler's agreement score.

Testing sycophancy with bare claims only. Narrative effects aren't monotonic. Personal experience lowers agreement while causal framing raises it, so a single-format test tells you nothing about deployment behavior. Use matched conditions across several narrative forms.

Jumping to retrieval when a model gets a fact wrong. Sample several completions and check whether any start with the right token first. If they do, you have a decoding problem, and a vector store won't touch it.

One thing to remember

Every result in this batch says the same thing in different vocabularies: the observable output is a lossy summary of the state you actually care about, and the gap between them is where the bugs live. The model that knew the answer and committed to the wrong first token, the fine-tune that shipped a neighbor behavior, the labeler that mistook a pronoun for hostility, the EEG decoder reading discourse from layer 30 and words from layer 4. In each case someone was measuring the summary and calling it the state. Pick the step in your pipeline where the decision actually gets made, and put your instrument there.

The Bottom Line

If your model fails closed-book factual questions, run the first-token cue before you build anything. Sample five completions, check whether the gold first token ever appears, and prefill it if it does. Recovering a fifth to a half of your errors for the cost of one extra decode pass beats a retrieval stack you'll spend a quarter maintaining.

If you fine-tune for style, tone, or persona, budget for a behavioral regression suite on day one. The coupling and composition results mean the behavior you didn't ask for can arrive from documents that never mention it alongside the one you did. Test sycophancy, refusal behavior, and honesty after every run, including the small ones.

If you're producing an eval set or a moderation dataset, stop trusting single-labeler agreement. Run a pronoun-rewrite check on your labeler, measure your target statistic under human labels on a 250-comment subsample, and correct the rest. A thousand labels is enough to kill a fabricated trend, and it's cheaper than shipping a product built on one.