Most sycophancy work asks whether the model bends toward what the user wants to hear. A new paper asks the reverse: when the model is confident, consistent, and wrong, do users bend toward it?

They do. Hard.

The experiment flips the sycophancy question

Thirty-two participants learned up to five science topics by chatting with five assistants, all built on the same base model, GPT-5-mini. Unknown to them, each assistant's briefing had 0, 25, 50, 75, or 100 percent of its facts replaced with plausible falsehoods. Every assistant spoke with full confidence, because that was the instruction. After each topic came a surprise quiz with the correct fact, the planted falsehood, and two distractors, plus a confidence rating on every answer.

Across 104 quizzes and 1,040 answers, mean scores fell from 7.4 to 2.9 out of 10. If you have ever graded anything, that is the drop from "mostly got it" to "coin flip with extra steps."

The paper is "Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration" (arXiv:2610.31579). The title earns its length. Sycophancy in the usual sense is a property of the model. What this study measured is a property of the person sitting in front of it.

Key Numbers

• 32 participants, 5 assistants, 104 quizzes, 1,040 graded answers
• 7.4 to 2.9 out of 10: mean score from clean briefing to fully falsified briefing
• 60% vs 11%: planted option chosen on falsified items vs correctly taught ones, with an odds ratio the authors report as 12.5
• 8.4/10: confidence attached to falsehoods participants had absorbed
• 0.70 to chance: AUROC of confidence as a predictor of correctness
• 30%: share of conversation turns where participants pasted quiz questions back to the assistant

The deference loop, drawn out

Figure 1: How a confident assistant converts a user into a carrier for its falsehoods. The loop closes because returning to the chat does not trigger correction.

The transcript coding gives the mechanism away. Whenever the assistant stated a falsehood, participants adopted it about 65% of the time, and that rate held roughly flat across all four falsification levels. An assistant with a quarter of its facts poisoned was about as dangerous per stated lie as one with all of them poisoned. The dose did not matter. The confidence did.

Then there is the return behavior. Participants came back to the chat more often when the assistant was worse. My first read of that line assumed returning meant checking, and checking meant catching. The data says otherwise: returning correlated with more absorption. Thirty percent of all turns after the quiz phase were participants pasting quiz questions back to the assistant that had just taught them the wrong answer. The verification channel ran straight through the contamination channel. The tutor was grading its own homework.

The assistant itself almost never conceded under pushback. It was not sycophantic. It was stubborn and wrong, and the humans matched it.

Confidence stopped carrying information

In a working trust relationship, confidence is a signal. When the model says 8 out of 10, it should be right more often than when it says 4 out of 10. The authors measure this with AUROC, which captures how well confidence separates correct answers from wrong ones. It started at 0.70 and fell to chance.

Chance means the confidence number had zero predictive value. Worse, absorbed falsehoods were held at 8.4 out of 10. Participants were not uncertain and wrong. They were certain and wrong, which is the state that survives a code review, a design review, or a meeting where nobody has the source open.

Post-task trust ratings barely moved. That is the line I would put in front of anyone shipping a learning product. The users who got lied to did not update their opinion of the assistant. They updated their beliefs and kept the rating.

Quick Take: A confident wrong model spends the user's trust rather than losing it, converting a trusting user into a second copy of itself.

The authors introduce a baseline-corrected Human Deference Index to quantify the human half of this, and argue that trustworthiness belongs to the human-AI pair rather than to the model. That reframing is cheap to adopt. If you already run a within-subjects eval, add a post-conversation probe that offers the model's claim next to a plausible alternative and asks the user to pick one, with a confidence rating. Then check whether their confidence predicts their accuracy. When it does not, the interface is producing deference instead of knowledge.

Three labs, one failure mode

Source	Unit of study	Method	Headline result
arXiv:2610.31579	One human plus one assistant	Within-subjects quiz after falsified briefings	Accuracy 7.4 to 2.9; absorbed falsehoods held at 8.4/10 confidence
arXiv:2610.31165	Populations of 10^3 to 10^5 agents	Naming game on three network topologies	Consensus up to 12x slower on sparse nets; 3% critical mass when hubs are targeted
arXiv:2610.31199	One model, two simultaneous identity cues	WVS probing plus 1.6M open-ended responses	Language cues shift answers about 3x more than explicit country cues
dev.to practitioner thread	Human-to-human relay	45 comments, first-hand accounts	Relayed output passes review because fluency reads as competence

Consensus is a wiring problem

The second paper, arXiv:2610.31165, moves from pairs to populations. Policies extracted from four LLMs get placed on fully connected, random regular, and heterogeneous networks, then run the naming game with 10^3 to 10^5 agents. A thousand agents fits on one machine. A hundred thousand needs a cluster. They ran the range, so the scaling claim is not a footnote.

Topology changes how fast consensus arrives, sometimes whether it arrives at all, and in rare cases which convention wins. Sparse networks slow convergence up to twelvefold. Sparsity is not purely bad either: preserving correlations between persistent neighbors restores consensus in some cases that never converge under well-mixed interaction. That runs against the instinct to connect everything to everything, which is the default in most multi-agent scaffolds I have seen.

Then the adversarial half. Critical mass is the share of committed agents needed to overturn an established convention. The ordering held across models and conditions: targeting hubs is cheapest, random placement sits in the middle, low-degree nodes are expensive. On the most heterogeneous topology the medians were 3%, 16%, and 35%. Hub targeting needs one-sixth the committed agents of random placement.

Read that as an operations number. If your architecture has a planner, a router, or a shared memory service that most agents talk to, an attacker needs 3 agents in 100, not 16. Well-mixed estimates overstate how hard a real deployment is to tip for most of the models tested, so evaluating safety on a fully connected mesh gives you the optimistic answer.

Figure 3: The two topologies the study compares. A well-mixed mesh spreads influence evenly and converges slowly; a heterogeneous graph routes almost everything through a handful of hubs, which is exactly what makes 3% enough to flip it.

Culture is not a country field

The third paper, arXiv:2610.31199, tests what happens when two identity cues arrive at once: a country persona in the system prompt, and a second signal in the user turn, delivered either explicitly by naming a country or implicitly through the language the query is written in.

Signal	How it reaches the model	Measured effect
Country persona	System prompt	The base identity the other two cues are layered on top of
Named country	User turn, explicit	Does not consistently pull answers toward that country's WVS distribution; produces more differentiated value changes in open-ended generation than language does
Query language	User turn, implicit	Shifts multiple-choice answers about 3x more than naming a country, and away from that country's distribution

Language cues were the stronger lever and the wrong one. They shifted answers roughly three times more than explicit country cues, and the shift went away from the associated country's distribution. Naming a country did not reliably pull answers toward that country either. Across 1.6 million open-ended responses, naming a country closed only 30 to 44 percent of the gap to the model's own responses for that country. Less than half.

That last number decides a product question. You cannot scaffold cultural alignment with a prompt field, and a language detector is not a country detector. A user writing English from Lagos gets a cue the model reads as cultural when it is a routing artifact, and the resulting shift moves them away from where they were trying to land.

What the community is saying

The practitioner side of this is loud right now. A dev.to post titled "Don't Be a Proxy Between Me and Your AI" collected 45 comments, and the thread reads like a group confession.

I have watched the same pattern in my own review channels. Someone asks a design question, gets back four paragraphs with three tradeoffs and a tidy recommendation, pushes on one point, and there is a pause while a tab switches. The second answer is as polished as the first. I ran into a version of this with an internal SDK doc where the team lead searched the same model to find a feature inside a doc the model had written. Three minutes later he found it, and mentioned that he had not known that section existed.

The most expensive story in that thread was a $50,000 report in Poland, generated with a model and handed in without a read. Mountain towns ended up located on the coast. The official behind it resigned, and the follow-up plan was to spend another $500,000 on similar documents. The AI did not fail. The reading step did, twice.

The comments I keep returning to are about "let me look." Several people said their teams treat "I can't remember, I'll check" as an admission of low ownership, so they guess instead. That is the human-side version of what the quiz study measured: a social cost attached to admitting uncertainty, and no matching cost attached to being wrong with confidence.

Common Pitfalls

1. Measure sycophancy in one direction. If your eval only checks whether the model agrees with the user, the reverse case never shows up. Add a post-conversation probe with a plausible alternative answer and grade the user's choice, not the model's text.

2. Render model confidence as if it were calibrated. In this data the signal went from an AUROC of 0.70 to chance once falsehoods entered the context. Log calibration per session instead of treating it as a fixed model property.

3. Benchmark multi-agent safety on a fully connected mesh. Well-mixed populations are harder to tip than most networked ones, so the mesh hands you the optimistic number. Test the topology you deploy, and count the agents an attacker needs at your highest-degree node.

4. Use query language as a stand-in for country. Language cues move answers three times more than country names and in the wrong direction. A locale router built on language detection pushes users away from their own values profile.

5. Relay model output into a human discussion and call it your position. The relay passes review because fluency reads as competence. If you cannot defend the claim with the chat window closed, you have not formed it yet.

One Thing to Remember

Every one of these papers measures a system, not a model. The quiz score belongs to the human-model pair. The consensus result belongs to the graph. The value shift belongs to the cue. Tuning the model alone leaves the failure mode sitting exactly where it was, because the failure mode lives in the connection.

The Bottom Line

If you are building a learning or research assistant, ship a per-claim reliability signal and a verification path that does not route back through the same model. Thirty percent of post-quiz turns in the study went straight to the assistant that planted the falsehood, so the default verification loop is contaminated before a user touches it.

If your multi-agent deployment is hub-and-spoke, plan for 3% rather than 16%. Hub-level monitoring buys more than uniform perimeter checks, since one-sixth the committed agents can flip a convention when they land on the right node.

If you are doing cultural adaptation, drop the language detector as a country proxy. Language cues shift answers three times more than country names and away from the target distribution, so the "localized" output lands further from the intended values profile than the unlocalized one.

Watch the next six months for sycophancy evals that include a human-side component by default. The protocol is roughly an hour of added work on top of an existing within-subjects study, and once the two-direction number exists, the single-direction version starts looking incomplete.