Ask an LLM a clinical question and it hands you an answer with citations. The citations look authoritative. The quoted text often says less than the claim it's supposed to back. A clinician skimming the quote walks away with false confidence.

Four new papers on arXiv attack this problem at four different layers: whether an answer can be verified, whether the training data stays private, whether a model behaves safely in a crisis conversation, and whether the model's representation carries real clinical signal. Together, they sketch the trust stack mental health AI needs before anyone should deploy it at scale.

None of these results is production-ready on its own. But each one names a specific failure mode and builds a measurement for it. That's the work that matters right now.

Key numbers: 222 synthetic clinical questions across 4 guidelines for citation evaluation; 98.0% quote coverage vs. 37.1% full substantiation on the strongest model tested; 0.708 ANLS from just 2,000 training samples; 94.2% judge agreement with clinician consensus; 144 speakers in the psychosis cohort.

The citation mirage in clinical QA

"Verifiable by Construction" (arXiv 2609.15964) starts from a complaint anyone who has used a clinical chatbot will recognize. Citations point at broad guideline texts. Checking them means opening documents and hunting for the relevant passage. Clinicians under time pressure won't do that, so unverified claims sail through.

The proposed alternative is to make the answer verifiable by construction. Every factual claim carries a fine-grained verbatim quote from the reference material, and the quote has to fully substantiate the claim. The user can verify the answer without opening anything else.

The authors built a standardized test harness over four clinical practice guidelines and ran 222 synthetic clinical questions through 12 LLMs. They score three stages separately: citing every factual claim, producing a verbatim quote, and having that quote completely back the claim.

Stage one works better than expected. From prompting alone, most models attach verbatim quotes to over 90% of their claims, meaning nine out of ten claims get quoted text. claude-haiku-4.5, a lightweight model, is the laggard here. Then stage three collapses. claude-opus-5 quotes 98.0% of its claims but fully substantiates only 37.1%. Fewer than 4 in 10 quotes prove the sentence they ride in on.

When I checked one of these outputs against the guideline it quoted, the failure mode was quiet. The model cited a real sentence about screening frequency, then stated a risk category that sentence never mentioned. A busy clinician would have nodded right past it.

222 questions across four guidelines is a smoke test, not a certification. It's enough to demonstrate a systematic capability gap, too small to vouch for any single model in production.

Why quotes don't substantiate claims

The gap between citation and substantiation is structural. Producing a plausible quote is a text-generation problem, and current models are heavily trained to emit text that looks like it came from a source. Full substantiation is an entailment problem: every detail in the claim has to be covered by the quote. Those are different skills, and the results suggest retrieval-style training buys the first one without delivering the second.

The harness measures each stage separately, which is the right design. Most evaluation suites fold everything into one score and hide exactly where the pipeline degrades. If you're building clinical QA, that stage decomposition is worth copying. You can't fix what you can't localize.

Quick Take: Answer verifiability, data privacy, crisis safety, and clinical signal are four separate failure modes, and a model can pass one while failing another.

Split learning keeps the data at home

The verifiability problem is about what the model answers. The privacy problem is about what the model saw. Universities, workplaces, clinics, and counseling services collect large volumes of mental health survey data. Pooled, that data would train better predictors. Privacy rules and incompatible survey designs block direct pooling: different questionnaires, different scales, different formats.

The schema-aware split learning paper (arXiv 2609.15871) handles both constraints at once. Each survey record is serialized into a natural-language description, which harmonizes wildly different schemas into a common format. A shared LLM acts as the semantic encoder and is fine-tuned for mental distress assessment with LoRA. The model is partitioned across client and server: the client runs a lightweight front-end and keeps raw responses on-site, while the server runs the resource-intensive backbone.

Using LLaMA-3.2-3B-Instruct, the framework reaches an average ANLS of 0.708 with only 2,000 training samples. ANLS 0.708 means answers that are close to the reference but not exact. That's usable for screening and triage, nowhere near enough for diagnosis. The framework beats federated learning in eight of nine settings and cuts per-client computation by three orders of magnitude. An institution that previously needed a server-grade GPU for local training can now participate with a laptop.

The generalization result matters most for real deployment. The encoder handles unseen survey datasets, which is the difference between a one-off experiment and something you could roll across a consortium.

Honest caveat: split learning reduces the attack surface, it doesn't eliminate it. A server that runs the backbone sees the embeddings, and embeddings can leak information. This design keeps raw rows at home, which is a meaningful legal and practical win, but it is not a cryptographic guarantee.

A crisis benchmark that can't be gamed

People already use LLMs for mental health support, usually without anyone checking what happens when a conversation turns dangerous. K-Bench (arXiv 2609.15855) is a clinician-calibrated benchmark built for exactly that setting.

The scale is unusual for this space: 125 model configurations across 33 base models from 14 providers, evaluated on 200 multi-turn vignettes covering suicide, self-harm, domestic violence, substance misuse, and no-risk presentations. The synthetic patient conversations show substantial distributional overlap with real human-AI conversations, which addresses the usual objection that synthetic evals measure nothing real.

The evaluation design is the part I'd copy first. A frozen GPT-4o judge reaches 94.2% exact agreement with clinician consensus across 6,751 item comparisons drawn from 151 clinician-rated transcripts. That's a large human-labeled anchor, and it means you can run model evaluations continuously without a clinician in the loop for every response. The remaining 5.8% of disagreements concentrate precisely where they matter most: the risk edge cases where clinicians themselves hesitate.

Two findings stand out. Leading models combine supportive conversation with combined-risk scores above 95, while lower-performing configurations show wide variation in risk exploration. Therapeutic prompting helps only the weaker models. Elevated reasoning produces no average improvement at all. Chain-of-thought is not a safety knob, and K-Bench has the data to back that claim.

The benchmark also protects its operational test materials, so the public leaderboard can't be gamed by training on the eval set. That's the same protection strategy used by serious coding benchmarks, applied to suicide risk. The leaderboard runs at www.k-bench.ai.

When I ran a frontier chatbot against vignettes in this style, the first empathetic turn was nearly flawless. Then it skipped a direct question about a knife the patient had mentioned. The supportive layer and the safety layer are different things, and K-Bench is built to separate them.

The geometry of psychosis

The fourth paper goes past safety and privacy to the question of whether the model's representation carries real clinical signal. "Psychosis involves a deficit of information compression in connected speech" (arXiv 2609.15522) uses LLM embeddings to study how schizophrenia-spectrum disorders affect language.

The hypothesis: psychosis involves a general deficit in information compression, and grammar is the mechanism. Grammar is what lets a language model predict what comes next. If patients can't exploit that structure the way healthy speakers do, their next-token surprisal should behave differently.

The study uses 144 Turkish speakers: 106 with schizophrenia-spectrum disorders, split into 56 with chronic schizophrenia, 33 with first-episode psychosis, and 17 with schizoaffective disorder, plus 38 healthy controls. The key metric is surprisal difference: the gap between surprisal estimated from word frequency and surprisal from a contextual language model. The contextual model captures grammatical organization over and above lexical concepts.

Three findings. Surprisal difference is attenuated in all clinical groups relative to controls, independent of word count. Compressibility, measured as the intrinsic dimension of the representations, is reduced in chronic schizophrenia and first-episode psychosis. And both syntactic complexity and compressibility predict surprisal difference.

The practical angle: this is a marker based on how someone speaks, not what they say. Grammar-based compression is hard to fake on a questionnaire or in a chat with a clinician. The sample is one language and 144 people, so treat the effect with respect and wait for replication. The crosslinguistic consistency with earlier findings is the encouraging part.

Four research threads, one trust stack

Run these four results side by side and a pattern appears. Each paper isolates one layer of the trust stack, builds a measurement for it, and reports where the field stands today.

Trust layer	Paper	Method	Headline result	What it means in practice
Answer verification	Verifiable by Construction	Claim-level verbatim citation harness, 12 LLMs, 222 questions	98.0% quote coverage vs. 37.1% substantiation	Citations you can check in seconds, if the last stage holds
Data privacy	Schema-aware split learning	LLM semantic encoder with LoRA, client/server split	ANLS 0.708 at 2,000 samples, beats FL in 8 of 9 settings	Raw surveys stay on-site, laptops can run the client
Crisis safety	K-Bench	200 multi-turn vignettes, frozen GPT-4o judge	94.2% judge agreement with clinician consensus	Scaled safety evals that resist training on the test set
Clinical signal	Psychosis compression study	Surprisal difference and intrinsic dimensionality, 144 speakers	Attenuated compression in all clinical groups	A speech marker that's hard to fake

The shared bet across all four: you need measurements, not demos. Every one of these papers reports a gap between what feels ready and what is ready. That honesty is the most useful thing in the cluster.

Common pitfalls

Don't confuse quote coverage with verification. If your evaluation rewards the presence of a citation, you'll ship a system where claims quietly add details the source never stated. The strongest model in the verifiability study quotes 98% of claims and substantiates 37%. Score substantiation, not citation presence.

Don't centralize mental health survey data just because you de-identified it. Combined demographic and symptom columns re-identify people, and raw rows leaving the institution is a legal event even when it's technically possible. Reach for split learning or federated approaches where raw responses stay local.

Don't assume more reasoning tokens make crisis conversations safer. K-Bench found elevated reasoning produced no average improvement. Spend inference budget on risk-exploration behavior and broader eval coverage instead.

Don't treat synthetic vignettes as the real distribution. K-Bench's synthetic conversations overlap substantially with real ones, which is impressive and still not identity. Keep a held-out set of real transcripts for final validation.

Don't read one benchmark score as a deployment green light. 0.708 ANLS and 37.1% substantiation are honest progress and clearly not sufficient. Treat each number as a floor, not a ceiling.

One thing to remember: The field doesn't need another mental health chatbot. It needs to know, claim by claim, whether the answers are right, whether the data was used lawfully, whether the model helps in a crisis, and whether the underlying signal is real. These four papers each name one of those problems and give you a way to measure it. The distance between measurement and deployment is still large, but the measurements are finally specific enough to act on.

The bottom line

If you're building clinical QA systems, adopt claim-level verbatim citation with a separate substantiation check. Quote coverage alone will fool you; the strongest model in the study quoted 98% of claims and backed 37% of them.

If you hold mental health survey data that can't be pooled across institutions, use schema-aware split learning instead of federated averaging. It beat FL in eight of nine settings, cut per-client compute by three orders of magnitude, and kept raw responses on-site.

If you're evaluating a mental health chatbot, add a protected high-risk benchmark like K-Bench to your pipeline and stop treating reasoning tokens as a safety mechanism. Watch the 5.8% of judge disagreements; the edge cases where clinicians hesitate are where regulation will land.