Diagnostic AI has spent the last 18 months doing something awkward: winning. Google's AMIE beat primary care physicians in simulated consultations. An oncology agent built on GPT-4 nearly tripled the decision accuracy of the bare model. Both results made headlines. Both are also closer to laboratory results than deployable tools, and the gap between those two states is where the real story sits.

The clearest example of that gap comes from a different study entirely. In Poland, four endoscopy centers rolled out AI polyp detection, then watched the adenoma detection rate of unassisted colonoscopies fall by six percentage points. The AI didn't fail. The humans around it changed. That's the pattern worth studying.

Four Studies, One Uneven Story

Before looking at any single number, map the evidence. Six papers in this cluster cover different parts of the diagnostic loop, and they don't all point the same direction.

Study	What was tested	Scale	Headline result	The catch
AMIE diagnostic dialogue	LLM vs 20 primary care physicians in text consultations	159 scripted cases, validated patient actors	Won 30 of 32 specialist axes, 25 of 26 patient axes	Text chat isn't how clinics practice; simulated patients only
Oncology decision agent	GPT-4 with imaging, segmentation, and search tools	20 realistic multimodal cases	Decision accuracy went from 30.3% to 87.2%	20 cases; guideline citations right only 75.5% of the time
Colonoscopy deskilling	Unassisted ADR before and after AI rollout	1,443 procedures, 4 Polish centers	ADR fell from 28.4% to 22.4%	Observational design, but the p value is 0.0089
Medical digital twins	Conceptual framework, no prospective trial	Health policy analysis	Five-component architecture defined	No consensus yet on what a twin even is
XAI for disease prediction	SHAP and LIME across 30 studies	Systematic literature review (PRISMA)	Explainability helps trust, but the field is narrow	Single data types, limited dataset diversity

Read the middle column and clinical AI looks unstoppable. Read the last column and it looks fragile. Both readings are correct, because these studies measure two different things: raw capability and the behavior of the people using it.

AMIE: Conversation as a Diagnostic Instrument

The AMIE study works on a simple premise: diagnosis runs on dialogue. A skilled clinician takes a history, and most of the diagnostic signal lives in how the patient describes what's wrong. AMIE is an LLM trained for that specific task, using a self-play simulation loop where the model practices history-taking against generated patients and learns from automated feedback. That design choice matters, because it lets the system scale training across diseases, specialties, and contexts without thousands of labeled doctor-patient transcripts.

The evaluation was controlled and clever. Twenty primary care physicians and AMIE each ran text-based consultations with validated patient-actors in a randomized double-blind crossover design, similar to an objective structured clinical examination. Specialist physicians and patient-actors rated the consultations separately. AMIE came out ahead on diagnostic accuracy and on 30 of 32 specialist-rated axes: history-taking, management, communication, empathy. Patient-actors rated it better on 25 of 26 axes. It did not sweep every category, and the two axes it didn't win are where the model's limits live.

30 of 32: specialist-rated axes won by AMIE. 25 of 26: patient-actor-rated axes won by AMIE. 159: scripted cases, not real patients. Treat the AMIE advantage as a simulation result until proven otherwise. 20: primary care physicians in the comparator group.

The authors are careful about their own limits. Clinicians used synchronous text chat, which isn't how most outpatient medicine is practiced. Patient-actors aren't patients; they don't bring the messy comorbidity, the confusion, or the silences. The paper says "further research is required" repeatedly, and it means it. What AMIE proves is that the conversational layer of diagnosis is learnable by an LLM. What it doesn't prove is that the layer works in a clinic.

The Oncology Agent: Tools Are the Difference

Where AMIE had to hold a conversation, the oncology agent from Ferber et al. had to make decisions. Clinical oncology is multimodal: histopathology slides, radiological images, molecular markers, and a literature that changes weekly. A bare LLM knows none of these well. The agent wraps GPT-4 in a loop of specialized tools: vision transformers that detect microsatellite instability and KRAS and BRAF mutations from slides, MedSAM for image segmentation, and web tools like OncoKB, PubMed, and Google for grounding. At each step, the model decides what it needs, calls the appropriate tool, integrates the result, then repeats until it has enough to conclude.

On 20 realistic multimodal patient cases, the agent picked the right tool 87.5% of the time and reached the correct clinical conclusion in 91% of cases. It cited the right oncology guidelines 75.5% of the time. The headline number is bigger: decision accuracy went from 30.3% with GPT-4 alone to 87.2% with the full agent.

Read 75.5% carefully. It means roughly a quarter of guideline citations need a human check. In a field where treatment decisions lean on guidelines, that's not a rounding error. The 20-case evaluation is small too. But the magnitude of the jump, 30.3% to 87.2%, tells you something structural: access to structured, current, multimodal knowledge was the bottleneck, not the model. Give the LLM tools and it reasons at a completely different level.

Quick Take: Across these studies, capability keeps climbing while integration keeps failing. The model is rarely the variable that decides whether clinical AI works.

The Deskilling Problem Nobody Modeled

The ACCEPT analysis is the study that should make every AI vendor uncomfortable. Four Polish endoscopy centers introduced AI polyp detection at the end of 2021, then assigned colonoscopies with or without AI by date of exam. Researchers compared the quality of unassisted examinations: 1,443 procedures, 795 performed before the AI rollout and 648 after. Adenoma detection rate on standard, non-AI colonoscopy fell from 28.4% to 22.4%. That's a 6.0 percentage point drop, with a 95% confidence interval from -10.5 to -1.6 and a p value of 0.0089. In plain terms: exposure to AI made endoscopists worse at detecting adenomas when the AI was off. The multivariable analysis put the odds ratio for AI exposure at 0.69, meaning an unassisted exam was about a third less likely to find an adenoma after the tool had been in use.

Six percentage points is roughly one additional missed adenoma per 17 exams. In a center doing 50 screening colonoscopies a week, that's three missed lesions a week. The likely mechanism is automation complacency, the same attention drift that aviation research has documented for decades. When the AI is on, the endoscopist's visual search relaxes because the system will flag anything suspicious. The skill doesn't vanish in a month, but the habit of active search erodes.

The uncomfortable implication: the human-plus-AI combination only works while the human-plus-AI skill stays intact. Any deployment that lets the human coast is quietly writing down the baseline. You don't see it until the AI fails, or gets turned off, and suddenly the detection rate is 22% instead of 28%.

Digital Twins and Explanations: The Missing Middle

The capability studies assume infrastructure that mostly doesn't exist yet. Two papers in this cluster describe that gap.

The medical digital twin policy paper proposes five components: the patient, the data connection, the patient-in-silico, the interface, and twin synchronization. The concept borrows from engineering, where a constantly updating virtual copy enables simulation and prediction. In medicine, the authors are explicit that enthusiasm is running ahead of consensus: nobody agrees yet on what a twin fundamentally is. The data connection is the hard part. A twin that updates in near-real time requires data plumbing most hospitals can't do today.

The paper's most useful argument is merging AI with mechanistic modeling. AI learns patterns from data; mechanistic models encode physiology; a twin uses both. LLMs, in this framing, become the interface and reasoning layer on top of a structured, current patient representation. That gives the LLM something to ground against, which is exactly what the bare GPT-4 in the oncology study was missing.

The XAI systematic review covers the other half: making these systems legible. It follows the PRISMA protocol across 30 studies on explainable disease prediction, and the headline is how thin the field is. SHAP and LIME dominate, most studies run on a single data modality, and dataset diversity is limited. Practical translation: the explanation tools we have explain model outputs, not clinical reasoning. A SHAP plot tells you which features pushed the prediction. It doesn't tell an oncologist whether the recommendation is safe. Put one in front of a tumor board and the question you get isn't about the plot. It's "is this right?" and "who's accountable?"

The broad healthcare AI review in this cluster makes the same point at system level: data privacy, algorithmic bias, interpretability, regulatory oversight, and human clinical oversight remain unresolved, and clinician AI literacy is the bottleneck. Every result in the first half of this article depends on those problems getting solved.

What Clinicians Are Actually Saying

The reception of these papers splits along predictable lines. The deskilling study landed hard in the gastroenterology world, mostly because it confirmed a fear that had been theoretical: the tool doesn't just change the exam, it changes the examiner. The response I keep hearing from endoscopists isn't "the AI failed." It's "we knew it would be like this, we just didn't have the numbers."

The AMIE paper gets more skepticism, and the skepticism is specific. Text chat is not how diagnosis happens in most settings, so the win feels abstract. Several physicians I've talked with pointed at the same detail: AMIE was compared against physicians working in a format they didn't train in. The comparison tested the format as much as the model.

The oncology agent gets the warmest reception, because its design is legible: the model calls tools, cites sources, and shows its work, and it still needs a human to check a quarter of the citations. That's a failure mode clinicians can work with. The digital twin and XAI papers get nods, then a question: where's the data? None of this moves, the argument goes, until the data connection problem is solved. That's the right instinct, and it's worth trusting.

Common Pitfalls

These five mistakes repeat across implementations of this technology.

1. Treating simulated evaluations as deployment evidence. AMIE's text-based format is a research artifact. If you plan an ambient documentation or intake product around that result, you're assuming the win transfers to a video visit with a patient who has three chronic conditions and poor recall. Run a local pilot before you believe it.

2. Never measuring the AI-off baseline. The ACCEPT result happened because nobody was watching the unassisted rate until the study did. If you deploy AI-assisted screening, audit unassisted performance quarterly. The number will drift. Catch it early.

3. Shipping a bare LLM for high-stakes reasoning. GPT-4 alone scored 30.3% on the oncology cases. That's a failing grade. Any deployment that matters needs tool integration, external knowledge sources, and a citation trail, or it isn't ready for a clinician to rely on.

4. Treating explanations as a compliance checkbox. A SHAP summary plot satisfies a governance slide and nothing else. The 30-study XAI review shows most explanation research tests single data types and narrow settings. Design explanations for the decision they support, not for the auditor.

5. Building the twin before the data connection. The digital twin framework puts synchronization in the critical path. Most health systems can't sync a problem list across two EHR modules in real time. Start with the data plumbing. The simulation is the easy part once the data moves.

One thing to remember: capability and integration are separate problems. AMIE, the oncology agent, the digital twin, and the XAI work all describe what a model can do under controlled conditions. The deskilling study describes what happens to the people around the model when the conditions change. Clinical AI will live or die on the second question, and the second question is the one the benchmarks can't answer.

The Bottom Line

If you're building a conversational diagnostic tool, copy AMIE's training loop (self-play, automated feedback, evaluation against validated patient actors), but don't treat its benchmark wins as production readiness. Stand up a real-world pilot with actual clinical channels first, and expect the first year to be about channel fit, not accuracy.

If you run an AI-assisted screening or imaging program, budget for quarterly unassisted audits starting the month after rollout. Six ADR points of silent drift is what you can't afford to miss, and measurement is the only control.

If you're deploying an LLM for clinical decision support, integrate specialized tools and citation on day one, because a bare model at 30.3% isn't a decision support system. Build the explanation and audit trail before you approach a regulator; certification frameworks will consolidate within 18 months, and systems that already log tool calls and source citations will clear review fastest.