What Patients Actually Want From Clinical AI: Explainability and a Doctor in Charge

Two numbers from the same survey sit oddly together. Asked whether they hold a generally positive view of AI in medicine, 57.6 percent of hospital patients said yes. Asked whether they would prefer to be treated at a facility that uses AI software, 71.4 percent said yes. People want the hospital down the road to have the technology more than they want to feel warm about the idea of it — a quietly revealing gap, and one the authors single out themselves.

The figures come from a cross-sectional survey run by the COMFORT consortium, led from the Technical University of Munich and Charité Berlin and published in JAMA Network Open. Between February and November 2023 it gathered 13,806 responses from patients at 74 hospitals across 43 countries, in 26 languages including German, with roughly two-thirds from the Global North and a third from the Global South. As a portrait of what patients say about clinical AI, nothing else comes close to its reach.

The two conditions

Strip the survey to its load-bearing findings and you are left with two preferences, not an approval rating. The first is for legibility: 70.2 percent of respondents wanted an AI whose reasoning they could follow, explicitly even if that meant accepting slightly lower accuracy than an opaque model. The second is for a human in the chair. When asked to choose between a physician and an AI of equal accuracy, 72.9 percent preferred a collaborative arrangement in which the physician makes the final call. Only 4.4 percent wanted fully autonomous AI in diagnosis; only 6.6 percent wanted physicians to work without it at all. Most patients, in other words, want the tool in the room and the doctor still deciding.

Neither preference was the property of a sceptical minority. Both held across age, education and self-rated AI knowledge, and the pull toward explainability in particular was independent of demographics. What patients are describing is not a verdict on whether AI is good. It is a specification: reasoning they can interrogate, and accountability that stays with a person.

Where the warmth runs out

The trust items expose where acceptance thins. Patients were comfortable with AI as a second pair of eyes — 67.9 percent trusted it as a second opinion for physicians — but markedly less so as a forecaster of their own future. Trust that AI could accurately tell them how they would respond to a treatment fell to 41.8 percent, the lowest figure of any trust item in the instrument. The nearer the technology moves to a prediction about one's own body, the cooler the response. That gradient, more than any single percentage, is what a careful reader takes away.

What the design will and will not bear

This is a survey, and the word governs the weight. It records what patients say, not what they would do — and stated attitudes are a thin predictor of behaviour at the bedside, where illness, fear and a trusted clinician's recommendation all intervene. It was a non-probability convenience sample: paper questionnaires handed out and posted in hospitals, completed by whoever was willing and able. The authors are candid that this likely produced low response rates and selection bias, and that the results should not be read as representative of any general population. They are also candid that recruitment ran largely through radiology departments, which tend to see more ambulatory, stable patients already primed to think about diagnostic technology. The percentages describe these respondents. Read that way they are genuinely informative; read as a referendum they would mislead.

Two further limits matter. Site-to-site heterogeneity was large enough (intraclass correlation 0.22) that the authors declined to make country-level comparisons — so this is an international average that need not fit any one system precisely. And one subgroup result cuts against a comfortable assumption: the sicker the respondent, the cooler the attitude. Among those rating their health as very poor, 29.2 percent held rather negative views of medical AI, against 5.3 percent of those in very good health. The people most likely to actually meet these systems are the least sold on them — not a reason to slow deployment, but a reason not to design it around the healthy and technologically fluent, who are the easiest to survey and the least representative of the ward.

Why it matters here

For European systems deciding where clinical AI belongs, the survey redirects the question. The usual contest is over raw performance; patients are naming two design requirements that performance alone does not satisfy. Explainability, to them, is not a regulatory nicety but a precondition of consent. Physician oversight is not a transitional phase on the road to autonomy but the arrangement the large majority actively prefer. A tool that is more accurate yet opaque, or one that writes the clinician out of the decision, is optimising for something patients did not ask for. The honest reading is modest and usable: acceptance is conditional, the conditions are nameable, and the patients most affected are the most cautious.

Source: Busch F, Hoffmann L, Xu L, et al. Multinational Attitudes Toward AI in Health Care and Diagnostics Among Hospital Patients. JAMA Network Open 2025;8(6):e2514452. A cross-sectional, non-probability survey of hospital patients — large and unusually international, but a sample of the willing rather than a representative population, recruited largely through radiology departments and measuring stated attitudes rather than behaviour. According to PubMed.