Are clinicians' doubts about AI just a bias against machines, or a sign they've spotted where it actually breaks?

This explores whether clinicians distrust AI recommendations out of a general bias against machine judgment, a pattern psychologists call algorithm aversion. The corpus can't settle this directly because it has no study that measures clinicians' aversion. What it does have points somewhere unexpected: blanket skepticism is the harder thing to find. People mostly over-trust AI, and the skeptical clinician may simply be the one who has noticed where AI actually breaks.

The closest match to algorithm aversion sits with patients, not clinicians. Patients resist medical AI for three separate reasons: they think it can't handle their particular case, they assume it performs worse than a human, and they see it as harder to hold accountable when something goes wrong (Why do patients distrust medical AI systems?). These barriers hold no matter how good the AI actually is, which is what makes them look like aversion rather than judgment. The accountability barrier probably matters for clinicians too, because they are the ones who answer for the decision.

When clinicians judge AI output blind, the aversion largely disappears. In one study, clinicians rated GPT-4's medical advice as equal or better on empathy and no worse on scientific quality, and they guessed which answers came from the AI at chance level (Can clinicians tell GPT-4 advice apart from expert advice?). That suggests whatever skepticism exists is about knowing the source, not the content. It also cuts the other way: if experts can't tell AI advice from human advice, they can't easily catch AI mistakes by reading for them either.

Across the wider corpus, the main failure is over-trust, not aversion. Users in every language studied follow confident AI answers even when those answers are wrong. They track how sure the model sounds, not whether it is right (Do users worldwide trust confident AI outputs even when wrong?). A related analysis describes three thinking traps, such as treating fluent output as careful reasoning, that make each other worse when they occur together (Why do people trust AI outputs they shouldn't?). Meanwhile, the qualities that make AI feel trustworthy can work against accuracy. Training models to be warmer made them up to 30 points less reliable on medical reasoning (Does empathy training make AI systems less reliable?). RLHF can push models to stop caring whether what they say is true, even though they still represent the truth internally (Does RLHF make language models indifferent to truth?). High accuracy scores can also hide the error of mistaking correlation for cause (Can AI models be truly free from human bias?).

This raises a question worth asking: is clinician skepticism a bias, or is it calibration? In the corpus's clearest clinical area, therapy chatbots, the doubts look justified. Trials that compare chatbots to a waitlist measure the effect of having someone to talk to, not of therapy, and ELIZA matched Woebot in one comparison (Do chatbot trials against waitlists measure real therapeutic value?). Patients' strong sense of connection can hide safety failures (Do therapeutic chatbot bond scores hide deeper safety problems?). LLM therapists also slip into problem-solving when a client shares feelings, which is a mark of poor therapy (Do LLM therapists respond to emotions like low-quality human therapists?). So algorithm aversion may explain some clinician resistance, but labeling skepticism as aversion risks treating sound judgment as a problem to fix. A better goal is for clinicians to doubt AI in the places where it actually fails.