Understanding, explaining, and utilizing medical artificial intelligence

Medical artificial intelligence is cost-effective and scalable and often outperforms human providers, yet people are reluctant to use it. We show that resistance to the utilization of medical artificial intelligence is driven by both the subjective difficulty of understanding algorithms (the perception that they are a ‘black box’) and by an illusory subjective understanding of human medical decision-making. In five pre-registered experiments (1–3B: N = 2,699), we find that people exhibit an illusory understanding of human medical decision-making (study 1). This leads people to believe they better understand decisions made by human than algorithmic healthcare providers (studies 2A,B), which makes them more reluctant to utilize algorithmic than human providers (studies 3A,B). Fortunately, brief interventions that increase subjective understanding of algorithmic decision processes increase willingness to utilize algorithmic healthcare providers (studies 3A,B). A sixth study on Google Ads for an algorithmic skin cancer detection app finds that the effectiveness of such interventions generalizes to field settings (study 4: N = 14,013).

Introduction. lgorithms are rapidly diffusing through healthcare systems1, providing support for outpatient services (for example, telehealth) and supply to match demand for inpatient care services2–4. Algorithmic-based healthcare services (medical ‘artificial intelligence’ (AI)) are cost-effective and scalable and provide expert-level accuracy in applications ranging from the detection of skin cancer5 and emergency department triage6,7 to diagnoses of COVID-19 from chest X-rays8. Adoption of medical AI is critical for providing affordable, high-quality healthcare in both the developed and developing world9. However, large-scale adoption of AI hinges not only on adoption by healthcare systems and providers but also on patient utilization, and patients are reluctant to utilize medical AI10–12. Patients view medical AI as unable to meet their unique needs10 and as performing more poorly than comparable human providers12 and feel it is harder to hold AI providers accountable for mistakes than comparable human providers11.

Discussion / Conclusion. Utilization of algorithmic-based healthcare services is becoming critical with the rise of telehealth services31, the current surge in healthcare demand2–4 and long-term goals of providing affordable and high-quality healthcare32 in developed and developing nations9. Our results yield practical insights for reducing reluctance to utilize medical AI. Because the technologies used in algorithmic-based medical applications are complex, providers tend to present AI provider decisions as a ‘black box’. Our results underscore the importance of recent policy recommendations to open this black box to patients and users16,33. A simple one-page visual or sentence that explains the criteria or process used to make medical decisions increased acceptance of an algorithm-based skin cancer diagnostic tool, which could be easily adapted to other domains and procedures.