An AI that beat hundreds of physicians on written patient cases: does that predict how it performs in a real clinic?

This explores whether strong results on hand-picked medical test cases, like written patient vignettes, tell us how an AI will actually do once it's working in a real clinic. One caveat first: the library has only one clinical study that bears on this. The most useful material is general research on where benchmarks and deployment part ways. That clinical study is striking. A large language model outperformed hundreds of physicians on differential diagnosis, triage and management, both on physician-adjudicated vignettes and in an emergency-room study Can language models reason better than physicians at diagnosis?. The ER component matters because it moves beyond tidy written cases into a real setting. Most of the curated-versus-real question is about exactly that step.

Other fields suggest a curated benchmark can miss a model's weaknesses even when its score is honest. Research on agent evaluation finds that capability splits into separate axes, such as task success, privacy compliance, memory over long tasks and behavior when the situation changes. A model that ranks first on one axis often ranks lower on another Does a single benchmark score actually predict agent readiness?. Diagnostic accuracy on vignettes is one such axis. Handling incomplete histories, protecting patient privacy and following a case over days are others, and a vignette score says little about them. Cybersecurity offers a cautionary parallel. Models score well on many benchmark tasks, but the step that matters most in the real world, exploitation, is barely measured Do cybersecurity benchmarks actually measure exploitation?. It's worth asking which clinical step plays that role. Candidates include deciding what to ask next, recognizing when not to act, or noticing that the case doesn't fit.

A second problem is which cases end up in a curated set. Work on simulated users for safety testing argues that test populations should cover the full range of possible people, including rare but consequential ones, rather than mirror the average Should persona simulation prioritize coverage over statistical matching?. Curated clinical cases tend to fall short both ways. They are cleaner than real presentations and overrepresent textbook-interesting conditions. They may also miss the unusual patients where errors cost the most.

The most surprising finding for clinical use is how stronger models fail. In long document workflows, weaker models degrade text in visible ways, such as deleting content. Frontier models instead corrupt it quietly, and the output still looks intact Does model capability change how documents degrade?. That study wasn't about medicine, but the risk carries over to settings like charting, discharge summaries and long patient records. As models improve, their errors may get harder for a busy clinician to catch, and a one-shot vignette benchmark would never reveal that.

Making evaluations more realistic doesn't solve everything. Interactive, multi-step evaluations move the old problems of comparability and reproducibility into a more complicated space rather than removing them Do interactive evaluations actually solve the benchmark comparison problem?. One promising direction is to judge a model by evidence of how it reached an answer, not just whether the answer was right Can infrastructure evidence replace terminal scores in benchmark validation?. For medicine, that means a strong vignette score is a real signal, but a narrow one. The questions that matter for deployment are how the model reached its answer, how it handles messy and rare patients, and whether its mistakes are visible.