BGU and Clalit Study: AI Mostly Knows When to Skip Unnecessary Tests
A Ben-Gurion University and Clalit Health Services study finds leading AI chatbots usually avoid unneeded medical tests, but struggle where doctors themselves disagree.
Researchers at Ben-Gurion University of the Negev and Clalit Health Services set out to answer a question that matters to anyone who has ever asked a chatbot about a symptom: does artificial intelligence know when a test or treatment is not needed? The peer-reviewed study, led by Dr. Guy Laban of BGU's Department of Industrial Engineering and Management and Dr. Dean Ariel of Clalit Health Services, with Dr. Stav Endelman of Rabin Medical Center-Beilinson Hospital, was published in Annals of Internal Medicine, the flagship journal of the American College of Physicians.
The team tested six leading large language models from Google, Anthropic, and OpenAI against hundreds of clinical scenarios built on the Choosing Wisely campaign, an international effort started in 2012 to cut low-value medical care. Each scenario paired a case where a test was unnecessary, such as imaging for a young man with simple back strain, with a closely matched case where the same test was genuinely needed, such as a cancer patient with new nocturnal back pain.
Across thousands of responses, the models missed truly necessary care in only 0.6 percent of cases and recommended unneeded tests or treatment in just 5.4 percent of cases, with no errors at all in eight of the fifteen recommendations studied. Most mistakes clustered around two genuinely disputed areas of medicine: unnecessary EKGs for patients mentioning heart palpitations and unnecessary ultrasounds for small, incidentally found thyroid nodules, where medical specialty groups themselves disagree on the right approach.
"One of the hardest parts of practicing medicine is knowing when to hold back," said Ariel, describing why the researchers wanted to test whether AI systems had absorbed that restraint. Laban noted that different models gave very different advice on the same ambiguous cases, and that the next question is whether AI tools will hold firm when an anxious patient pushes back for an unneeded test.
The findings suggest that AI tools used by doctors and patients can generally be trusted to avoid pushing for unnecessary, costly, and sometimes harmful tests, while flagging the specific gray areas, like thyroid nodules and heart palpitations, where even human experts disagree. This kind of evidence can help health systems worldwide use AI more safely as a support for clinical decisions rather than a replacement for them.
Sources
- Israeli study finds AI chatbots can identify unnecessary medical tests | The Jerusalem PostThe Jerusalem Post | JPost.com, October 21, 2026
- AI Models Usually Avoid Unnecessary Medical Testsbgu.ac.il, October 21, 2026
Summarised in our own words from the reporting above.
Tikkun olam, today. This is part of tikkun olam, the Jewish idea of repairing the world, and every one of us can add to it.