The US tech company Microsoft says it has taken "a real step towards medical superintelligence". Microsoft's AI team presented research results that show how artificial intelligence (AI) can sequentially investigate and solve the most complex diagnostic challenges in medicine. According to a blog post on Monday, the company's new AI system, the Microsoft AI Diagnostic Orchestrator (MAI-DxO), can predict diseases four times more accurately and diagnose them more cost-effectively than a group of experienced doctors.

Microsoft's AI division, Microsoft AI, has had its AI tool imitate the work normally carried out by a human doctor. The aim was to find out whether the tool could correctly diagnose patients with a disease. The Microsoft team used 304 case studies from the New England Journal of Medicine (NEJM) to develop a test called the Sequential Diagnosis Benchmark (SD Bench). To do this, a language model broke down each case into a step-by-step process that a doctor would follow to arrive at a diagnosis. "We evaluated a comprehensive set of generative AI models against the 304 NEJM cases. The baseline models tested included GPT, Llama, Claude, Gemini, Grok, and DeepSeek," the blog post states. Beyond basic benchmarking, the Microsoft AI Diagnostic Orchestrator (MAI-DxO) was developed, a system designed to emulate a virtual panel of physicians with different diagnostic approaches working together to solve diagnostic cases.

MAI-DxO outperformed human doctors, achieving an accuracy of more than 80 percent in some cases compared to 20 percent for doctors. "MAI-DxO performed best in combination with OpenAI's o3, correctly solving 85.5% of the NEJM benchmark cases," it says. In addition, costs were reduced by a fifth as the AI selected less expensive tests and procedures.

Doctors are usually characterized by the breadth or depth of their expertise, Microsoft writes. However, no single doctor could cover the entire complexity of the NEJM case series. With artificial intelligence, on the other hand, there is no such compromise. "It can combine both the breadth and depth of expertise and draw clinical conclusions that exceed the capabilities of a single physician in many aspects of clinical reasoning," says Microsoft.

"This orchestration mechanism – multiple agents that work together in this chain-of-debate style – that's what's going to drive us closer to medical superintelligence", Mustafa Suleyman, CEO of Microsoft AI, told the US tech magazine Wired. Suleyman was co-founder of Google DeepMind before becoming CEO of Microsoft's AI division.

AI-based diagnostic tools have the potential to reshape healthcare, Microsoft writes in its blog post. "Artificial intelligence could empower patients to self-manage routine aspects of care and provide clinicians with advanced decision support for complex cases." But even though MAI-DxO excelled at the most complex diagnostic challenges, further testing is needed to evaluate the AI tool's performance in more frequent, everyday presentations, the group said. Experts interviewed by Wired also believe that the next step in validating the potential of Microsoft's system should be to demonstrate the tool's effectiveness in a clinical trial. This would compare the results of the AI with those of real doctors treating real patients.

The company has not yet decided whether Microsoft will attempt to market its AI system, according to Wired, citing a company executive who wished to remain anonymous. According to the same source, Microsoft could also integrate the AI system into its intelligent search engine Bing to help users diagnose illnesses. The company could also develop tools to help doctors improve or even automate patient care. According to Suleyman, these systems will be tested more and more in the real world over the next few years.

(akn)