Mass General Brigham AI is hosting the Healthcare AI Challenge, a multi-institutional virtual, interactive series of events where healthcare professionals can explore and assess the latest artificial intelligence (AI) healthcare technologies in real-world healthcare scenarios. The rankings generated by the challenge can serve to provide industry, healthcare stakeholders, and the public with a transparent analysis of AI solutions’ performance across a range of healthcare data and clinical scenarios.
The Healthcare AI Challenge Collaborative is launching with Mass General Brigham, Emory Healthcare, the Department of Radiology at the University of Wisconsin School of Medicine and Public Health, the Department of Radiology at the University of Washington School of Medicine, and the American College of Radiology Radiology was chosen as the first in a series of healthcare AI events given the historic use of AI solutions to make rapid and meaningful impact in the field against the wave of generative AI model proliferation. Additional member institutions will be announced and onboarded in subsequent phases.
“The velocity of AI innovations and breadth of their healthcare applications continues to increase. This unprecedented growth leaves clinicians struggling to determine the effectiveness of these innovations in safely delivering value to healthcare providers and our patients,” said Keith Dreyer, DO, PhD, chief data science officer at Mass General Brigham, and leader of Mass General Brigham AI, the healthcare system’s AI business. “The Healthcare AI Challenge is a collective response to the complexities involved in advancing the responsible development and use of AI in healthcare. This new approach strives to put clinicians in the driver’s seat, allowing them to evaluate the utility of different AI technologies and, ultimately, determine which solutions have the greatest promise to advance patient care.”
Participating healthcare professionals will have access to late-breaking AI solutions they can assess for effectiveness on specific medical tasks in a simulated environment. Participants with relevant healthcare credentials can then provide their feedback on the solutions’ performance and utility, which will generate publicly available insights and analytics.
By crowdsourcing input from healthcare professionals, the Healthcare AI Challenge seeks to create continuous, consistent, and reliable expert evaluations of AI solutions in medicine. Scaling the evaluation of the technologies and sharing the insights can result in societal benefit for healthcare stakeholders and patients globally.
“We need to go beyond collaboratives that come to consensus on how to think about AI,” said Alistair Erskine, MD, chief information and digital officer at Emory Healthcare and Emory University. “We need healthcare delivery communities to provide real-world experience of the application of AI at the point of care. That is what the Healthcare AI Challenge is designed to do.”
The rankings generated by the challenge can serve to provide industry, healthcare stakeholders, and the public with a transparent analysis of AI solutions’ performance across a wide range of healthcare data and clinical scenarios. User activity and expert feedback are expected to provide valuable insights to help AI developers better understand how healthcare professionals consider AI healthcare solutions in the context of providing clinical value. In turn, developers can enhance these technologies so that they are fit-for-purpose, commercially viable, and clinically relevant, which may foster collaborations and knowledge-sharing to help ensure responsible adoption of AI in healthcare settings.
The results of the Healthcare AI Challenge can be followed at HealthcareAIChallenge.org. For details about participating, email collaborations@aihealthcarearena.ai.
