WHO and ITU establish benchmarking process for artificial intelligence in health

6 July, 2019

Below are the citation and selected extracts of a Comment in this week's print issue of The Lancet. Full text here: https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(19)30762-7/fulltext

CITATION: Comment| volume 394, issue 10192, p9-11, july 06, 2019

WHO and ITU establish benchmarking process for artificial intelligence in health

Thomas Wiegand et al.

The Lancet, Published:March 29, 2019DOI:https://doi.org/10.1016/S0140-6736(19)30762-7

Growing populations, demographic changes, and a shortage of health practitioners have placed pressures on the health-care sector. In parallel, increasing amounts of digital health data and information have become available. Artificial intelligence (AI) models that learn from these large datasets are in development and have the potential to assist with pattern recognition and classification problems in medicine—for example, early detection, diagnosis, and medical decision making. These advances promise to improve health care for patients and provide much-needed support for medical practitioners...

Two UN agencies, WHO and the International Telecommunication Union (ITU), established a Focus Group on Artificial Intelligence for Health (FG-AI4H) in July, 2018. FG-AI4H is developing a benchmarking process for health AI models that can act as an international, independent, standard evaluation framework.

To establish this evaluation and benchmarking process, FG-AI4H is calling for participation from medical, public health, AI, data analytics, and policy experts. Topic groups are being formed by communities of stakeholders allowing FG-AI4H to develop its processes for AI evaluation and benchmarking specific for each health topic. Each topic use case will be reviewed for its relevance and should impact a large and diverse part of the global population or solve a health problem that is difficult or expensive. The AI models are expected to offer improvements over current practices in quality or efficiency that would be expected to lead to better health outcomes or cost-effectiveness. Once formed, topic groups will provide a forum for open collaboration among stakeholders who agree on a pragmatic, best-practice approach for benchmarking each use case, including defining the application scenario and desired output of AI models in that use case, identifying adequate sources of training and testing data, and facilitating the preparation of multisource heterogeneous data. All data for training and testing are expected to be of high quality, ethically generated, and accompanied by detailed information about their format and properties. Thus far, FG-AI4H has developed 11 topic groups in areas such as cardiovascular disease risk prediction, ophthalmology (retinal imaging diagnostics), and AI-based symptom checkers, but this approach is expected to be expanded to other tasks...


Best wishes, Neil

Let's build a future where people are no longer dying for lack of healthcare information - Join HIFA: www.hifa.org

HIFA profile: Neil Pakenham-Walsh is coordinator of the HIFA global health campaign (Healthcare Information For All - www.hifa.org ), a global community with more than 19,000 members in 177 countries, interacting on six global forums in four languages. Twitter: @hifa_org FB: facebook.com/HIFAdotORG neil@hifa.org