Good Machine Learning Practice for medical devices

Developments in Artificial Intelligence (AI) and machine learning (ML) carry a wealth of opportunity as economic drivers with wide social benefits and global transformative potential. The UK, US and Canada have put together guiding principles for machine learning in medical device development to help shape a thriving sector. 

Transformative potential

The transformative potential of AI and ML in healthcare and medical device development is substantial. The healthcare industry generates a large volume of real-life data every day that can be harnessed by AI and ML for the development and improvement of life-saving products. To date we have seen AI and ML provide:

• imaging systems to diagnose skin cancer
• smart sensors that can estimate the probability of a heart attack
• systems that can predict a protein’s 3D structure with enormous potential in drug design.

These few examples give a snapshot of the crucial role AI and ML already plays in the life sciences sector.

Ten guiding principles

October 2021 saw ‘Good Machine Learning Practice for Medical Device Development: Guiding Principles’ jointly published by the U.S. Food and Drug Administration (FDA), Health Canada, and the United Kingdom’s Medicines and Healthcare products Regulatory Agency (MHRA). These regulators have together identified ten guiding principles to promote safe and effective development with the aim of building confidence and trust from investors and society. The principles that developers should now be considering in medical device development are outlined below.

Developers should aim to:

1. use multi-disciplinary expertise throughout the product life cycle to focus on in-depth understanding of clinical workflow and model integration
2. implement good software engineering and security practices in areas including risk management, design process, and cybersecurity
3. ensure that clinical study participants and data sets are representative of the characteristics of the intended patient population
4. ensure that training datasets are maintained and kept independent of test sets
5. base select reference datasets on best available methods that are clinically relevant and well-characterised
6. model design to available data and reflect the intended use of the device to ensure it supports the active mitigation of known risks
7. address and focus on the performance of the Human/AI team by looking at how the output is interpreted
8. use testing to demonstrate device performance by using clinically relevant conditions and test plans
9. provide users such as healthcare providers and patients with clear and essential information and
10. deploy models that are monitored for performance and ensure re-training risks are managed with a focus on monitoring, maintaining, and improving the model in the real world.

Striking the right balance

As with any evolving technology, the questions in development of medical devices deploying AI and ML are complex. Striking the right risk-benefit balance will present ongoing challenges.

Over the next six to twelve months the UK Government plans to publish the pro-innovation national position on governing and regulating AI, as part of its National AI Strategy. This includes piloting an AI standards hub. However, for now, these principles should assist stakeholders to advance responsible innovations, and assist with investor and consumer confidence.