AI policy too often drifts into a false dichotomy: that we can either regulate AI or innovate with it. More often, the opposite holds. Without trustworthy institutions for regulating, evaluating, and deploying AI, much of the innovation we hope for cannot happen.
The report from the National Commission into the Regulation of AI in Healthcare published today moves beyond that type of old-fashioned framing of what policy is for. Established by the Medicines and Healthcare products Regulatory Agency (MHRA), the National Commission makes clear how a good regulatory framework creates the conditions for innovations that people want and that they can trust. Its findings also point to the next phase of policy development: how to create the institutional capabilities to adopt, adapt, monitor, and learn from the use of AI in healthcare.
Public confidence in AI for health will not come from PR campaigns telling people that the technology is safe. It can be fostered if people encounter systems that make their care better, that are demonstrable safe and effective, with effective safeguards that they understand, with clear routes for redress if things go wrong, and with mechanisms to hold those developing and using AI accountable.
As AI becomes embedded in organisations, safety becomes an organisational problem. Asking “is this model safe?” is necessary but is not enough. Heathcare institutions also need to ask: Is this system being used safely, here, by these people, for this purpose? This is why the Commission is emphasising lifecycle and system-wide responsibly for AI.
In our public dialogues with ai@cam, we consistently hear that people want AI to support healthcare workers, with humans able to devote time to improving care for patients while remaining in control of decision-making. But from a technical perspective, we also see repeatedly that AI systems can undermine human judgement. Without care “human in the loop” becomes a superficial form a governance: a person remains nominally responsible for a decision that has, in practice, already been shaped or settled by the system. This becomes more important as increasingly capable agentic AI systems can take actions on our behalf. If we allow such systems to displace the points where we would normally exercise human judgement, we accumulate a form of agentic debt. Paying down this debt requires a new focus on the human judgement layer in decision making; for example, protecting it through policies, evidence requirements, and boundaries around what actions AI can take.
Accountability cannot be automated. AI may process information or execute tasks, but someone needs to retain meaningful authority. An “AI-enabled NHS” should not mean machines making more decisions; it should mean an NHS where clinicians are better able to make good decisions.
That is why implementation will matter so much in the next phase of AI for health. Regulation can establish requirements for - for example - transparency or responsibility. But the practices that develop on wards, in diagnostic pathways, or clinical workflows will influence how AI is integrated. These require a mix of local knowledge, technical capability, professional judgement, and careful governance.
The aim is to build healthcare institutions that can deliver effective care by using AI systems well. To deliver this, we need to create the conditions in which clinicians, patiences, healthcare organisations, and regulators can learn in collaboration, with regulation and innovation working together to ensure AI delivers public value.
To find out more about the National Commission into the Regulation of AI in Healthcare and read its report, visit the MHRA pages on GOV.UK