Regulating AI within the Medical Diagnosis of Clinical Professionals
Today, I want to discuss a topic that greatly intrigues me. This summer, I have started to work at a GP surgery, which has provided ample opportunity to reflect on the intersection of legality and healthcare, particularly with the integration of AI into the NHS. This article will specifically explore the use of AI in medical diagnosis. Significant change is being implemented within society and this raises a lot of questions in my mind. As a society, are we valuing efficiency over empathy? What about the older generation who do not use technology, will they be neglected by the services they helped build? The future , as of this point, remains hopeful as reformation within our processes could offer accessible healthcare for everyone or transform our NHS into an automated system where beneficence takes the backseat for the pursuit of improved statistics.
The courts have established rules for determining when a human doctor's clinical judgment is negligent. Yet, when that judgment is increasingly influenced by an AI system, the difficult question becomes whether the law should continue to place responsibility primarily on the doctor. Sources like the New Law Journal have raised their concerns that legal anomalies could leave clinical professionals unfairly exposed to claims.1
Before diving deeper into this topic, I want to discuss how AI has started being used within the healthcare sector, specifically within the NHS.
What is AI?
As described in NHS guidelines, 'ArtificiaI Intelligence (AI) is the use of digital technology to create systems capable of performing tasks commonly thought to require human intelligence.'2
How are the systems made?
Some AI systems are made by collecting a large amount of healthcare data, for example, patient records, X-rays and MRIs, symptoms for triage and information about previous treatments and outcomes. This data is then labelled by experts to identify what the data shows, eg, an X- ray showing abnormalities and X-rays which are normal. This gives AI examples which it can learn from. This will then be analysed through a machine learning algorithm to look for patterns which eventually start to learn statistical relationships within the data, for example going back to the X- ray example, certain shapes associated with abnormalities. Once trained, it is tested using new data and is checked by professionals to assess how well it works and to clinically validate the system.
What about data protection rules, how is this legal?
Health data constitutes a special category of personal data under Article 9(1) of the UK GDPR, that prohibits its processing, unless an exception under Article 9(2) applies.3 Article 9(2) (h) could show us that this processing of data is valid as it is necessary for medical diagnosis and the treatment within the health and social care systems.4
How is AI used in NHS practice?
AI is being implemented in various different ways to attempt to release some of the burden that is placed upon our NHS. Greater efficiency causes faster diagnosis, which may in turn cause faster treatment can be implemented before disease or illness gets worse.
What are the benefits of using AI in medical diagnosis?
Medical imaging - Analysing, clarifying and flagging abnormalities on X-rays, MRIs which speeds up diagnosis.
Potentially reduces human error- AI can process large datasets consistently and may assist clinicians in identifying patterns that could otherwise be overlooked.
Triage - What has been implemented into the GP surgery I work at, analysing symptoms and determining what healthcare service a patient should use and flagging what severity the symptoms are to prioritise appointments. This also reduces the workload for a 'triaging doctor' which theoretically means more patients can be seen, breaking barriers to accessing healthcare.
AI being multilingual - AI-powered translation can help patients communicate with medical professionals who do not share the same first language, potentially allowing them to explain their symptoms more clearly.
Administrative work and clinical notes - Drafting patient summaries, letters and other routine documentation, meaning clinicians can have more patient contact.
Risk prediction- Analysing patient data to identify certain demographic characteristics and family history to estimate the risk of developing certain health issues, which aids prevention plans.
Cancer and stroke detection - Identifying suspicious abnormalities on scans to help prioritise patients accordingly.
It is important to note AI is meant to be used to assist a clinical decision-making not become a replacement for professional judgement.5
Now we understand what AI is and how it works let's discuss what the concerns of using AI in medical diagnosis are.
What are the concerns of using AI in medical diagnosis?
Black box theory - This is the issue where AI generates a conclusion but cannot explain how that conclusion was reached ( similarly to how with a black box you can't see the internal workings, just the input and the output.) A famous example of this was the Post Office Horizon IT scandal, where sub-postmasters (those who worked in the post offices) were falsely accused of stealing money.The Horizon litigation established serious problems with the system, including bugs/errors/defects capable of causing discrepancies. .6 This caused extensive litigation in Bates & Others v Post Office Ltd [2019], where the technology was proven faulty and any qualifying convictions were quashed through the government passing the Post Office (Horizon System) Offences Act 2024.7 Although this system was not AI, it raises a huge concern regarding transparency within technology, and who should be blamed if things go wrong. If it is hard to disprove, does this leave clinical staff more susceptible to sanctions, or would the sanctions be on those who create the system, or the NHS organisation who deployed it? The law is unclear on this as of yet, but if we apply existing clinical negligence law like in Bolam and Bolitho, the clinician would be judged for applying a certain standard of professional care under their professional opinion and that this professional opinion must be able to withstand logical analysis. AI's opacity could make it more difficult for clinicians to demonstrate that reliance on its output was professionally defensible..8
Bias within the algorithm - The algorithms work by synthesising lots of data at once, which means small errors can add up. If this data is made based on a certain gender or ethnicity, this could leave those of minority backgrounds underrepresented within the data, meaning AI could miss certain symptoms which would lead to conditions being missed, which undermines the so called 'consistency' AI is meant to bring. Those who are older may not have the same access to technology as others in society do, leaving them at an unfair disadvantage. This could also could potentially engage the Equality Act 2010's prohibition of discrimination on the basis of protected characteristics such as race and age.9 Anomalies in data could also lead to hallucinations and false readings, which could be detrimental for patient care.
Skill degradation and over reliance - If clinicians start to develop automation bias - meaning the clinician starts to accept AI as the more reliable source, without challenging how it came to a conclusion, which may happen under lots of pressure or an intense workload. Skill degradation is a longer term concern, which clinicians may experience over time if they become overly dependent on AI-generated recommendations. This would turn a clinician from the decision maker to the verifier, which would impact patient care, turning a profession of non-maleficence and beneficence into a robotic automated system. If that system were to fail in 100 years on from now would clinicians be sufficiently trained to give adequate care without the input of technology?
Impact on the climate and society- The NHS has pledged to net zero for emissions it directly controls by 2040 but AI systems rely on data centres that consume substantial amounts of electricity and, in some cooling systems, water, contributing to environmental impacts.10 This means they need to balance the public interest as well as technological innovation. Over time patients may start to believe AI is more correct than clinical staff, causing a loss of trust in human interaction, and reduce the demand for human involvement in healthcare, thus potentially decreasing jobs in society. This could have a big impact on society as the idea of the 'modern day professional' could cease to exist.
My conclusion-
After researching and hearing different types of people's opinions on this topic I find myself torn between the idea of not needing to fix what isn't necessarily broken and the excitement of innovation. Then I think back to the Victorian era, around the time that the first petrol powered car was made. Some people would have been confused by the idea that a machine could take people to places rather than horses or walking. Yet, it is now a staple in our society. Could you imagine how distraught the first caveman was when he found fire. Innovation drives our society towards better futures; this has been proven countless times with inventions throughout history.
I think AI can be extremely helpful, especially when helping more people access healthcare and achieving faster diagnosis. In a multi cultural society like the UK, people being able to talk on the phone in their native language gives those who may not understand or speak English well a newfound confidence to talk about their symptoms clearly. I definitely think utilising the clinician's time, as they spend less time triaging, is very helpful as more time is spent on planning the patients' treatment. This could alleviate the strain on the NHS as it would become a more efficient system.
Although I understand all the negatives of AI, they can all be avoided if proper governance is in place. I believe AI use can be helpful when used responsibly by the clinician and there is transparency within how the systems work, so if there are hallucinations they can be fixed. Ultimately fragmenting responsibility, partly to the clinician, the AI system manufacturer and the NHS body seems like the fairest conclusion as they all have a part to play but it would depend on what the issue was. For example responsibility would depend on the source of the harm and the applicable legal framework.
If people like the elderly and those who can't access technology properly are shown and given time by digital champions within the NHS maybe through leaflets, videos and tutorials, on how to use AI, this could make the algorithm stronger as it would be more reflective of the population. As a last resort, healthcare should retain accessible non-digital routes so that patients are not disadvantaged by the adoption of AI. AI should be limited to certain things and thoroughly checked as society needs jobs to run properly. We also need to prioritise the care of our planet as there is no point of having better systems if our future on the planet is cut short due to selfish choices made by companies. Global warming is one of the biggest threats to Planet Earth and humanity, and we need to ensure we work hard to mitigate climate change and reduce greenhouse gas emissions, not catalyse it.
There needs to be a healthy balance between demands of society and protecting our planet, perhaps researching into more sustainable ways of using AI. Traditional methods should not be discarded, like how a computer can crash, AI systems can fail, become unavailable or produce erroneous outputs and we shouldn't be unprepared for this in the future. We do not need AI verifiers: we need Doctors, nurses HCAs, paramedics who know their profession, but use AI for efficiency without reliance. They should always be the safety net to turn to.
Overall, I believe if regulated properly the future of AI stays exciting and we remain in the relatively early stages of widespread AI adoption in healthcare. Teething issues are bound to happen, but we should be made aware of the implications of AI so companies and individuals have clear procedures on what to do in case it goes wrong.
As always I would love to read your thoughts on this topic, please contact me through the form, and I look forward to reading your responses!
Zoya Khalid
Beyond The Verdict.
References made in this article:
1 The New Law Journal 2026 Volume 176, Issue 8165, JuneArticlesAI clinical negligence risk – 176 NLJ 8165, p5 (1).
2 NHS England Digital, ‘Artificial intelligence’ (11 May 2026) https://digital.nhs.uk/data-and-information/information-governance/guidance/artificial-intelligence accessed 25 August 2026.
3 UK GDPR, Article 9(1); Article 9(2).
4 UK GDPR, Article 9(2)(h).
5 NHS England, ‘Artificial intelligence (AI) and machine learning’ (14 June 2023, last updated 7 April 2025) https://www.england.nhs.uk/long-read/artificial-intelligence-ai-and-machine-learning accessed 4 September 2026.
6 Bates and others v Post Office Ltd (No 6: Horizon Issues) [2019] EWHC 3408 (QB).
7 Post Office (Horizon System) Offences Act 2024, s 1.
8 Bolam v Friern Hospital Management Committee [1957] 1 WLR 582; Bolitho v City and Hackney Health Authority [1998] AC 232.
9 Equality Act 2010, s 4.
10 NHS England, ‘Delivering a “Net Zero” National Health Service’ (1 October 2020, updated 24 September 2025) https://www.england.nhs.uk/greenernhs/a-net-zero-nhs/ accessed 4 September 2026
Comments