Chapter 4
When an AI system contributes to a medical error, who bears responsibility? The patient who was harmed? The physician who followed the AI recommendation? The hospital that deployed the system? The developer who created it? The accountability question in healthcare AI is among the most complex and consequential in medical law, touching fundamental questions about agency, causation, and the nature of medical decision-making in an algorithmic age.
Traditional medical malpractice frameworks assume a human decision-maker whose actions can be evaluated against professional standards of care. When physicians make errors, established legal doctrines determine negligence, causation, and liability. But AI systems complicate this framework in fundamental ways: algorithms cannot be sued, cannot hold medical licenses, and cannot be held personally accountable. When AI recommendations contribute to harm, responsibility must be allocated among multiple human actors—none of whom may have complete control or understanding of the system.
This diffusion of responsibility creates what scholars call an "accountability gap"—a situation where no single party can be clearly identified as responsible for AI-related harms. Developers may argue they cannot control how systems are deployed. Hospitals may claim they trusted the vendor's validation. Physicians may assert they followed what appeared to be sound AI recommendations. Patients are left without clear paths to redress when AI-influenced care goes wrong.
Healthcare AI involves numerous actors, each with different roles, capabilities, and potential liability exposure. Understanding these roles is essential for allocating responsibility appropriately and designing governance structures that ensure accountability.
| Stakeholder | Role | Potential Liability Basis | Key Concerns |
|---|---|---|---|
| AI Developers | Design, train, and validate AI systems | Product liability, negligent design, failure to warn | Unclear whether AI is "product" or "service"; validation standards uncertain |
| Healthcare Institutions | Select, deploy, and oversee AI systems | Vicarious liability, negligent selection, duty to supervise | Institutional negligence claims for AI selection and monitoring |
| Physicians | Use AI recommendations in clinical decisions | Medical malpractice, failure to exercise judgment | Liability for over-reliance or unjustified rejection of AI |
| Data Providers | Supply training data and maintain data quality | Negligent data curation, privacy violations | Data quality issues that lead to biased or inaccurate AI |
| Regulators | Approve and oversee AI medical devices | Sovereign immunity limits; regulatory failure claims rare | Balancing innovation with safety oversight |
| Patients | Consent to AI-assisted care; report problems | Assumption of risk; contributory negligence | Meaningful consent; access to information about AI involvement |
Medical malpractice requires proving that a physician breached the applicable standard of care, causing injury to the patient. As AI becomes integrated into clinical practice, the standard of care itself evolves. Questions arise: Is using AI that improves outcomes required by the standard of care? Is failing to override a clearly erroneous AI recommendation malpractice? Is over-reliance on AI without independent clinical judgment negligent?
Product liability law holds manufacturers responsible for defective products that cause harm. Whether AI systems constitute "products" subject to these doctrines is unsettled. Software has traditionally been treated as a service in some jurisdictions, limiting product liability exposure. However, FDA-approved AI medical devices may be treated as products, potentially exposing developers to strict liability for defects.
| Liability Theory | Requirement | Application to AI | Challenges |
|---|---|---|---|
| Manufacturing Defect | Product deviates from design | AI performs differently from specifications | Software typically consistent; "defect" unclear for ML |
| Design Defect | Design is unreasonably dangerous | AI architecture has inherent flaws | Defining "reasonable" AI design standards |
| Failure to Warn | Inadequate warnings of known risks | AI documentation doesn't disclose limitations | What warnings are sufficient for complex AI? |
| Strict Liability | Liability without fault for defective products | AI causes harm regardless of developer care | Should AI developers bear strict liability? |
Beyond malpractice and product liability, general negligence principles may apply to various actors in the AI ecosystem. Developers may be negligent in training or validating systems. Hospitals may be negligent in selecting, deploying, or monitoring AI. Data providers may be negligent in data curation. Each requires proving duty, breach, causation, and damages—which becomes complex when multiple actors contribute to harm.
Proving causation—that the AI defect or misuse actually caused the harm—is particularly difficult:
The FDA regulates AI systems that meet the definition of medical devices, including Software as a Medical Device (SaMD). The regulatory pathway and classification affect liability exposure—FDA clearance or approval may influence what constitutes reasonable care and may affect product liability claims.
| FDA Pathway | Requirements | Liability Implications |
|---|---|---|
| 510(k) Clearance | Substantial equivalence to predicate device | Limited preemption of state claims; manufacturer must demonstrate equivalence |
| De Novo Classification | Novel device with appropriate controls | Establishes new classification; may set industry standard expectations |
| PMA Approval | Demonstrated safety and effectiveness | Stronger preemption of state claims; extensive evidence of safety |
| Predetermined Change Control Plan | Approved plan for AI modifications | Modifications within plan may have reduced scrutiny; clear update boundaries |
Different jurisdictions take varying approaches to AI medical device regulation, creating a complex international landscape for developers and healthcare systems operating globally.
| Jurisdiction | Regulatory Body | Approach | Liability Context |
|---|---|---|---|
| United States | FDA | SaMD framework; total product lifecycle approach | Preemption debates; product vs. service |
| European Union | EMA + Notified Bodies | MDR with AI-specific requirements; AI Act overlay | CE marking requirements; AI Act liability provisions |
| United Kingdom | MHRA | AI as a Medical Device (AIaMD) roadmap | Post-Brexit regulatory evolution |
| Japan | PMDA | STED framework for AI medical devices | Harmonization with international standards |
| China | NMPA | AI medical device classification system | Domestic liability framework developing |
Given the multiple actors involved in healthcare AI, some scholars propose shared responsibility models that allocate liability based on each party's role, knowledge, and ability to prevent harm. These models recognize that AI-related harms often result from cumulative failures across the development-deployment chain rather than any single actor's fault.
In practice, liability is often allocated through contracts between AI developers, healthcare institutions, and other parties. License agreements, deployment contracts, and service level agreements may contain indemnification clauses, liability caps, and risk allocation provisions. However, contractual limitations may not extend to patient claims and may be challenged as unconscionable or contrary to public policy.
Healthcare institutions should establish governance structures that ensure accountability for AI deployment. This includes oversight committees, clear roles and responsibilities, documentation requirements, and processes for addressing AI-related incidents.
| Governance Element | Purpose | Key Components |
|---|---|---|
| AI Oversight Committee | Strategic oversight and policy development | Clinical leadership, IT, legal, ethics, quality |
| Validation Committee | Technical review before deployment | Data scientists, clinical informaticists, domain experts |
| Incident Response Team | Address AI-related adverse events | Risk management, clinical quality, IT, legal |
| Documentation Standards | Ensure accountability trails | Decision logging, version control, audit trails |
| Training Programs | Ensure appropriate AI use | Clinician education, competency assessment |
AI developers should implement practices that demonstrate accountability for the systems they create. This includes rigorous validation, transparent documentation, post-market surveillance, and processes for addressing identified issues.
The legal landscape for AI liability is rapidly evolving. Courts are beginning to address AI-related claims, legislators are considering new frameworks, and professional organizations are developing standards that may influence liability determinations. Healthcare stakeholders must monitor these developments to understand their evolving obligations.