Annex III: Which Clinical AI Is High-Risk?
EU AI Act Annex III, Section 5 classifies the following categories of AI systems as high-risk in the healthcare context:
- AI intended to be used as a safety component of a medical device or as a medical device itself, within the meaning of EU Regulation 2017/745 (MDR) and EU Regulation 2017/746 (IVDR).
- AI used for evaluation of the eligibility for treatment, care, or services — including clinical pathway recommendations, surgical planning, and resource allocation.
The practical scope of this classification is extremely broad. Any AI system that outputs a clinical recommendation — a diagnosis suggestion, a risk score, a treatment option, a drug dosing recommendation — and that clinicians are expected to act upon, is almost certainly covered. The key threshold is intended use: if the system is designed to inform clinical decisions, it is high-risk regardless of whether it ultimately requires clinician confirmation.
Ask: Does this AI system output a recommendation, risk score, diagnosis, or treatment suggestion that a clinician would reasonably rely on in making a clinical decision? If yes, it is almost certainly Annex III high-risk. The test applies whether the system is deployed at the point of care (bedside), in population health analytics, in administrative prior authorization, or in remote monitoring. "Clinical decision support" as a category is not limited to formal CDSS products — it includes any AI that processes patient data to generate clinically actionable outputs.
MDR/IVDR Overlap and the Gap Analysis Requirement
EU AI Act Article 8(1) provides limited double-regulation relief: where an AI system is already subject to MDR or IVDR, conformity with certain MDR/IVDR requirements satisfies corresponding AI Act requirements. However, this overlap is narrower than many vendors expect.
Article 9: Risk Management for Clinical AI
Article 9 requires a risk management system that is a continuous, iterative process throughout the entire AI lifecycle — not a one-time pre-market activity. For clinical AI, this obligation is substantially more demanding than the ISO 14971 risk management familiar from MDR compliance.
AI-Specific Risk Categories
Clinical AI risk management under Article 9 must address risk categories not covered by traditional device risk analysis:
Post-Market Risk Management
Article 9 requires that the risk management system be updated based on real-world performance data. For clinical AI, this means the vendor must collect post-deployment outcome data, analyze performance against pre-specified metrics, and update the risk management file when adverse events or unexpected performance gaps are identified. This creates a feedback loop between post-market surveillance (MDR Annex III) and AI risk management (Article 9) that must be explicitly managed.
Article 13: Transparency for Clinical AI
Article 13 requires that high-risk AI systems are designed and developed to ensure transparency — sufficient for deployers (healthcare organizations) and users (clinicians) to understand the AI's purpose, capabilities, and limitations.
For clinical AI, Article 13's transparency requirements translate into specific technical and documentation obligations:
- Instructions for use with AI-specific content: The clinical AI system's IFU must include: the intended purpose and the clinical decisions it is designed to inform; the expected accuracy and its variation across patient subpopulations; known clinical conditions or patient characteristics where performance is uncertain or degraded; the level of oversight required from clinical users; and procedures for detecting and reporting unexpected AI outputs.
- Confidence and uncertainty disclosure: The AI system must communicate its confidence level or output uncertainty to users in a manner that supports clinical judgment. Binary outputs (positive/negative) without confidence information may not satisfy this requirement for diagnostic AI.
- Performance metric disclosure: Article 13 requires that providers make performance metrics available in a standardized format accessible to deployers — sensitivity, specificity, AUC, positive predictive value, and subgroup performance statistics.
A common gap in clinical AI transparency is the discrepancy between marketing claims ("99% accuracy") and IFU content (which must describe performance in specific clinical contexts with appropriate statistical qualification). The AI Act's transparency requirements are assessed against the IFU and technical documentation — not marketing materials. Vendors should audit their IFU against Article 13 requirements before AI Act enforcement begins, as unsupported performance claims in IFUs are a documented enforcement target.
Article 14: Human Oversight in Clinical Settings
Article 14 requires that high-risk AI systems are designed and developed to enable effective oversight by human persons during the period of use. In clinical settings, this obligation requires careful attention to interface design, workflow integration, and training requirements — because the AI Act evaluates not just whether human oversight is theoretically possible but whether the system design genuinely supports it.
What Article 14 Requires for Clinical AI
Article 17: Quality Management System
Article 17 requires providers to implement a quality management system covering all aspects of the AI system's lifecycle. The QMS must address eight specific subject areas — several of which have no direct equivalent in existing MDR QMS requirements.
QMS Elements Specific to AI
Beyond the standard QMS elements familiar from ISO 13485 medical device quality systems, Article 17 requires the QMS to address:
- Data governance: Policies and procedures for training data selection, labeling, quality assurance, and management of data used throughout the AI lifecycle. This includes documentation of training datasets — sources, exclusion criteria, preprocessing, and annotation procedures.
- AI model validation: Procedures for validating AI system performance prior to deployment and after material changes, including validation study design, statistical analysis, and acceptance criteria.
- Post-market AI monitoring: Systematic collection and analysis of real-world performance data, including procedures for detecting performance degradation, demographic disparities, and unexpected clinical outcomes associated with AI use.
- AI incident response: Procedures for detecting, documenting, and responding to AI system failures — including workflows for suspending AI use when unexpected behavior is detected, notifying the notified body and competent authority when required under Article 62, and conducting root cause analysis.
- Change management for AI: Procedures that define when changes to AI models, training data, or deployment configuration constitute a material change requiring reassessment of conformity.
Article 12: Logging Requirements
Article 12 requires that high-risk AI systems automatically log events relevant for identifying risks and ensuring traceability throughout the AI system's lifetime. For clinical AI deployed in healthcare organizations, logging requirements have direct implications for the AI governance infrastructure that must be in place at the point of care.
CoreGuard Coverage for Healthcare AI Compliance
CoreGuard's governance infrastructure addresses the AI Act's technical requirements in healthcare settings, providing the audit trail, policy enforcement, and documentation artifacts that Articles 9, 12, 13, and 17 require.
| EU AI Act Article | Requirement | CoreGuard Coverage |
|---|---|---|
| Article 9 — Risk Management | Continuous AI risk identification and mitigation | Satisfied — Real-time output anomaly detection; demographic performance monitoring; configurable risk thresholds; post-market surveillance data collection |
| Article 12 — Logging | Automatic event logging with tamper evidence | Satisfied — HMAC-SHA256 signed decision certificates with model version, input context hash, output classification, and chain integrity verification |
| Article 13 — Transparency | Deployer access to performance metrics and limitations | Satisfied — Policy pack configuration documents classification thresholds and decision rules; output statistics exportable per Article 13 format |
| Article 14 — Human Oversight | System design enabling effective clinician oversight | Partial — Configurable escalation thresholds route uncertain outputs to human review queue; override logging; stop-use policy enforcement. Interface design is vendor-specific. |
| Article 15 — Accuracy and Robustness | Adversarial input detection; accuracy monitoring | Satisfied — Input classification detects adversarial patterns and out-of-distribution inputs; performance metric dashboards track accuracy metrics over time |
| Article 17 — Quality Management System | QMS covering data governance, AI monitoring, incident response | Partial — CoreGuard provides technical controls and audit artifacts; QMS procedures and documentation must be developed by the provider. CoreGuard's compliance documentation package includes AI QMS template for healthcare. |
Enforcement Timeline
EU AI Act enforcement timelines for healthcare AI are critical for compliance planning:
- August 2024: AI Act enters into force.
- February 2025: Prohibited AI systems obligations apply.
- August 2025: GPAI model obligations and governance provisions apply.
- August 2026: High-risk AI system obligations (Articles 9–17) apply to new systems placed on the market. This is the primary deadline for clinical AI vendors.
- August 2027: High-risk AI obligations extend to AI systems already on the market before August 2026 that have undergone substantial modification.
- August 2030: Full application including AI systems embedded in MDR-regulated devices already on the market without modification.
The August 2026 deadline for new clinical AI systems is 18 months from the date of this article. Given the documentation depth required by Articles 9, 11, 13, and 17 — and the need to conduct clinical validation studies, build QMS procedures, and implement logging infrastructure — organizations that begin compliance work in 2026 will struggle to meet the deadline. Healthcare AI vendors should initiate gap analysis against the high-risk AI requirements immediately and prioritize the QMS and logging infrastructure, which have the longest lead times.
Frequently Asked Questions
Meet EU AI Act Article 9, 12, and 17 Requirements for Clinical AI
CoreGuard provides the tamper-evident logging, output monitoring, and governance documentation infrastructure that clinical AI vendors need to satisfy EU AI Act high-risk obligations before August 2026.