Healthcare — Use Case

Clinical AI Oversight — HIPAA-Compliant Governance for Healthcare AI Systems

Deploying AI in clinical environments means every inference touches protected health information, clinical decision workflows, and patient safety. CoreGuard enforces HIPAA-compliant AI governance with deterministic pre-execution policy evaluation, PHI access gating, and signed audit certificates that satisfy OCR audit requirements.

See CoreGuard Enterprise Plans

The Clinical AI Governance Challenge

Healthcare AI deployments face a governance challenge with no clean analogue in other industries. Clinical AI systems — whether natural language interfaces on EHR data, clinical decision support tools, medical imaging AI, or patient-facing chatbots — operate in environments where every query potentially touches Protected Health Information (PHI), where erroneous outputs can directly affect patient safety, and where the regulatory framework imposes strict accountability obligations on both covered entities and their business associates.

The failure mode in clinical AI is not hypothetical. An AI assistant with access to an EHR that surfaces medication contraindications to an unauthorized user has committed a HIPAA breach, regardless of whether the information was incidentally included in an AI response rather than a direct database query. A clinical decision support system that generates a diagnosis suggestion it is not FDA-authorized to make has created both a regulatory and a patient safety exposure simultaneously.

Post-generation content filtering — placing a guard after the LLM that scans outputs — cannot address these risks. By the time a guard detects that an AI response contains PHI belonging to a patient the querying clinician is not authorized to access, the inference has already occurred and the response has been generated. The HIPAA minimum necessary standard requires that PHI access be restricted before disclosure, not filtered after generation.

Common Clinical AI Risk Scenarios

PHI LEAK AI assistant surfaces PHI for patients outside the querying clinician's care team via context retrieval

DIAGNOSIS Clinical chatbot generates diagnostic impressions outside its authorized CDS scope

DOSING LLM-assisted prescribing tool recommends dosing without checking patient-specific contraindications

ROLEPLAY Patient-facing AI manipulated into acting as a physician via prompt injection

AUDIT GAP No pre-execution evaluation record available when OCR investigates AI-involved PHI access

HIPAA Requirements for AI Systems

HIPAA does not have an AI-specific rule, but its existing Privacy Rule, Security Rule, and Minimum Necessary standard impose substantive obligations on AI systems that access, process, or transmit PHI. The Office for Civil Rights (OCR) has confirmed in guidance that AI systems operated by covered entities and their business associates are subject to HIPAA's full requirements.

MINIMUM NECESSARY

45 CFR §164.502(b) — Covered entities must make reasonable efforts to limit PHI use to the minimum necessary to accomplish the intended purpose. For AI systems, this means access to patient records must be scoped to the clinical context of the authorized query — not the full patient population accessible to the AI's data connection.

ACCESS CONTROLS

45 CFR §164.312(a)(1) — Technical safeguards must implement technical policies that allow access to ePHI only to authorized persons. AI systems must enforce role-based access controls that prevent clinicians from retrieving PHI for patients outside their care relationship.

AUDIT CONTROLS

45 CFR §164.312(b) — Covered entities must implement hardware, software, or procedural mechanisms to record and examine activity in information systems that contain or use ePHI. AI inference requests involving ePHI must be logged with sufficient detail for OCR audit review.

INTEGRITY CONTROLS

45 CFR §164.312(c)(1) — Electronic ePHI must be protected from improper alteration or destruction. AI-generated summaries or modifications of clinical records must be traceable to the AI system and model version that generated them.

BAA OBLIGATIONS

45 CFR §164.308(b) — Business Associate Agreements must cover AI vendors with access to PHI. Governance evidence — including pre-execution policy evaluation records — supports BAA compliance documentation.

PHI Access Gating with CoreGuard

CoreGuard's healthcare policy pack implements the HIPAA minimum necessary standard at the inference layer. Before any LLM call proceeds, CoreGuard evaluates the requesting clinician's role, their active care team relationships, the patient identifiers referenced in the query, and the authorized scope of the AI system being used. If the query would require surfacing PHI for patients outside the clinician's authorized scope, the request is blocked before the LLM executes.

This pre-execution approach is architecturally distinct from post-generation PHI scanning. A post-generation scanner detecting PHI in an AI response means the LLM has already processed the patient's data and incorporated it into a response. The HIPAA violation occurred at generation. CoreGuard's gate prevents the LLM from accessing out-of-scope PHI by blocking the request before inference begins — the minimum necessary standard is enforced before any PHI is accessed, not after.

Enforcement model: CoreGuard does not scan AI outputs for PHI patterns. It evaluates whether the requesting user is authorized to access the PHI that would be required to answer the query, before the query reaches the LLM. This implements the HIPAA minimum necessary standard as a pre-condition to inference, not as a post-generation remediation.

// CoreGuard healthcare policy pack — PHI access gate
// Evaluates pre-execution; blocks before LLM call

{
  "policy_set": "healthcare_v2",
  "rules": [
    {
      "id": "phi.minimum_necessary",
      "condition": {
        "patient_id_in_query": true,
        "user_care_relationship": "none"
      },
      "disposition": "BLOCK",
      "citation": "45 CFR §164.502(b)"
    },
    {
      "id": "cds.diagnosis_scope",
      "condition": {
        "request_category": "diagnosis",
        "system_authorization": "cds_only"
      },
      "disposition": "BLOCK",
      "citation": "FDA SaMD guidance, Dx scope"
    }
  ]
}

Clinical Decision Support Enforcement

Clinical decision support (CDS) AI systems occupy a complex regulatory space. The FDA's CDS guidance distinguishes between software that meets the non-device CDS exemption (decision-independent of the clinician, low risk) and Software as a Medical Device (SaMD) subject to 510(k) clearance or De Novo authorization. AI systems in the CDS category must stay within their authorized clinical function — providing information to support clinical decisions, not replacing clinical judgment with autonomous diagnosis or treatment recommendations.

Diagnosis Scope Enforcement

Blocks AI responses that constitute diagnostic conclusions rather than decision support. A response presenting differential diagnoses for clinician consideration is within scope; a response stating "the patient has condition X" is blocked as outside authorized CDS scope and potential unauthorized practice.

Prescribing Guardrails

Evaluates medication-related queries against patient allergy records, contraindication databases, and prescribing authorization scope before the LLM generates any medication recommendations. Queries from non-prescribers attempting to access prescribing decision support are blocked at the authorization layer.

Role-Based Clinical Context

Enforces distinct policy sets by clinical role: attending physician, resident, nurse, pharmacist, patient, and administrative staff each have different authorized AI interaction scopes. Requests that exceed the querying role's authorization are blocked with a policy citation returned to the caller.

Patient Identity Verification

Requires that patient identity references in AI queries can be resolved against the querying clinician's active care team roster. Prevents AI systems from being used as a side channel to access records of patients who are not under the clinician's active care.

CoreGuard Healthcare Policy Pack

The CoreGuard healthcare policy pack is a pre-built, HIPAA-aligned policy set covering the most common clinical AI governance requirements. Organizations deploy the pack as a starting configuration and extend it with institution-specific rules using the policy-as-code interface.

The pack covers seven policy domains, each mapping to specific HIPAA provisions and clinical governance requirements:

PHI Access Gating — Minimum necessary enforcement per 45 CFR §164.502(b)
Role-Based Authorization — Clinical role policy enforcement across 12 standard role types
CDS Scope Enforcement — FDA CDS guidance compliance, diagnosis and treatment scope limits
Emergency Override — Break-glass access with mandatory audit trail and supervisor notification
Patient-Facing AI Limits — Stricter limits for patient portals: no clinical advice, escalate to care team
Research Context Separation — De-identified research queries separated from clinical queries
Third-Party AI Vendor Gating — Governs AI embedded from third-party EHR vendors and clinical apps

Each policy domain produces signed Governed Decision Certificates on every enforcement event. Emergency override uses are recorded with the overriding clinician's identity, timestamp, and stated emergency justification — satisfying both HIPAA audit control requirements and Joint Commission documentation standards.

Policy Pack — At a Glance

Policy domains7

Pre-built rules43

Clinical role types12

Policy evaluation latency< 2 ms

HIPAA provisions addressed8

Audit Trail for Clinical AI

HIPAA's audit control requirement — 45 CFR §164.312(b) — requires mechanisms to record and examine activity in systems that contain or use ePHI. For AI systems, this means audit logs must capture not just that an inference occurred, but what policy was in effect at the time, whether the request was authorized, and what disposition was applied. CoreGuard generates a Governed Decision Certificate for every inference request involving clinical data.

■

Pre-Execution Timestamp

Every certificate records an ISO 8601 timestamp generated before the LLM call begins. This timestamp proves that policy evaluation preceded inference — the evidentiary record OCR examiners require when investigating AI-involved PHI access events.

■

Policy Version Binding

Each certificate records the policy set version active at evaluation time. When a policy update occurs, all subsequent certificates reference the new version. Historical certificates remain bound to the policy version that governed them — enabling point-in-time policy reconstruction during audits.

■

Request Hash Binding

A SHA-256 hash of the request content is embedded in each certificate, cryptographically binding the certificate to the specific query it governs. Certificates cannot be reused across queries or retroactively generated for queries that were not evaluated.

■

HMAC-SHA256 Signature

Every certificate is signed with the organization's governance signing key using HMAC-SHA256. Signatures are verifiable offline without querying any live system, supporting air-gapped audit review environments common in regulated healthcare.

■

BLOCK Event Escalation

Policy BLOCK events are forwarded to configurable notification endpoints — SIEM, HIPAA compliance management platform, or on-call security team — enabling real-time awareness of potential PHI access attempts and policy violations as they occur.

Ready to govern your clinical AI?

CoreGuard's healthcare policy pack deploys in under an hour and begins generating HIPAA-compliant audit certificates on your first inference. Speak with our healthcare governance team about your specific clinical AI use cases and regulatory requirements.

Schedule Enterprise Call