AI Security for Healthcare

Executive summary

Healthcare organizations are deploying artificial intelligence directly into the paths that patient data travels — the clinical note, the diagnostic read, the patient message, the query against the record. Each of these AI systems creates a new class of exposure for protected health information (PHI) that traditional network, endpoint, and application security tools were never designed to detect. Deflected secures this AI layer, and encrypts everything it handles with post-quantum cryptography so that data which must remain confidential for a patient's lifetime is protected against both today's attackers and tomorrow's quantum computers.

This page is written for the people accountable for that risk: chief information security officers, chief medical information officers, privacy officers, heads of clinical AI, and the compliance and legal teams who have to stand behind every deployment. It explains where AI touches PHI, how the AI-era threat model differs from the one your existing controls address, how these risks relate to HIPAA and the broader regulatory landscape, and precisely how Deflected's products and services close the gap. Every technical term is defined the first time it appears.

AI in healthcare: opportunity and new risk

The clinical case for AI is genuine. Documentation burden is a leading driver of clinician burnout, diagnostic backlogs delay care, and patients increasingly expect responsive digital access. AI addresses all three. But every place AI reduces friction is also a place where PHI now flows through a probabilistic system that can be manipulated, that can leak, and that can behave in ways its builders did not intend. Understanding the opportunity means understanding where the new risk sits.

Clinical documentation and ambient scribes

Ambient clinical documentation tools listen to a patient encounter and draft a structured note — history, assessment, and plan — for the clinician to review. These systems ingest some of the most sensitive spoken content in medicine: symptoms, diagnoses, medications, mental health disclosures, and social history. The audio, the transcript, and the generated note are all PHI. A scribe pipeline typically spans a capture device, a speech model, a large language model that structures the note, and an integration back into the electronic health record. Each hop is a place where data can be intercepted, retained longer than intended, or exposed through a misconfigured integration.

Diagnostic and triage models

Machine-learning models now assist with reading images, flagging abnormal results, stratifying risk, and triaging incoming cases. These models influence clinical decisions, which makes their integrity a patient-safety issue and not merely a data-privacy one. A model that can be subtly manipulated to under-flag a finding, or that behaves differently on a crafted input, is a threat to care itself. Many of these models are also sourced from third parties, which introduces supply-chain considerations covered later on this page.

Patient-facing chatbots

Health systems, payers, and digital-health companies deploy conversational AI to answer coverage questions, help with scheduling, provide medication guidance, and support chronic-condition management. These assistants often need access to a patient's own record to be useful — which means they operate over PHI and, critically, must return the right patient's information and only that patient's information. A patient-facing surface is, by definition, exposed to untrusted input from the public, making it the most heavily probed part of a healthcare AI estate.

Retrieval-augmented generation over EHR data

Retrieval-augmented generation (RAG) is a pattern where an AI system fetches relevant documents — clinical notes, guidelines, prior results — and injects them into a model's prompt so its answers are grounded in real data. In healthcare, RAG is powerful because it lets a model reason over a patient's actual chart. It is also risky because the retrieved content is now part of the prompt: if the retrieval scope is too broad, another patient's notes can end up in the context; if a retrieved document contains hidden instructions, those instructions reach the model. RAG turns the boundary of "what this model can see" into a security control that must be enforced continuously.

Agentic clinical and administrative workflows

The newest and highest-risk pattern is the agent: an AI system granted tools and autonomy to take actions — querying the EHR, drafting orders, updating records, submitting prior authorizations, or messaging patients. Agents compress multi-step work into a single request, but they also convert a language-level compromise into a real-world action. An agent tricked into the wrong behavior does not just say the wrong thing; it can do the wrong thing to a record or a workflow. As healthcare moves from AI that drafts to AI that acts, the blast radius of every AI-layer vulnerability grows.

Regulatory and compliance landscape

Healthcare AI does not get a regulatory exemption because it is new. The obligations that already govern patient data apply in full to AI systems, and a growing body of AI-specific regulation is being layered on top. The frameworks below are the ones a healthcare security and compliance program should hold every AI deployment against. This section is a plain-language map, not legal advice; your privacy officer and counsel remain the authority for how each applies to your organization.

HIPAA Privacy and Security Rules, and PHI

The Health Insurance Portability and Accountability Act (HIPAA) and its implementing regulations remain the foundation. The Privacy Rule governs the use and disclosure of protected health information — individually identifiable health information held or transmitted by a covered entity or business associate. The Security Rule requires administrative, physical, and technical safeguards for electronic PHI (ePHI), including access controls, audit controls, integrity controls, and transmission security. When an AI system processes PHI, it falls squarely within scope: the model, its prompts, its logs, and its outputs are all handling ePHI and must be covered by appropriate safeguards. A vendor that processes PHI on a covered entity's behalf is typically a business associate and must be bound by a business associate agreement.

The HITECH Act and OCR enforcement

The Health Information Technology for Economic and Clinical Health (HITECH) Act strengthened HIPAA, notably by establishing the Breach Notification Rule and increasing enforcement and penalty structures. Enforcement sits with the Department of Health and Human Services Office for Civil Rights (HHS OCR), which investigates breaches and compliance failures and can impose corrective action and civil monetary penalties. The practical implication for AI is direct: an unauthorized disclosure of PHI through a model's output, an over-broad retrieval, or a compromised agent can constitute a reportable breach, with all of the notification and enforcement consequences that follow. Demonstrable safeguards and audit evidence are what stand between an incident and a finding.

FDA oversight of AI/ML as Software as a Medical Device

When an AI system is intended to diagnose, treat, or inform clinical management, it may meet the definition of a medical device and fall under U.S. Food and Drug Administration (FDA) oversight as Software as a Medical Device (SaMD). The FDA has developed dedicated policy for AI/ML-based devices, including its work on predetermined change control and good machine-learning practice, reflecting that these models learn and change over time. Security is part of this picture: the integrity of a diagnostic model and protection against tampering are relevant to both safety and regulatory standing. Manipulation of a clinical model is therefore not only a security incident but potentially a device-integrity issue.

The EU AI Act and high-risk health uses

The European Union Artificial Intelligence Act (EU AI Act) introduces a risk-based regime for AI systems. Many healthcare applications — including AI that is a safety component of a medical device, and AI used in clinical decision-making — are treated as high-risk, triggering obligations around risk management, data governance, technical documentation, logging, human oversight, accuracy, robustness, and cybersecurity. Any organization serving patients or operating in the EU, or building products for EU markets, should assume these obligations apply to their clinical AI and plan the controls and documentation accordingly.

GDPR and special-category data

The General Data Protection Regulation (GDPR) treats health data as a special category of personal data subject to heightened protection, with strict conditions for lawful processing, requirements for data protection by design and by default, and obligations around breach notification and data subject rights. AI systems that process the health data of individuals in the EU must satisfy these requirements — including demonstrating a lawful basis, minimizing data, and securing it appropriately. GDPR's emphasis on protection by design aligns closely with encrypting PHI with durable, forward-looking cryptography rather than treating security as an afterthought.

AI-era threats specific to healthcare

The threats below are not hypothetical variations on ordinary IT risk. They arise from how AI systems actually work — natural-language instructions, learned behavior, and retrieval of untrusted content — and they land with particular force in healthcare because the data at stake is regulated, sensitive, and long-lived.

PHI leakage through model output

An AI system can disclose protected health information simply by generating it. A model with access to records or notes can be led — deliberately or accidentally — into producing a patient's name, medical record number, diagnosis, or history in its response. Traditional data-loss prevention watches files and network flows; it does not read a model's natural-language output for regulated identifiers escaping in prose. In healthcare, a single such disclosure to the wrong recipient can be a reportable breach.

Prompt injection in clinical and patient workflows

Prompt injection is the defining AI vulnerability: an attacker hides instructions inside untrusted input — a patient message, an uploaded document, a retrieved note — to hijack the model's behavior. In a patient chatbot, an injected instruction can attempt to coax the assistant into revealing information it should not. In a clinical summarization or agentic tool, injected content in a retrieved document can override the system's instructions or redirect an action. Because so much of healthcare AI operates over free text that originates outside the organization, prompt injection is a first-class threat, not an edge case.

Manipulation of diagnostic and clinical-decision models

Models that inform clinical decisions are targets for manipulation. An attacker who can influence a model's behavior — through poisoned training data, a crafted input, or a compromised dependency — can push it to produce subtly wrong outputs: under-flagging a finding, mis-stratifying risk, or biasing a recommendation. Unlike a data breach, this is an integrity and patient-safety threat, and it can be difficult to detect precisely because the output still looks plausible.

Adversarial inputs

Adversarial inputs are specially constructed inputs designed to cause a model to misbehave — for example, perturbations to an image or a carefully worded query that reliably produces an incorrect or unsafe result. In a diagnostic context, adversarial inputs threaten accuracy; in a conversational context, they are a vector for jailbreaks and data extraction. Defending against them requires testing models the way an attacker would, continuously, rather than assuming that accuracy measured on clean data holds under attack.

Model supply-chain risk in third-party medical models

Few healthcare organizations build their models from scratch. They adopt third-party diagnostic models, pretrained foundation models, open-source components, and external datasets. Each of these is a link in a supply chain that can carry poisoning, backdoors, or hidden triggers introduced upstream. A backdoored model can behave normally until it encounters a specific trigger, at which point its behavior changes — a serious risk when that model reads images or informs care. Vetting models and datasets before they enter the pipeline is now a core security function.

Harvest now, decrypt later against long-lived records

Health records are among the longest-lived sensitive data any organization holds. A diagnosis, a genetic marker, or a mental-health history remains sensitive for a patient's entire life and beyond. Adversaries are already capturing encrypted data today to store and decrypt later, once quantum computers can break today's public-key cryptography — a strategy known as harvest now, decrypt later. For data with a multi-decade sensitivity horizon, classical encryption alone is no longer a sufficient long-term guarantee. This is why post-quantum protection matters more in healthcare than almost anywhere else.

How Deflected protects healthcare

Deflected addresses the threats above with a combination of always-on software products and expert services. The capabilities below are the ones most relevant to a healthcare environment. Each links to its full breakdown; here we describe what it does in a clinical and PHI-specific context.

Compliance and audit-readiness

Security that cannot be demonstrated is incomplete — and in healthcare, demonstration is the whole game. Regulators, auditors, and enterprise partners expect an organization to prove its AI is governed. Deflected maps its controls and the evidence it produces to the frameworks that matter most, so your AI layer contributes directly to your compliance posture rather than becoming an unmonitored gap.

• HIPAA Security Rule safeguards — Deflected's audit logging supports audit controls; its inspection and blocking support access and transmission-security objectives for ePHI; and its evidence trail supports the accountability required when PHI is processed by AI systems.
• NIST AI Risk Management Framework (AI RMF) — the leading voluntary framework for identifying and managing AI risk across the model lifecycle. Deflected's discovery, testing, and governance work map to its govern, map, measure, and manage functions.
• SOC 2 — the trust-services criteria that enterprise and healthcare procurement teams rely on to evaluate a vendor's security posture, supported with implemented controls and evidence.
• EU AI Act — for high-risk healthcare AI, Deflected supports obligations around logging, robustness, cybersecurity, and technical documentation.

For a deeper treatment of how these mappings work in practice, see our compliance overview. It is important to be precise about scope: Deflected supports readiness. It is not an accredited auditor or certification body, and it is not a substitute for a complete HIPAA compliance program, a Security Rule risk analysis, or legal counsel. Its role is to make the AI layer of your organization defensible and demonstrable, so that the broader compliance program your privacy and security teams own has solid ground beneath its newest and fastest-moving component.

Real-world scenarios

The following scenarios are illustrative — constructed to show how these threats unfold and where Deflected intervenes. They are not accounts of specific customers or incidents.

A patient chatbot coaxed into revealing another patient's PHI

A health system deploys a patient-facing assistant that can answer questions about appointments and results by retrieving from the patient's record. An attacker, interacting through the public chat surface, crafts a series of messages containing hidden instructions — telling the assistant to ignore its scoping rules and to treat a supplied identifier as the authorized patient. Because the assistant assembles its prompt from user input and retrieved data, the injected instruction becomes part of what the model reads, and a naively built system attempts to comply, pulling another patient's information into the response. This is prompt injection combined with a broken authorization boundary, and the result is an unauthorized disclosure of PHI — a reportable breach. With Prompt Firewall inline, the injected instructions are detected as adversarial input in the request path and neutralized before they reach the model, and any PHI in a response destined for the wrong recipient is caught and blocked on the way out. Every one of these decisions is logged, giving the privacy team a clear record for investigation.

A clinical summarization tool leaking notes

A clinical summarization tool condenses a patient's chart into a concise brief for the care team. To do so it retrieves notes and results and passes them to a model. Two failure modes converge. First, an over-broad retrieval scope occasionally pulls in a note that belongs to a different encounter or patient, so PHI that should never have been in the context is now available to the model — and to the output. Second, a retrieved document that originated outside the organization contains embedded instructions that manipulate the summary. Without inspection, both problems surface silently in the generated brief. Prompt Firewall inspects the assembled prompt and the resulting summary, flagging out-of-scope PHI and embedded instructions; Continuous AI Red Team would have surfaced the over-broad retrieval and the injection susceptibility in advance by testing the tool the way an attacker would, turning a latent leak into a fixed finding before it reached production.

A third-party diagnostic model with a hidden backdoor

A radiology group licenses a third-party model to assist in reading images. The model performs well in evaluation. Unknown to the group, it carries a backdoor introduced upstream in its training: on inputs containing a specific, innocuous-looking trigger, its behavior shifts to under-flag a class of findings. In normal use nothing appears wrong, which is exactly what makes the risk severe — it is an integrity and patient-safety threat that ordinary accuracy metrics do not reveal. Model Supply-Chain Security vets the model, its provenance, and its dependencies before deployment, probing for poisoning, backdoors, and hidden triggers, and produces a documented sign-off. Where a model is already in use, Continuous AI Red Team provides ongoing adversarial evaluation to detect anomalous behavior under crafted inputs, and AI Incident Response is on retainer if a compromise is confirmed.

A staff member pasting PHI into a public chatbot

A well-meaning staff member, trying to save time, pastes a block of clinical text into a public consumer AI tool to reword a patient letter. The text contains identifiers. That single action moves PHI outside the organization's controlled environment, potentially in violation of policy and creating breach exposure — and it happens without any malicious intent and without touching a monitored application. Shadow AI Discovery surfaces this class of unsanctioned usage across the organization, quantifies the exposure, and gives the security and privacy teams what they need to bring the behavior under a governed policy — often paired with sanctioned, safer alternatives so staff keep the productivity benefit without the risk.

Why Deflected for healthcare

Two properties make Deflected especially suited to healthcare, and they follow directly from the nature of patient data.

Quantum-secured by default

Every byte that flows through Deflected is protected with post-quantum cryptography — encryption based on algorithms designed to resist attacks from both classical and quantum computers. This is the default across the platform, not a premium add-on. Deflected uses the standards finalized by the U.S. National Institute of Standards and Technology (NIST):

• ML-KEM-1024 (formerly CRYSTALS-Kyber, NIST FIPS 203) for key encapsulation, at a 256-bit quantum security level.
• ML-DSA-87 (NIST FIPS 204) and SLH-DSA (NIST FIPS 205) for digital signatures that remain unforgeable in a post-quantum world.
• Hybrid X25519 + ML-KEM key exchange, running a proven classical algorithm alongside the post-quantum one, so you remain protected even if either scheme is ever weakened.
• AES-256-GCM for symmetric encryption of data at rest and in transit.

Protecting PHI for its full lifetime

The practical benefit for a healthcare organization is a guarantee that maps precisely to the nature of its data: a diagnosis recorded today must stay confidential for decades, and post-quantum protection is what keeps it confidential against the harvest-now, decrypt-later threat. Combined with AI-layer defenses — prompt-injection protection, PHI-leak detection, model supply-chain vetting, and continuous adversarial testing — Deflected protects patient data across its entire lifetime, not just against the attacks that exist today. It is designed to complement your existing network, endpoint, identity, and cloud security and the safeguards of your EHR vendor, adding the AI-specific layer those tools were never built to provide.

Getting started: your first engagement

Deflected is built to reach production without becoming a project that never ends. A typical first engagement in a healthcare environment follows a deliberately short path, sequenced to reduce risk fastest where PHI is most exposed:

1. Map — we identify where AI touches PHI across your organization: ambient scribes, diagnostic and triage models, patient chatbots, RAG pipelines over EHR data, and any agentic workflows, along with the data flows that connect them.
2. Discover — Shadow AI Discovery surfaces unsanctioned AI usage and quantifies where patient data may already be leaving controlled environments, so the highest-exposure gaps are addressed first.
3. Integrate — always-on products such as Prompt Firewall are placed inline in the request path with low latency and safe fallbacks that never break a clinical workflow, and inspection is calibrated to your applications and policies.
4. Validate — Continuous AI Red Team and, where relevant, Model Supply-Chain Security test your models and pipelines adversarially, producing prioritized findings and a documented posture.
5. Govern and operate — you get a readable dashboard, alerting on what matters, an immutable audit log of every decision, control and evidence mappings for your compliance program, and expert services on standing retainer for the moments that need a human.

Engagements are scoped to your environment, and appropriate agreements — including a business associate agreement where Deflected processes PHI on your behalf — are put in place as part of onboarding. The goal is a defensible, demonstrable AI layer that keeps pace with how quickly your clinical and administrative teams are adopting AI.

Frequently asked questions