HIPAA Compliance Verification for Healthcare AI Vendors

A practice evaluating an AI vendor usually asks the wrong question. It asks whether the vendor is HIPAA-compliant, as though compliance were a certificate HHS hands out after an inspection. HHS certifies nothing. No federal agency issues a "HIPAA-certified" seal for software, and a vendor using that phrase is marketing a term with no legal standing. Compliance is a property of a specific deployment: whether this configuration, handling this PHI, in this workflow, satisfies the Privacy Rule, the Security Rule, and the Breach Notification Rule. The verification burden sits with the healthcare organization, not the company selling the tool.

The same vendor platform can pass that test in one use case and fail it in another. Three deployments, three risk profiles, one vendor name on all three contracts.

Consumer-grade AI tools compound the confusion. Enterprise versions of those same products may offer BAA-eligible configurations, but you have to confirm that directly with the vendor and configure it correctly. A product name does not carry compliance status with it from one tier to another.

A practice that selects a vendor because it advertises a compliance badge, flips a setting, and moves on has not conducted a HIPAA compliance review. It has substituted a vendor's marketing claim for its own verification, which is the exact gap this guide is built to close.

The four conditions that must hold simultaneously for an AI deployment to be compliant

Four conditions have to hold at the same time, for the specific deployment under review, not for the vendor's platform as a general proposition. Drop any one of them and the deployment is non-compliant regardless of how the other three look on paper.

A signed business associate agreement has to cover the service before PHI reaches it. Any vendor that accesses PHI on a healthcare organization's behalf qualifies as a business associate under HIPAA, and that status extends past the primary vendor to AI platforms, MLOps tooling, model developers, and any subprocessor in the chain that touches patient data.

PHI has to stay inside a boundary the organization actually controls. On-premises, private-cloud, and air-gapped deployments keep prompts, retrieved documents, and generated output on infrastructure the practice or its contracted environment governs directly. Cloud deployments can satisfy this condition too, but only on a BAA-eligible tier where you spell out data residency controls instead of assuming them.

Every access to PHI has to be logged, and for AI systems that means logging prompts, retrievals, and generated output. That is a materially higher bar than the logging standard most practices apply to conventional software, and it is the condition vendors most often fail quietly.

Access has to be limited by role, and the limit has to govern what the model itself is permitted to retrieve, not merely who is allowed to open the application. If a human employee pulling a chart has to meet the Privacy Rule's minimum necessary standard, so does machine retrieval.

A proposed 2026 update to the HIPAA Security Rule, not yet finalized at the time of this writing, would raise encryption from an "addressable" specification to a required one for all ePHI, including data processed by AI systems, and would add new vulnerability scanning and network segmentation requirements across any system that handles ePHI. Healthcare operations platforms that deploy AI agents across EHRs and payer portals, Novoflow among them, have to satisfy all four conditions in concert: a signed BAA covering the service, PHI boundaries enforced through the organization's own infrastructure controls, comprehensive audit logging of every prompt and retrieval the agent performs, and role-based limits on what the agent itself can pull from each connected system. Every section that follows is an expansion of one of these four conditions, with the specific documentation and questions needed to verify it.

What a BAA for an AI vendor must say

A signed BAA is necessary, and it is routinely mistaken for sufficient. Many agreements in circulation today were written for traditional EHR and billing vendors years before generative AI entered clinical workflows, and they say nothing about model training, inference logging, or exposure introduced by an MLOps pipeline. A covered entity holding that kind of document has a signature on file and a gap in its protection at the same time.

An AI-specific BAA has to address four things explicitly: how the AI model interacts with PHI, what happens to that data after it's processed, whether it is used to improve the model, and what security measures protect PHI as it moves through the AI pipeline. The most consequential of these is the guarantee that patient data is never used to train models that benefit other clients. That commitment has to appear in the BAA itself, in explicit terms. Language in a general privacy policy promising data "won't be used for training" does not carry the weight of a BAA commitment, and a vendor that points to its privacy policy instead of its contract language hasn't answered the question.

The clearest illustration of how this breaks down is a voice AI stack. A signed agreement with the platform does not automatically extend to its subprocessors. On developer-first platforms built around "bring your own API keys," sourcing and maintaining each of those underlying agreements becomes the practice's own responsibility, often at enterprise pricing tiers the practice has to negotiate independently, and the practice carries full liability for any gap in that chain.

For AI agents that operate across multiple connected systems, the BAA needs to be explicit about how the agent interacts with PHI at each connection point: what it retrieves from the EHR versus what it reads off a payer portal, what happens to that data once it's processed, and a contractual guarantee that data collected during the agent's operation is never used to train models benefiting other clients, whether inside the same vendor's product or a third party's.

How to verify the PHI boundary

A vendor's language about data privacy stays a marketing claim until you check it against the actual architecture handling the deployment. "Your data stays private" means something specific and verifiable: where inference physically runs, under what infrastructure, and under whose control. It does not mean whatever a sales conversation implies.

Two architectures give a practice meaningful control over that boundary. The first is on-premises or air-gapped deployment, where PHI never leaves the provider's own network. The second is private-cloud deployment on government-grade or BAA-eligible infrastructure, where the practice retains data residency controls rather than ceding them to a shared environment. Hathr.AI, for example, runs on FedRAMP High, AWS GovCloud infrastructure, the same category of servers approved for use by the Department of Health and Human Services, and contractually guarantees that customer data is never used to train AI models; it signs a BAA promptly and extends that coverage to every plan, including single practitioners, with no seat minimums. Cloud AI under a properly signed BAA, through Microsoft Azure, AWS, or Google Cloud, can also be compliant. You need to confirm that the specific service tier and configuration in use matches what the BAA actually covers, because the default configuration on these platforms is frequently not the BAA-eligible one.

The boundary extends further than storage. When an AI agent reads a patient record, an EHR screen, or a payer portal and feeds that content into a prompt or a retrieval query, that content is PHI being processed by the AI system, and it has to be covered by the deployment's privacy controls, not just by whatever encryption protects the database at rest.

This is where computer-use agents open a gap that most vendor compliance reviews haven't caught up to. An agent that operates screens, reading EHR displays, navigating payer portal interfaces, interacting with desktop software, processes whatever PHI is rendered on those screens in the course of doing its job. So legacy vendor agreements create BAA obligations most were never written to address, because they were drafted assuming data moves through structured transfers, not through something closer to a human reading a monitor.

Verifying the boundary for this kind of agent means asking where inference runs, on which cloud provider and tier, whether the compute environment is shared with other customers, who can access the underlying infrastructure, and whether the vendor can produce its subprocessor list on request. It also means confirming where the agent's prompts and retrieved data actually persist once generated, whether they pass through the vendor's cloud infrastructure, whether they land in a centralized log the practice itself controls, and whether the architecture supports an audit trail precise enough to show what the agent pulled from the EHR, what it read on a payer portal, and where its output ended up. On-premises or air-gapped deployments offer the cleanest version of that control. Cloud-based agents can still meet the standard, but only if you enforce data residency controls and role-based retrieval limits at the integration layer, not leave them to the platform's default settings.

Audit logging and access control for AI systems

Practices already understand audit logging and access control standards for conventional healthcare software, but they get applied inconsistently to AI systems, largely because most vendor assessments were built before anyone had to account for what an AI system uniquely does with PHI.

If a vendor can produce sign-in logs but not prompt-and-response logs, it has not met the audit logging bar for an AI deployment handling PHI. Retention windows matter here too: the BAA and accompanying security documentation should state how long prompt logs, retrieval logs, and output logs are kept, and who has access to them once they're stored.

Access control follows the same logic. Role-based limits have to govern what the model itself can retrieve. If a scheduling agent can pull clinical notes that have nothing to do with scheduling, because access controls were set at the application level rather than at the point of retrieval, the Privacy Rule's minimum necessary standard has already been violated, regardless of what the login screen enforces. No vendor configures it that way automatically.

Administrative safeguards carry their own AI-specific weight: documenting AI system risk assessments, writing policies for AI-specific data handling, training staff on proper use of these tools around PHI, and building incident response procedures that account for AI failure modes, including what happens when an agent hits an unexpected screen state or fails partway through a workflow. A practice evaluating a vendor on this front should be asking what the audit log actually captures, how long logs are retained, whether logs can be exported for the practice's own compliance review, and what role-based retrieval controls exist at the model level, not just at the login screen.

The AI risk assessment healthcare organizations are now expected to conduct

Regulators now expect healthcare organizations to run AI-specific risk assessments that go beyond the traditional annual Security Risk Analysis. The Federal Register formally withdrew the non-finalized provisions of HTI-2 effective December 29, 2025, citing a focus on deregulation and the emergence of new AI technologies, and a parallel proposal under HTI-5 would remove AI model card requirements. Neither move was centered on AI oversight specifically: HTI-2's withdrawn provisions concerned USCDI v4, imaging exchange, and public health interoperability. But the practical effect of less federal scaffolding is that the work of vetting an AI vendor's model, training data, and update process shifts onto the provider organization itself, which makes building an internal verification process more urgent, not less.

A standard Security Risk Analysis evaluates known, fixed systems.

That assessment belongs in its own document, separate from the standard SRA, and should cover the specific PHI the system accesses, the architectural controls in place, the full subprocessor chain, the BAA terms, the results of any third-party security audits the vendor has had conducted, and the practice's own incident response procedure if the AI system fails or is breached. Physical safeguards still apply wherever any of this runs on-premises: server rooms, workstations with access to the AI system, and backup storage for AI-generated data all have to meet the same physical security standard as any other environment handling ePHI.

Verifying AI agents that operate software screens instead of receiving data through an API

Computer-use agents that operate EHR screens, payer portals, and other software interfaces create a form of PHI exposure that standard vendor compliance questionnaires were not built to capture, because the PHI in question is processed visually rather than transmitted as a structured data packet, and most BAA templates assume the latter.

When an agent reads a patient record rendered in a browser, navigates a payer portal to check an authorization status, or types patient information into a scheduling field, the PHI displayed in that interface is being processed by the AI system in the course of that action. The exposure is continuous and contextual as the agent moves through a screen, so audit logging in this setting has to capture entire agent sessions, not isolated data queries.

A practice evaluating a computer-use agent should ask how the system logs sessions that involve PHI, what the boundary of the agent's access actually is, whether it can navigate to records outside the scope of the task it was assigned, how its session is authenticated, and what specific controls prevent it from reaching PHI beyond the minimum necessary for that workflow.

Subprocessor coverage becomes a sharper concern in this setting. If the agent's screen-reading and decision-making depend on an underlying vision model or language model processing screen captures, that model is a subprocessor handling PHI in its own right, and it requires BAA coverage of its own, independent of whatever agreement covers the orchestrating platform. A practice that confirms the platform's BAA and stops there has verified one layer of a system that may have several.