HIPAA is the most common compliance gate we see for healthcare AI projects in the US market — and the one teams most often underestimate. The framework itself is straightforward; the hard part is matching modern AI architecture (vector databases, LLM providers, agent frameworks) to a regulatory regime designed in the 1990s. This checklist is the four-step process we walk every healthcare AI project through at NKKTech, drawn from deployments at US hospital systems and digital-health startups.
What HIPAA Actually Requires of AI Systems
HIPAA's Security Rule requires administrative, physical, and technical safeguards on any system that creates, receives, maintains, or transmits Protected Health Information (PHI). For AI systems, this practically means: every vendor that processes PHI must have a signed Business Associate Agreement (BAA) with you, every data flow must be documented and minimized to the least PHI needed, every access to PHI must be logged and attributable to an individual, and you must conduct an annual risk assessment plus a pre-deploy compliance review. Penalties for violations range from $137 per record (tier 1) to $2.07M per category per year (tier 4) — and the OCR has been actively enforcing AI-related violations since 2024.
Step 1: BAAs With Every Vendor That Touches PHI
List every service your AI system uses: LLM provider, embedding provider, vector database, observability platform, error tracker, logging service. For each, either get a signed BAA or ensure no PHI flows through it. As of mid-2026, the major LLM providers with HIPAA-eligible BAAs are: OpenAI (Enterprise tier only), Anthropic (Enterprise tier), Azure OpenAI (with the appropriate Azure subscription tier), and AWS Bedrock (with BAA on the AWS account). Google Vertex AI is BAA-eligible as well. Hugging Face Inference is NOT BAA-eligible at the public API tier — you'd need a dedicated endpoint. The pattern we see most often violated: a team uses OpenAI's standard API (no BAA) instead of the Enterprise/Azure tier, processes PHI through it, and discovers the gap during a security audit. Fix this on day one.
📥 Free Download: Vietnam Offshore Dev Cost Guide 2026
Real developer rates, project cost breakdowns, and a budget planning template. Used by 200+ startup founders.
Ready to build?
NKKTech delivers AI Development projects from $30K.
Fixed scope. Senior Vietnam engineers. 14-day kickoff.
Step 2: Data Flow Architecture (Where PHI Can and Can't Go)
Diagram every flow. Mark every node with whether it sees PHI. Eliminate every node that does not strictly need PHI to function. Common architectural patterns we use to minimize PHI exposure: de-identify (strip 18 HIPAA identifiers) before sending data to non-BAA-covered tools like analytics or A/B testing platforms; use synthetic data in dev/staging environments (never copy production PHI to non-prod); restrict PHI to BAA-covered regions (AWS us-east, Azure East US, etc. — don't let it accidentally egress to a non-covered region via a misconfigured CDN or backup); pseudonymize for ML training (replace identifiers with one-way tokens before any model fine-tuning). Document the resulting architecture in a data flow diagram; the OCR will ask for this during an audit.
Step 3: Audit Logging and Access Controls
Every PHI access must be logged with: the individual who accessed it, the timestamp, the patient record (or scope of records), the action taken (read, write, delete, query), and the system that mediated the access. Logs must be retained 6 years minimum. For AI systems specifically: every LLM call that includes PHI must log the prompt (with appropriate handling — the prompt itself is now PHI), the response, the user on whose behalf the call was made, and the model version. Access controls follow least-privilege: human users authenticate via SSO with MFA, machine principals authenticate via short-lived tokens (15 minutes max), and PHI-touching code paths require a documented business need. We implement this via OpenTelemetry spans tagged with a phi=true attribute, routed to a HIPAA-compliant logging backend (Datadog HIPAA edition, AWS CloudWatch with appropriate config).
Step 4: Pre-Deploy Compliance Review
Before any AI deployment that touches PHI: (1) update your risk assessment to include the new system, (2) verify BAA coverage of every component, (3) review audit logging end-to-end (sample 10 PHI accesses and trace them through the logs), (4) confirm encryption at rest and in transit (TLS 1.2+ for transit, AES-256 for rest), and (5) document the deployment in your incident-response runbook so on-call engineers know how to handle PHI-related alerts. We bundle these into a 1-page deploy checklist for every healthcare client. For the broader compliance picture — including GDPR, PDPA, PIPEDA, APPI, and EU AI Act — see our AI Compliance Guide for 2026.
📥 Free Download: Vietnam Offshore Dev Cost Guide 2026
Real developer rates, project cost breakdowns, and a budget planning template. Used by 200+ startup founders.
Ready to build?
NKKTech delivers AI Development projects from $30K.
Fixed scope. Senior Vietnam engineers. 14-day kickoff.
10+ years building AI systems for Toyota, Sony, and Rakuten in Japan. Founded NKKTech in 2018 with a senior-only engineering model.
Want to build this with NKKTech?
Building a healthcare AI system and need a compliance review? Book a free 30-minute call with a NKKTech engineer who has shipped HIPAA-compliant AI for US hospital systems. We'll review your architecture, flag the gaps, and recommend a remediation order.
Book a Free Call