Is Customer Data Safe with AI Tools? A Guide for Medical Practices

AI Adoption Is Exploding in Healthcare — So Is the Risk

The numbers tell two stories at once. Ambient AI scribes cut clinical note-writing time by 72% at Rush University Medical Center, and an AMA study found physicians save 2.5 hours daily from ambient documentation tools. Yet those same tools process the most sensitive data a practice holds, and healthcare breaches affected 139.7 million people in the past year.

The adoption curve explains why this tension can't wait. According to industry research, 71% of U.S. hospitals ran at least one EHR-integrated predictive AI tool in 2024, and ambient documentation is now the most widely adopted AI use case among healthcare systems. Domain-specific AI implementation grew 7x over 2024. Efficiency gains are real, and practices are adopting AI regardless of whether their safety practices have caught up.

That gap is where the danger lives. Generative AI systems pose what researchers call acute privacy and security risks precisely because of their vast data requirements and opacity — models need enormous inputs to work well, yet neither you nor the vendor can fully see what happens to data once it enters the system. Risks span the entire lifecycle: data collection, model development, and deployment.

And the cost of getting it wrong extends past penalties:

• Civil penalties under HHS 2026 tier guidelines can exceed $2.15 million for the most serious violations.
• A single incident — like the 1.6 million patient records shared without proper consent in the Google DeepMind–NHS case — becomes public record permanently.
• Patient trust, once compromised, is far harder to rebuild than any compliance file. Researchers note that any compromise of medical data damages not just breach statistics but patients' perceived reliability of medical institutions themselves.

This is why patient data safety can't be an afterthought bolted on after deployment. The same model can be compliant in one configuration and a HIPAA violation in another — deployment, not the tool, determines compliance. A free AI assistant that helps a staff member draft a letter can expose PHI the moment a patient record is pasted into it, even if nothing is ever technically breached, because consumer tools often use inputs to improve their models by default.

The practical takeaway for any practice, clinic, or healthcare business adopting AI: treat every AI tool that touches patient data as a compliance decision, not a productivity experiment. That means verifying vendor claims rather than trusting marketing badges, requiring signed BAAs across the entire vendor chain, and knowing exactly which tools your staff are authorized to use. Vetting the building blocks matters too — published research has found roughly 26% of agent skills carry at least one vulnerability, which is why SkillzBench is building free security scanning and trust scoring for AI agent skills before they ever reach your workflow. Efficiency is worth pursuing. Doing it safely is what makes it sustainable.

No AI Tool Is Inherently HIPAA Compliant — Deployment Decides

A vendor badge that says "HIPAA compliant" tells you almost nothing. There is no federal HIPAA certification, seal, or registry, and the HHS Office for Civil Rights does not pre-clear or endorse any product — so every such claim is a vendor's self-assessment, nothing more. The FTC has already taken action against digital health platforms for misrepresenting their compliance status, so skepticism is warranted, not paranoid.

The reason is structural: HIPAA's Security Rule is technology-neutral. It contains no AI-specific provisions, and the same Privacy, Security, and Breach Notification Rules govern PHI whether it's handled by a human, traditional software, or an AI model. As HIPAA Journal puts it, AI doesn't sit outside HIPAA — it's simply another mechanism through which PHI may be used or disclosed. The practical consequence is that the same AI model can be fully compliant in one deployment and a violation in another, depending on configuration, encryption, access controls, and legal terms.

That's why consumer and free AI assistants are off-limits for patient data. They don't offer Business Associate Agreements, and many use your inputs to improve their models by default — so pasting a patient record into a free chatbot can expose PHI even if nothing is ever technically "breached." A signed BAA is the floor, not the ceiling: a legal analysis from Fisher Phillips notes that vendor terms of service often contradict the BAA, that training a commercial model on PHI is generally impermissible without de-identification, and that many vendors can't even show an unbroken BAA chain through their downstream LLM and API providers.

Two regulatory gaps deserve special attention, because they sit entirely outside your control:

• When a patient shares their own health information with an AI chatbot for medical advice, the AI developer is neither a covered entity nor a business associate — so that data falls outside HIPAA entirely.
• De-identified data carries no restriction on use or disclosure, and re-identification risk is significant for dominant tech companies like Meta, Google, and Microsoft, which hold massive amounts of personal information that can be triangulated.
• The FDA has issued no guidelines or regulations covering LLMs such as ChatGPT or Bard, leaving a wide accountability vacuum for AI failures.

The stakes of getting deployment wrong are real: 139.7 million people were affected by healthcare data breaches in the previous year, and top-tier civil penalties under HHS 2026 guidelines exceed $2.15 million. For practices evaluating AI tools, the discipline that matters is verification — collecting risk analyses, subprocessor lists, and breach terms rather than trusting badges. The same verification mindset applies to the AI agent skills practices increasingly bolt onto their workflows; since published research found roughly 26% of agent skills carry at least one vulnerability, SkillzBench is building a free directory that security-scans and benchmarks every skill so teams can see the evidence before deploying. Compliance always rests on deployment and a signed BAA — never on the tool's name.

The Five-Pillar Deployment Checklist for Safe AI Use

The good news: safe AI deployment in a medical practice doesn't require reinventing compliance — it requires applying the same HIPAA discipline you already use, with a few AI-specific additions. Compliance experts converge on a five-pillar checklist, and a platform missing any one of them cannot be considered compliant.

The five pillars of compliant AI deployment

• Encryption. PHI must be protected in transit with TLS 1.2 or higher and at rest with AES-256. A proposed HHS rule would make encryption mandatory rather than merely "addressable," so treat it as non-negotiable now.
• Access controls. Role-based access control, unique user IDs, and multi-factor authentication are the baseline. Critically, access control must also govern what the AI itself can retrieve at runtime — that boundary matters as much as, if not more than, the human permissions around it.
• Audit logging. Log every AI interaction involving PHI, including agent actions, so you can demonstrate minimum-necessary use and investigate incidents.
• A signed BAA — with an unbroken chain. Every downstream LLM and API provider needs its own BAA, and many vendors cannot show one. Make the BAA and security addendum control over the vendor's terms of service.
• Data minimization. De-identify before input where possible, removing all 18 HIPAA identifiers under the Safe Harbor method, and apply minimum-necessary standards to everything AI touches.

These pillars matter because AI introduces breach triggers that classic security programs don't anticipate. According to HIPAA Journal analysis, a breach can occur when staff paste PHI into a third-party AI tool without a BAA, when a vendor's AI re-identifies de-identified data, or when AI-generated outputs are shared with more than the minimum necessary information. The stakes are real: 139.7 million people were affected by healthcare data breaches in a single year, and top-tier civil penalties now exceed $2.15 million.

Deployment — not the tool — determines compliance, which is why verification beats trust. A vendor's "HIPAA compliant" badge is a self-assessment; OCR does not certify or endorse any product, and deficient risk analysis remains the most common failing in its investigations. Collect risk analyses, subprocessor lists, and breach terms instead, and route every new AI tool through a standing privacy and security review gate. That same verification mindset applies to the agent skills and connectors your technical staff may bolt onto AI workflows — SkillzBench is building free security scanning, version pinning, and sandbox benchmarking so teams can see which skills are safe before deploying them.

Finally, keep humans in the loop. AI should augment rather than replace clinical judgment, and every AI output touching patient care or documentation deserves validation before it becomes part of the record.

Verify Vendors, Keep Humans in the Loop

When a vendor's sales deck says "HIPAA compliant," what you're actually looking at is a self-assessment. There is no federal HIPAA certification, seal, or registry, and OCR does not pre-clear or endorse any product — so a badge on a website proves nothing about how the tool will behave in your practice. The FTC has already taken action against healthcare platforms for overstating compliance, which is why legal analysts at Fisher Phillips urge practices to collect evidence instead of trusting marketing.

That evidence should include the vendor's own risk analysis documentation, a complete subprocessor list, and breach notification terms. Do not accept a SOC 2 report or HITRUST certification as a substitute — neither replaces a deployment-specific Security Rule risk analysis for your organization. Ask specifically whether the vendor can show an unbroken BAA chain through every downstream LLM and API provider, because many cannot.

Operationally, the fix is structural. Build a standing intake gate so every new AI tool — from ambient scribes to coding assistants — passes through privacy and security review before anyone touches it with real patient data. The same rigor should apply to the agent skills and extensions your technical staff install; research has found roughly 26% of published agent skills carry at least one vulnerability, which is why SkillzBench is building a free directory that security-scans, version-pins, and benchmarks every skill before it reaches an agent. Vetting the components your AI runs on is part of the same discipline.

Verification alone is not enough — keep a human in the loop. AI auto-coding matched expert human coders 96% of the time, but only with human review in the workflow. And retrospective accuracy can collapse under real conditions: Epic's widely deployed sepsis model identified high-risk patients 87% of the time over full hospital stays, but just 62% when limited to pre-sepsis-criteria data. High performance on paper does not demonstrate clinical utility.

Finally, train your workforce. Staff should know which tools are authorized, why pasting a patient record into a free consumer assistant is off-limits (those tools offer no BAA and often use inputs to improve their models), and how to report AI outputs that look wrong. A few concrete rules to enforce:

• No PHI in any tool that has not cleared the intake gate and signed a BAA.
• Every AI-generated note, code, or summary gets human review before it enters the record.
• Document every AI tool in your Security Rule risk analysis and asset inventory.

The floor is rising, too. A proposed HHS rule would make encryption mandatory rather than "addressable," several state laws now require explicit consent before sensitive data feeds automated processing, and the FTC continues to actively enforce health-data privacy. Practices that verify vendors and keep humans in the loop today will not be scrambling when the rules harden tomorrow.

Vetting the AI Skills and Agents You Connect to Your Workflow

If you've vetted your AI vendor down to the BAA chain, there's one layer most practices forget: the skills and agents you connect to that vendor's workflow. Published research found that roughly 26% of AI agent skills carry at least one vulnerability — meaning nearly one in four components bolted onto your AI setup could be a quiet entry point for exposure.

Agent skills are small instruction packages that extend what an AI agent can do — drafting notes, summarizing records, automating billing steps. They're shared freely across developer communities, and most arrive with no security review at all. The same verification discipline you'd apply to a clinical AI vendor applies here, just at a smaller scale.

What that discipline looks like in practice:

• Security-scan every skill before it touches your workflow — never install on trust alone.
• Pin versions so a skill can't silently change underneath you after you've approved it.
• Know exactly what data each skill and agent can access at runtime, not just which staff member is logged in.
• Re-check periodically; skills update, and upstream changes can introduce new risk.

That last point matters more than it sounds. Access control must govern what data a model or agent can retrieve while it's running — a boundary that matters as much as the human permissions around it. An agent with broad runtime access to records turns a minor skill flaw into a potential PHI disclosure.

This is the gap SkillzBench is being built to close. It's a free, independent directory of AI agent skills that crawls and dedupes skills from GitHub, pins every version to a specific commit, runs license checks and security scans, and publishes a trust score from security, quality, license, and maintenance signals. Its planned benchmarking tests each skill in a sandbox — with and without the skill — so you can see the measured lift rather than a vendor's marketing claim, and scores stay separate from any advertising revenue.

Its free MCP connector adds another safeguard: per-agent loadouts, where you toggle which skills and tools each agent can see. The agent only accesses what you've chosen — a direct implementation of the data-minimization principle that HIPAA guidance treats as a core compliance pillar.

Vetting at the skill layer takes minutes when the scanning, versioning, and benchmarking work has been done for you. SkillzBench is currently in pre-launch; you can join the waitlist to get access when the directory and MCP tool go live.

ctaText: Join the SkillzBench waitlist — get free access to scanned, version-pinned agent skills when we launch.

socialProofText: Built on the same verification standard healthcare demands: every skill security-scanned, version-pinned, and benchmarked before you install it.