Policies, vendor checks, evidence

A proportionate AI governance program your team will follow and an auditor can read

We help build the AI inventory, risk tiers, vendor checks, review procedures and monitoring that buyers, boards and regulators can ask about. We work alongside your counsel and do not give legal advice.

Book a discovery call

Short answer

What is AI governance for a healthcare business?

AI governance is the set of decisions, owners and records that control how a healthcare organization uses AI: which uses are allowed, how risky each one is, who reviews outputs, what vendors may do with data, and how problems are found and fixed. Done well, it is proportionate to risk and easy to follow.

The problem

Problems this service solves

Nobody can list the AI in use

Tools arrive through vendors, free trials and individual staff. When a buyer or regulator asks for a list, someone assembles it by hand.

Policies exist, but not for the jobs people do

A general AI policy tells a billing specialist little about appeal letters. It also does not tell a front desk lead what to do when a chatbot gets a symptom question.

Vendor promises go unchecked

A tool described as HIPAA-ready still needs a business associate agreement (BAA) and a look at sub-processors. Someone also has to ask whether your data trains the vendor's models.

Human review exists only on paper

A policy saying a person reviews AI output does not say who. It also leaves out how many items, with what authority to reject, and what gets logged.

Incidents have no path

When a tool sends the wrong message to a patient, staff do not know whom to tell. The report waits for someone to decide who owns it.

Context

Why this is harder in healthcare, and where it opens up

What makes it hard

Rules differ by state, service line and country. California AB 3030 and Texas TRAIGA both require disclosure, in different situations and from different dates.
Several items are proposals or drafts. The proposed HIPAA Security Rule overhaul (NPRM) of 6 January 2025 is not final, and the ICO's draft automated decision-making guidance of 31 March 2026 is not law.
Law moves. Colorado's SB 26-189 repealed and replaced SB 24-205 before it started, so a program built to the old law targets a framework that never took effect.
NIST and ISO frameworks are voluntary and generic. Neither says whether a particular clinical output is safe.
Clinical, privacy, security, legal and commercial leaders each own a piece. Without one owner of the register, the pieces do not connect.

Where the opportunity is

A register lets leaders say yes faster, because low-risk uses are pre-approved and high-risk ones get the right review.
Buyers and partners can ask for governance evidence in security reviews, and having it ready makes those conversations easier.
BAA, data processing, training-data and sub-processor terms become consistent across vendors instead of negotiated from scratch.
Staff get clear rules, which aims to reduce both unsanctioned tool use and unnecessary caution.

How it works

The mechanics, in order

01
Inventory
We catalog every AI use, including features inside existing software and tools staff use outside the approved list. Each entry records purpose, data, users, vendor and owner.
02
Tier by risk and jurisdiction
Each use is tiered by what it touches and what a mistake costs: internal drafting without patient data, patient-facing messages, or anything near clinical, coverage, hiring or mental health decisions.
03
Set policy and decision rights
Short role-based rules say what is allowed, who approves new uses and what must be disclosed, written for the front desk lead and billing manager, not for a lawyer.
04
Check vendors and contracts
Due diligence covers data use, retention, sub-processors, hosting location, security evidence and human-oversight features, checked against the use before purchase.
05
Monitor and keep evidence
Sampled output review, override logs, complaint tracking and incident drills, with a scheduled re-review of each tier, recorded in a short file you can show a buyer, board or regulator.

Human + AI

What AI does, and what people own

AI can speed up the paperwork of governance, but accountability cannot be delegated to it. Leaders accept the risk, and the people closest to each workflow decide whether a control is workable.

AI handles

Draft register entries from tool lists, invoices and system logs for the owner to confirm
Extract data-use, retention, sub-processor and hosting terms from vendor contracts
Summarize new rules, guidance and enforcement notices for the compliance officer to assess
Create first drafts of data protection impact assessments (DPIAs), risk assessments and role playbooks
Sample outputs and flag patterns such as repeated corrections for a reviewer

People decide and review

Executives set risk appetite and sign off on high-risk uses
The compliance or privacy officer owns policy, DPIAs and incident decisions
The medical director decides whether a use touches clinical judgment and what review it needs
Counsel advises on legal interpretation and contract terms
Front-line reviewers decide on each output and have authority to reject it

What we build

What is included

AI use inventory and register

A living list of every AI use with owner, purpose, data, vendor and tier.

Risk tiering

Criteria that sort uses by data sensitivity, patient or customer impact and degree of human review.

Policies and role playbooks

One-page rules for front desk, billing, marketing, clinical and executive roles. Teaching them is covered under AI training.

Vendor due diligence

Checklists covering BAAs, UK data processing terms, hosting, sub-processors and use of your data for model training.

DPIAs and risk assessments

A DPIA is a UK GDPR step to perform when required. We prepare drafts for your privacy officer and support the US HIPAA risk analysis.

Monitoring and incident handling

Sampling plans, drift checks, complaint routes and an incident playbook with named owners.

Use cases

Examples from healthcare operations

Illustrations of how this work looks in practice. They describe workflows, not client results.

Ambient scribe procurement checklist

NHS England's scribing guidance lists what to address before deployment: patient information, legal basis, DPIA, supplier assurance, clinical safety, clinician review, security and monitoring. US clinics get a HIPAA risk analysis and BAA review instead.

Medical Clinics

Utilization review controls

California SB 1120 (1 January 2025) requires a licensed professional, not a tool, to make medical-necessity determinations. We document how software supports reviewers at health plans and administrators, where CMS-0057-F also sets decision timeframes for affected payers.

Health Insurance

State-by-state map for administrative AI

Illinois (1 August 2025), Nevada (1 July 2025) and Colorado (HB 26-1195, 12 August 2026) each limit AI in therapy, and Illinois and Colorado add consent rules for recording. We turn them into a one-page rule for administrative AI at mental health organizations.

Mental Health

Evidence pack for buyer reviews

Customer security teams ask how you govern the AI inside your product. One pack covering inventory, testing, change control and incident handling, plus NHS DTAC and DCB0129 clinical safety evidence, answers them once.

Digital Health

Review of AI candidate matching

Test for adverse impact, since the US Equal Employment Opportunity Commission (EEOC) says Title VII applies to algorithmic selection. In the UK, ICO advises a DPIA during procurement, which matters to healthcare staffing agencies.

Healthcare Staffing

Committee intake form

A one-page form for any department proposing an AI tool: purpose, data, owner, vendor documents, review design and success measures.

Hospitals & Systems

Growth link

Governance as a sales and expansion asset

For providers, good governance lets low-risk tools clear quickly. For vendors, billing firms and consultancies, it answers the questions buyers ask in security reviews. It also protects revenue, since one reportable incident can damage a contract.

Pre-approved low-risk uses move from idea to use without waiting for a committee
A ready evidence pack supports security questionnaires and procurement conversations
Consistent contract terms cut renegotiation with each new vendor
Disclosure language for patient-facing AI, such as California AB 3030 and Texas TRAIGA, is drafted once and reused across AI agents

Our approach

How an engagement runs

Phase 1

Scope with a small working group

A compliance officer, an operations lead, IT and a clinical lead agree scope and who decides.

Phase 2

Inventory and tier

We build the register, including tools used outside the approved list, and assign tiers with owners.

Phase 3

Pilot the controls on two live uses

Policies and review designs are tested on real workflows before wider rollout.

Phase 4

Vendor and contract pass

Existing vendors are checked against the standard set, and gaps become renewal asks.

Phase 5

Hand over the operating rhythm

Your team takes over the review calendar, sampling and incident process.

Outcomes

What usually improves

A current register

Leaders can see which AI is in use, who owns it and how risky it is.

Faster decisions on low-risk uses

Pre-agreed criteria replace case-by-case debate.

Consistent vendor terms

Contract and security questions are asked the same way every time.

Evidence ready to share

Policies, assessments and monitoring records exist before a buyer or regulator asks.

What this depends on

Governance does not make an AI tool safe or accurate. It makes responsibility, testing and review explicit. Results depend on leadership attention, inventory completeness and whether staff follow the process. No framework or certificate shows that a given clinical output is safe.

Industries

Where it fits best

01MSOs & GroupsOne governance model across sites after acquisitions: a central register and shared playbooks, with local exceptions recorded instead of informal.02Pharma & Life SciencesUnder ABPI Code clause 8.1, AI-assisted review does not transfer responsibility, so a nominated signatory still certifies the material.03TelehealthAd-tech and AI vendors need review together. Texas TRAIGA requires disclosure of AI use in a healthcare service no later than the date service is first provided.04Home & Senior CareIn England, the Care Quality Commission (CQC) does not approve individual AI products. It expects human oversight, staff training, risk management and governance.05Medical Billing & RCMClient contracts and BAAs, human-reviewed coding with audit samples, and a clear answer on how appeal drafts are controlled.06Outsourcing & BPOA contractor handling PHI for a covered entity can be a business associate. HHS gives an AI patient-portal chatbot as an example.

Limits

What to watch out for

The HIPAA Security Rule NPRM (6 January 2025) is not final. The 2026 regulatory agenda lists it under Long-Term Actions with a July 2027 target, not a legal deadline. Existing HIPAA duties, including risk analysis and BAAs, apply now.
Colorado SB 26-189 became law on 14 May 2026 and replaced SB 24-205. Its principal automated decision-making duties begin 1 January 2027. Do not prepare for the original act.
In the UK, the Data (Use and Access) Act 2025 amended UK GDPR rather than replacing it. ICO's final automated decision-making guidance is listed for winter 2026.
NIST AI RMF and ISO/IEC 42001 are voluntary. CHAI and the Joint Commission have described a future voluntary certification, which does not exist yet.
We help build the program and the evidence. We are not a law firm, and legal interpretation for your organization belongs with qualified counsel.

Questions

Frequently asked questions

Do we need AI governance if we only use a scribe and a chatbot?

Yes, but a small program. Two tools still involve patient data, vendor terms, review duties and disclosure rules in some states. A register, two risk assessments and a one-page policy is a proportionate start.

Which framework should we follow?

NIST's AI Risk Management Framework (January 2023) and generative AI profile (26 July 2024) are voluntary. ISO/IEC 42001 (December 2023) is a management system standard. We borrow their structure for owners, testing, change control and incident handling, and pursue certification only if buyers ask.

What is a DPIA and do we need one?

It is a data protection impact assessment under UK GDPR. ICO guidance says to perform one when required, and NHS scribing guidance lists it among pre-deployment steps. The related US exercise is the HIPAA Security Rule risk analysis, which applies now.

Who should own AI governance?

One named executive owner with a small group covering operations, compliance, clinical leadership and IT. In a small practice, that can be the practice manager and medical director meeting on a schedule.

What about staff already using unapproved tools?

We start by asking, not punishing. The inventory records what staff actually use, safe alternatives are approved for those jobs, and blocking is reserved for PHI risks. Habits are then taught through AI training.

How often does the program need updating?

Whenever a new use, vendor change or rule appears, and on a set calendar. ICO's final guidance is due winter 2026, and MHRA plans draft guidance on adaptive AI devices by December 2026.

From the blog

Start with the list of what you already use

Tell us which AI tools your teams use today, including ones nobody approved. We will show how we would tier them and which two to review first.