5 Key Contracting Considerations for Digital Health Companies Working with AI Vendors

Artificial Intelligence (AI) is rapidly transforming digital health — from patient engagement to clinical decision-making, the changes are revolutionary. Contracting with AI vendors presents new legal, operational, and compliance risks. Digital health CEOs and legal teams must adapt traditional contracting playbooks to address the realities of AI systems handling sensitive and highly regulated health care data.

To assure optimal results, here are five critical areas for digital health companies to address in the contract negotiation process with potential AI vendors:

1. Define AI Capabilities, Scope, and Performance

Your contract should explicitly:

• Describe what the AI tool does, its limitations, integration points, and expected outcomes.
• Establish measurable performance standards and incorporate them into service-level agreements.
• Include user acceptance testing and remedies, such as service credits or termination if performance standards are not met. This protects your investment in AI-driven services and aligns vendor accountability with your operational goals.

2. Clarify Data Ownership and Usage Rights

AI thrives on data, so clarity around data ownership, access, and licensing is essential. The contract should state the specific data the vendor can access and use — including whether such data includes protected health information (PHI), other personal information, or operational data — and whether it can be used to train or improve the vendor’s models. Importantly, your contract should ensure that any vendor use of data aligns with HIPAA, state privacy laws, and your internal policies, including restricting reuse of PHI or other sensitive health data for purposes other than the vendor providing the services to your company or other purposes permitted by law. There is much greater flexibility to license access for the vendor to use your de-identified data to train or develop AI models, if the company has the appetite for such data licensing. 

You should also scrutinize broad data licenses. Be careful not to assume liability for how a vendor repurposes your data unless the use case is clearly authorized in the contract.

3. Demand Transparency and Explainability

Regulators and patients expect transparency in AI-driven health care decisions. Require documentation that explains how the AI model works, the logic behind outputs, and what safeguards are in place to mitigate bias and inaccuracies.

Beware of vendors reselling or embedding third-party AI tools without sufficient knowledge or flow-down obligations. The vendor should be able to audit or explain the tools it licenses from third parties if those AI tools are handling your company’s sensitive health care data.

4. Address Liability and Risk Allocation

AI-related liability, especially from errors, hallucinations, or cybersecurity incidents, can have sizable consequences. Ensure the contract includes tailored indemnities and risk allocations based on the data sensitivity and function of the AI tool.

Watch out for vendors who exclude liability for AI-generated content. This may be acceptable for internal tools but not for outputs that reach patients, payors, or regulators. Low-cost tools with high data exposure can pose a disproportionate liability risk, which is especially true if liability caps are tied only to the contract fees. 

5. Plan for Regulatory Compliance and Change

With evolving rules from federal and state privacy regulators, vendors must commit to ongoing compliance with current and future requirements. Contracts should allow flexibility for future changes in law or best practices. This will better help ensure that the AI tools your company relies on will not fall behind the regulatory curve — or worse, expose your company to enforcement risk due to noncompliance or outdated model behavior.

Incorporating this AI Vendor Contracting Checklist into your vendor selection process will help CEOs systematically manage risks, compliance, and innovation opportunities when engaging with AI vendors.

AI Vendor Contracting Checklist:

• Define AI scope, capabilities, and performance expectations.
• Clarify data ownership, access, and privacy obligations.
• Require transparency and explainability of AI processes.
• Set clear liability, risk, and compliance responsibilities.
• Establish terms for updates, adaptability, and exit strategy.

AI solutions in the health care space continue to rapidly evolve. Thus, digital health companies should closely monitor any new developments and continue to take necessary steps towards protecting themselves during the contracting process. If you have any questions on AI and onboarding new AI solutions, please contact any of the authors or any of the Partners or Senior Counsel in Foley’s Cybersecurity and Data Privacy Group or Health Care Practice Group.