What AI governance controls should insurers require for third-party AI vendors (audit rights, incident reporting, data use, and model changes)?
An insurer buying AI should agree what the supplier may do with data, which model changes need notice or approval, and how incidents and rollback work. Match contract rights and monitoring to the use case and the firm's obligations.
Book an AI Risk & Efficiency Audit
Below is a governance control set insurers can require contractually and operationally. It focuses on audit rights, incident reporting, data use, and model change control, but it also includes the supporting pieces that make those clauses enforceable in real life.
Start with a simple principle: outsource the capability, not the accountability
Even if a vendor supplies the model, the insurer still owns outcomes. That means you need controls that answer four questions:
- Can we see what the vendor is doing? (auditability, transparency)
- Can we constrain what the vendor is allowed to do? (data use, access, sub-processors)
- Can we detect and respond when things go wrong? (monitoring, incidents, SLAs)
- Can we stop or roll back safely? (change control, kill switch)
Explore Managed AI Governance & Support
1) Review the Proposed Audit Rights and Assurance Evidence
Match supplier evidence to the underwriting, claims or other proposed workflow. Identify what a standard assurance report covers and what additional model, data or operational evidence is needed. Consider the following contract requirements:
- Audit rights: the right to audit, or to have a qualified third party audit, the vendor controls relevant to the service (security, data handling, model governance).
- Assurance evidence to request: the reports, certificates, test summaries and control records relevant to the proposed service. Check issuer, period, scope, exceptions and permitted reliance; do not treat SOC 2 and ISO 27001 as interchangeable assurance labels.
- AI-specific assurance: documented model lifecycle (training, evaluation, release), monitoring approach, and test coverage for known failure modes (bias, drift, adversarial abuse).
If an on-site audit is unavailable, agree the scope and frequency of independent assessment and the right to request targeted evidence after material incidents. Check whether those alternatives meet the firm's applicable obligations.
2) Data use controls (purpose limitation, retention, and training restrictions)
Insurers should be explicit about what data the vendor can use, for what purpose, for how long, and with what safeguards. Include controls such as:
- Purpose limitation: vendor may process insurer data only to provide the contracted service, not to improve unrelated products.
- No training by default: insurer data (including prompts/outputs) must not be used for vendor training or fine-tuning unless explicitly approved in writing per use case.
- Retention limits: clear retention windows for raw inputs, derived features, logs, and outputs, plus deletion obligations on termination.
- Data segregation: tenant isolation, encryption at rest and in transit, and controls preventing cross-client leakage.
- Sub-processor governance: an approved sub-processor list, notification of changes, and the right to object to new sub-processors that increase risk.
For GenAI vendors, add requirements around prompt and output logging: what is logged, what is redacted, who can access logs, and how long logs are kept.
3) Model and prompt change control (visibility, approval paths, and rollback)
A change to the model version, thresholds, prompt template or feature pipeline may affect the workflow. Define which changes require notice, testing or approval, and include the following in the contract discussion:
- Change notification: advance notice for material changes (model version, decision thresholds, feature engineering, prompt templates, retrieval sources).
- Risk-tiered approval: define which changes require insurer approval versus notification only. For example, high-impact decisioning systems should treat model swaps like a production release gate.
- Release notes: what changed, expected impact, test results, and any new limitations.
- Rollback capability: the ability to revert to a prior version quickly, with defined timelines.
- Change freeze windows: optional restrictions around peak operational periods (catastrophe events, year-end).
If rollback or version pinning is unavailable, document the limitation and test the proposed fallback before approving the use. Decide whether the residual risk is acceptable for that workflow.
4) Performance, drift, and bias monitoring (operational KPIs plus risk KPIs)
Governance is not only contractual. You need operational monitoring. Require the vendor to provide, and the insurer to receive, periodic reporting that includes:
- Performance metrics aligned to the use case (accuracy, precision/recall, false positives/negatives, business KPIs like leakage or claims cycle time).
- Drift indicators: input distribution shifts, missingness spikes, schema changes, and model score distribution changes.
- Bias/fairness checks where applicable: clear definitions, cohort monitoring, and mitigation actions.
- Data quality: alerting on out-of-range values, unusual patterns, and upstream feed changes.
Also define who reviews these reports and what thresholds trigger escalation. Name who reviews the signal, who decides the response and how that decision is recorded.
5) Incident reporting and response SLAs (including AI-specific incident types)
Traditional incident clauses often focus on security breaches only. For AI vendors, define incidents broadly and include clear SLAs:
- Security incidents: confirmed or suspected data compromise, unauthorised access, vulnerability exploitation.
- Model incidents: material performance degradation, unacceptable error spikes, confirmed bias harms, or unsafe GenAI outputs that reach customers.
- Integrity incidents: model manipulation, prompt injection leading to policy violations, data poisoning signals.
Contractual requirements:
- Notification timelines: agree severity definitions, the initial notification deadline, update frequency and contact route. Confirm that the supplier timetable supports the insurer's applicable reporting and response obligations; an hours-based deadline or daily update is a contractual choice to assess, not a universal rule.
- Containment actions: immediate steps the vendor must take (disable risky features, revert versions, block abusive inputs).
- RCA and corrective actions: root cause analysis within a defined window, plus a remediation plan and verification evidence.
- Regulatory support: cooperation clauses for regulator inquiries and audit requests.
6) Explainability, documentation, and evidence packs (so you can defend decisions)
Identify the explanation and record requirements that apply to the particular underwriting, claims or fraud use, including the relevant regulator, ombudsman or customer process. Agree supplier evidence that supports those requirements, such as:
- Decision traceability: the ability to show which model/version ran, when, and what inputs were used.
- Reason codes or explanation artefacts appropriate to the use case.
- Model documentation: intended use, limitations, known failure modes, and monitoring expectations.
- Evidence packs for audits: testing summaries, bias assessments where applicable, and change logs.
Even if the vendor resists sharing proprietary details, they should still provide sufficient evidence for governance and accountability.
7) Access control and least privilege (especially for data and model endpoints)
Require controls around who can access data, model outputs, and admin functions:
- Role-based access control with periodic access reviews
- Separation of duties for production changes
- Strong authentication and logging
- Restrictions on vendor support access, with time-bounded, approved sessions
8) Exit and continuity (because vendors change, and you need leverage)
Governance includes getting out safely. Require:
- Data return and deletion obligations with verification
- Model portability where feasible (export formats, feature definitions, documentation)
- Business continuity: plans for downtime and degraded modes
- Transition support: timeboxed assistance if you switch vendors or bring capability in-house
A practical checklist insurers can use in procurement
- Do we have audit rights (or an acceptable independent assurance alternative)?
- Is data use limited to the service, with no training on our data by default?
- Are retention, deletion, and segregation explicitly defined?
- Do we get advance notice and approval rights for material model/prompt changes?
- Do we receive monitoring reports (performance, drift, and fairness where relevant)?
- Are AI model incidents included in incident definitions and SLAs?
- Can we trace decisions by model version and produce evidence for audits?
- Do we have rollback, kill switch, and exit provisions?
Conclusion
Before signing, test the proposed terms against a material model change and an incident scenario. Check who is notified, what evidence is available, who may pause the use and which fallback has been tested. Record any rights the vendor cannot provide and the decision on those gaps.
If you want a pragmatic vendor control pack (AI clauses, evidence requirements, and an operating cadence for monitoring and change approvals) tailored to your underwriting/claims use cases, book an AI Clarity Consultation. Bring the supplier's proposed terms, a change example and an incident scenario to discuss the support needed.
Get Implementation Support
Want to find the practical AI opportunity?
If this topic is part of a real workflow decision, we can help you identify where AI will save time, where it creates risk, and what needs governance before rollout.
