Why documenting AI data preparation matters
If your firm trains or fine-tunes a model, record its data sources, preparation and validation choices. For a supplier model, record what the supplier discloses and your own use-case tests.
A reviewer should be able to see which data choices and tests supported the decision to use the system. Keep the record in a form the firm can maintain.
The safest answer is yes, with proportionate records
Record data preparation, assumptions, availability and suitability for the datasets the firm controls. Match the detail to the use case. Where a supplier controls training data, record its disclosures and limitations alongside your own validation and testing.
Make the record understandable without the original builder present, and show where human review occurs. Check which confidentiality, data-protection and sector-specific obligations apply to the proposed use.
Check whether your AI use is audit-ready
What should be documented?
Use the following areas to organise the record:
- Purpose: what the system is meant to do, who will use it and what decisions it may influence.
- Data source: where the training, validation or testing data came from, including any third-party, client, internal or synthetic sources.
- Preparation steps: cleaning, filtering, labelling, deduplication, redaction, sampling and any enrichment applied before use.
- Assumptions and limits: known gaps, weak coverage, excluded groups, old data, proxy variables, edge cases and reasons the data may not reflect live conditions.
- Review evidence: who approved the dataset, what checks were run, what issues were found and what was changed before deployment.
Keep the source chain so a reviewer investigating a questionable output can trace the data choices.
How much evidence is enough?
Choose a record the team can keep current: a register entry, project note, model card, DPIA annex, vendor review or control checklist. Its detail should reflect the proposed use and the evidence available.
For higher-risk uses, consider dataset summaries, sampling and exclusions, quality and bias checks, retention, access and sign-off. Separate the evidence your firm holds from information a supplier has not disclosed, and document tests using your own data and prompts.
Build a governance rhythm for AI controls
Where firms usually go wrong
Write down preparation decisions while the work is being done, including why examples were removed and whether client data was included. Review those choices against the firm's data rules.
Keep supplier documents alongside a record of your own use case, prompts, integrations and review steps. Supplier assurance does not document how staff use the system.
A simple operating model
A team can start with these steps and adapt them to the system:
- Create a short AI use-case register before data is prepared.
- Attach a data preparation note where the firm prepares data; for supplier models, attach available disclosures and local test records.
- Record sensitive data decisions, including redaction, client confidentiality and retention settings.
- Run a small validation set that reflects real work, not only clean examples.
- Require human review and issue logging before the system affects external or regulated work.
Review what the record actually shows before deciding whether the system can be used for the intended work.
Conclusion
Keep a record of the data used, assumptions, tests and the person who approved the use. Note any information the firm could not obtain and how that limit affected the decision.
Turn AI controls into a working implementation plan
Need a safer AI route for your firm?
If this article reflects a live decision in your practice, the useful next step is to map the workflows, confidentiality risks, supplier controls and governance gaps before tools spread informally.
