The ICO Audit Framework Applied to a Live AI Deployment

The audit rarely begins with a letter from Wilmslow. It begins internally: a customer-service assistant built on a third-party model has been live for eight months, a prospective client sends a long privacy annexe, and the board asks whether the deployment would survive scrutiny. The DPO opens the ICO's audit framework, finds the artificial intelligence toolkit, and meets a tracker of control measures assuming documentation the project never produced.

Auditing a system already processing personal data is a different exercise from assessing one at design stage. Every gap describes live processing happening now, and each finding carries an implicit question about whether it should continue. That is uncomfortable. It is also why a live audit produces better evidence: you can test what the system does, not what the design document promised.

What the framework is, and what it is not

The ICO's data protection audit framework is a set of toolkits covering distinct areas, among them accountability, security, data sharing, breach management, artificial intelligence and age appropriate design. Each comes with a downloadable tracker of control measures, the risk of not having them, and space for your position. It is a self-assessment instrument: completing it produces no certificate, no ICO opinion and no safe harbour. Nor does using it mean the ICO is coming. The Commissioner audits by consent and can compel an audit through an assessment notice, but most organisations will only ever run these toolkits on themselves. Their value is setting out, in the regulator's own words, what it expects to see.

Keep three ICO products separate. The audit framework toolkit is the assessment scaffold. The AI and data protection risk toolkit is a distinct, risk-statement-based tool organised around the AI lifecycle. The guidance on AI and data protection carries the substantive interpretation. All three get revised, so download current versions rather than reusing an earlier project's copy, particularly while the automated decision-making changes made by the Data (Use and Access) Act 2025 are being commenced through secondary legislation. What is in force for your deployment is something to check, not assume. Note the boundary: this is a UK GDPR and Data Protection Act 2018 instrument, so a clean toolkit says nothing about EU AI Act conformity.

Who is actually responsible

Your organisation is the controller if it decided to deploy the system, chose what it would be used for, and fed it personal data. That holds even where the model and infrastructure belong to someone else. The vendor is normally your processor for the deployment and must act on your documented instructions under an Article 28 contract.

The vendor stops being only a processor the moment it determines a purpose of its own: retaining prompts to improve its model, running its own safety analytics, generating product analytics from your users' inputs. For those purposes it is a controller in its own right, with its own lawful basis and transparency obligations, and the contract should say so rather than pretend everything is processing on instruction. The ICO's published outcomes report on AI tools used in recruitment described exactly this: providers presenting themselves as processors while making decisions only a controller makes. Read the terms of service and the data processing addendum, not the sales deck.

Internally, separate three roles that routinely collapse into one. The system owner, the director who commissioned the tool, owns the risk and the remediation budget. The DPO advises, monitors compliance and is the contact point for the ICO; a DPO cannot own a control and then assure it, and should not have decided the processing they must monitor. Internal audit provides independent assurance, so it cannot be the function that completed the self-assessment.

If the same system is deployed into the EU and falls within a high-risk category, a different split applies. Under the EU AI Act the deployer duties sit on you: human oversight, relevant input data, monitoring operation, log retention, informing affected workers. Conformity assessment, technical documentation and CE marking sit on the provider. A DPIA is not a fundamental rights impact assessment, and neither is a conformity assessment. Much of the detail there still depends on harmonised standards and implementing acts not all finalised.

Working through the toolkit on something already running

Area assessed	What is asked for	Evidence that satisfies	Where it usually fails
Lawful basis	A basis for each purpose, including model improvement	Basis documented per purpose, legitimate interests assessment where relied on, ROPA updated	One basis recorded for "the AI tool" across several distinct purposes
DPIA	Completed before processing began, residual risk accepted by a named person	Dated DPIA, DPO advice, owner sign-off, review trigger	Written after go-live, or never revisited when use expanded
Roles and contracts	Controller and processor analysis, Article 28 terms, transfers	Executed contract, sub-processor list and change notice, transfer mechanism and risk assessment	Terms reserving training rights that contradict processor status
Transparency	What people are told about the AI use	Privacy information naming the use specifically, plus notice at the point of use	A generic reference to "technology" buried in a policy nobody reads
Retention	What enters prompts, what is kept, for how long	Schedule covering prompts, outputs and logs, plus evidence deletion executes	Prompt and output logs absent from the ROPA and schedule
Accuracy and fairness	Performance and differential outcomes monitored in production	Defined measures, monitoring records since go-live, action taken on drift	Treating statistical accuracy as the accuracy principle; vendor benchmarks only
Human review	Whether decisions are solely automated with legal or similarly significant effects	Decision-path analysis, a reviewer with authority to override, sampled overrides	A reviewer who approves everything and has no standing to disagree

Test behaviour, not documents

Sampling is the advantage of auditing live, and the part most self-assessments skip. Pull a randomised set of real interactions from the last quarter and read what the system did. Take a recently closed subject access request and check whether anyone searched the conversation logs. Ask for proof the retention job ran, not that it is configured. Ask the reviewer when they last overrode an output. Findings drawn from production are far harder to argue with.

Where this stops, and when to take advice

A completed toolkit is a management tool: not independent assurance, not a certification, not legal advice. It cannot tell you whether a judgement call was right, only that you recorded one. Escalate rather than resolve in the spreadsheet:

• Special category data, or ordinary data from which health, biometric or similar inferences are drawn.
• Children among the users, which brings the age appropriate design toolkit and the Children's code into scope too.
• Solely automated decisions with legal or similarly significant effects, particularly while the Data (Use and Access) Act 2025 changes are being commenced.
• International transfers requiring a transfer risk assessment, and sub-processor chains you cannot see the end of.
• EU AI Act classification, where the answer turns on standards and implementing acts not yet all in place.
• Any conclusion that live processing is unlawful. That is a question about pausing, notification and privilege, and it belongs with counsel first.

On exposure, keep the figures straight. The UK and EU GDPR maximum reaches up to 4% of global annual turnover; under the EU AI Act, prohibited practices reach up to 7% and most other obligations up to 3%. These are statutory ceilings, rarely the realistic outcome. What actually lands on a mid-size organisation is the pause, the remediation, and the client questionnaire that now needs a different answer.