Skip to main content

Governance

Reading an AI Vendor's SOC 2 Report for What It Leaves Out

A practitioner's guide to reviewing an AI vendor's SOC 2 report: who actually sets the scope, the five passages that carry the information, and the AI governance questions the report was never designed to answer.

Reading an AI Vendor's SOC 2 Report for What It Leaves Out

The request arrives with a deadline attached. A business unit has chosen an AI tool: a drafting assistant for the claims team. Legal has the contract in redline, procurement wants the review closed before quarter end, and the vendor's trust portal produces a SOC 2 Type II report, ninety-odd pages, unqualified opinion. The covering email says the vendor is "SOC 2 certified, so we should be fine."

Your job is not to read it cover to cover. It is to establish what the report was scoped to cover, what it excluded, what it quietly hands back to you, and which of your real questions it was never designed to answer. Most of the material risk in an AI vendor relationship sits in that last category.

Who is responsible for what

Correct the vocabulary first. SOC 2 is an attestation engagement, not a certification scheme, and nobody is "SOC 2 certified".

The vendor's management owns the scope. Management writes the system description, selects which Trust Services Criteria categories apply, describes the controls and signs a written assertion. The service auditor, an independent accounting firm, opines on that assertion. The auditor tests the controls management identified, inside the boundary management drew, and does not widen it or decide which product lines are covered. If the AI feature you are buying is absent from the system description, that is not an audit failure and a better report will not fix it: it is a vendor scoping decision, raised commercially.

Nor does any of it transfer accountability. Under UK and EU GDPR you remain the controller where you determine purposes and means; Article 28(1) obliges you to use only processors providing sufficient guarantees, and Article 5(2) makes you accountable for demonstrating that you assessed them. Under the EU AI Act you are the deployer of a system you put into use under your own authority. You become the provider, a much heavier set of obligations, if you place your name or trade mark on a high-risk system already on the market, substantially modify one, or modify the intended purpose of a system, including a general-purpose AI system, so that it becomes high-risk.

The five passages that carry the information

  • The opinion. Type I addresses control design at a point in time; Type II addresses design and operating effectiveness across a stated period. Note whether it is unqualified, qualified, adverse or a disclaimer, and which categories are named: Security, the common criteria, is the only mandatory one, and Availability, Confidentiality, Processing Integrity and Privacy are often omitted.
  • The period covered, against today's date. A period that ended months ago says nothing about the intervening time. Vendors bridge it with a "bridge letter", a management representation that nothing material changed. The vendor signs it, not the auditor, and no opinion covers it.
  • The system description boundary. Which named product, which editions and tiers, which environments and regions. Read it looking for your SKU and your specific feature, not the company name.
  • Subservice organisations and the method used. Under the carve-out method a subservice organisation's controls are excluded from the description and testing. Foundation model providers and cloud hosts are commonly carved out, so the part of the stack performing inference may sit entirely outside the report.
  • The test results. Read the tests, not the "no exceptions noted" column. Inspecting a policy is not evidence a control operated, and a sample of one across a twelve-month period is thin. Where exceptions appear, read management's response and decide whether it describes a fix or an excuse.

Then find the complementary user entity controls: the controls the report assumes you operate, such as access reviews, SSO and MFA configuration, permission provisioning and log monitoring. They are obligations transferred to you inside a document most organisations file and never revisit. Extract them into your control register with named owners, or the hole in the assurance chain is yours.

Where the AI actually sits in the boundary

AI features are usually newer than the audit period covering them. A vendor with a mature SOC 2 history may have added the assistant six months ago, in a separate service, delivered through a carved-out model provider. The report can be entirely clean and say nothing about the component you are worried about.

Processing Integrity is the criterion people assume covers model quality. It does not. Where selected at all, it addresses whether processing is complete, valid, accurate, timely and authorised as designed, a concept that fits a payments pipeline and a probabilistic language model badly. No Trust Services Criterion asks whether the output is correct.

What the report will not tell you

Question you need answeredWhat the SOC 2 tells youWhere the answer must come from
Are our inputs and outputs used to train or fine-tune models?Nothing. Training use is not a Trust Services Criterion.Product terms and the DPA, in writing, per tier and environment, plus any carved-out model provider's terms.
Which models sit behind the feature, and where do they run?Usually nothing; the model provider is typically carved out.Sub-processor list, hosting region commitments, transfer mechanism and transfer risk assessment.
How long are prompts and outputs retained upstream?Vendor-side retention may be tested; upstream retention, including abuse-monitoring copies, is not.DPA retention terms and the upstream provider's policy; confirm whether zero retention applies to your configuration.
How accurate is it on our data, and has discriminatory impact been assessed?Nothing on either.Vendor evaluation evidence, your own pre-deployment testing on representative data, your DPIA where required, and a fundamental rights impact assessment if you are among the deployers required to complete one.
Can we obtain the logs our own duties require?Logging controls may be tested; the log content and export available to you are not.Contract and technical documentation. Deployers of high-risk systems must retain logs under their control for at least six months, subject to other law.
Is the vendor the AI Act provider, and for what?Nothing.A written statement of role, intended purpose and instructions for use, plus a declaration of conformity where required.

None of this criticises SOC 2. It simply was not built for AI governance questions, and the gap is where your exposure sits: up to 3% of global annual turnover for most EU AI Act obligations, up to 7% for prohibited practices, and up to 4% under UK or EU GDPR.

And if they hold ISO/IEC 42001

Treat it as different evidence, not stronger evidence. The certificate means a certification body has audited the vendor's AI management system: scope, Statement of Applicability, internal audit programme, management review, nonconformity and corrective action. It does not certify a model. Ask for the certificate, read its scope statement, and ask which Annex A controls the Statement of Applicability excluded and why.

Limits, and when to take advice

This is general guidance, not legal advice, and the ground underneath it is moving. The EU AI Act's harmonised standards are still under development, several implementing acts and guidance documents remain outstanding, and the phased application timetable has itself been subject to amendment proposals. Confirm the current position for the obligation you rely on rather than assuming a date you read last year still holds.

Take specialist advice where the classification is contested: whether a system is high-risk, whether your configuration makes you a provider rather than a deployer, or whether an intended purpose has materially changed. Take it too where the deployment touches a regulated route such as medical devices, where transfers rest on a contested mechanism, and where the vendor will not put a training-use position in writing. A SOC 2 report is a useful input to those decisions. It is not one of them.

  • SOC 2
  • Vendor due diligence
  • Third-party risk
  • EU AI Act
  • GDPR
  • ISO 42001

More guides

Start Free AI Compliance Review