Key Takeaways
- A SOC 2 report says a vendor operated controls over a described system during a closed period. It says nothing about which models that vendor calls, or what happens to your prompts.
- Third-party AI risk management fails at the seam: tools you approved years ago now embed generative AI, and no procurement event triggered a re-review.
- Three defaults cause most of the damage: provider-side prompt retention, enterprise inputs used for training, and unmonitored subprocessor changes.
- An annual questionnaire assesses a vendor once, then lets it ship AI features unexamined for eleven months.
- Demand evidence, not attestation: dated tenant configuration, a contractual training prohibition, and an adversarial test summary.
Your CRM vendor published a release note in March: AI-assisted summaries, now generally available. Nobody filed a security review, because nobody bought anything. The contract was signed in 2023, the risk assessment was completed in 2023, and the SOC 2 report in your vendor file covers a period that ended before the feature existed. Every opportunity note, call transcript and customer email in that tenant now goes to a model you have never assessed.
SOC 2 and the annual questionnaire are not defective. They answer a question written before your vendors had models inside them. Third-party AI risk management needs a different unit of assessment — the AI feature, not the vendor — and a trigger other than the renewal date.
What a SOC 2 Report Actually Tells You About a Vendor’s AI
A SOC 2 examination reports on controls at a service organization against the trust services criteria the vendor selected. Cranium AI holds a SOC 2 Type 2 report and takes it seriously; a vendor who cannot produce one is telling you something. It is bounded in four ways that matter once AI is in the product.
The scope is written by the vendor. The system description sets the boundary — which product, which environment, which processes. A new AI assistant can sit outside it entirely and the report still comes back clean.
The period is historical. A Type 2 covers a window that already closed, often six to twelve months, reported weeks later. A feature that shipped in month eleven was never tested, and a bridge letter extends the same system description — not new functionality.
The criteria do not ask AI questions. They cover governance, risk assessment, monitoring, logical access, change management, and incident response. None of that requires a vendor to name its model provider, disclose whether that provider retains prompts, or say whether the feature acts on your data rather than only describes it.
Subservice organizations are often carved out. Under the carve-out method, controls at a subprocessor — including the model provider — are excluded from testing. They appear as an assumption, not a tested control.
So the report is clean, every sentence true, and you still cannot say what happens to your data when the AI feature touches it.
What questions should you ask SaaS vendors using embedded generative AI?
Ask about four things: the provider chain, retention and training defaults, isolation and autonomy, and change notification. A standard questionnaire has no row for these.
Model and provider chain
- Which models does this feature call, from which providers, and in which regions do those calls terminate?
- Is that list a contractual commitment, or a product decision you can change without telling us?
Retention and training defaults
- Are prompts, retrieved content, and outputs retained by you — and separately, by the model provider?
- Is zero retention configured on the provider API for our tenant, and can you show us that configuration?
- Do you fine-tune, train, or evaluate any model on customer inputs or outputs? Is that opt-in or opt-out, and at what level — account, tenant, or feature?
Isolation, autonomy, and change
- Does the feature enforce our permission model per user, or read the tenant with a service account?
- Can it take actions — write records, send messages, call external systems — or only generate text?
- Does a successful prompt injection meet the incident definition in our contract?
- Will you notify us before adding a model provider, changing a processing region, or turning on an AI feature by default on our tenant?
Why can’t an annual questionnaire keep up?
The two clocks are not close. Your assessment cadence is annual; your vendor’s AI release cadence is monthly, and the changes that matter to you are never announced as security-relevant. A new model provider is a performance improvement. An AI feature switched on across all tenants is a product launch. A new subprocessor is a line on a web page.
The questionnaire also captures the wrong artifact: what someone believed on the day they answered, usually quoting policy rather than a configuration. Nothing links that answer to the setting it describes, so nothing detects a change. The framework side of this we covered in Applying NIST’s AI RMF to Third-Party AI. The operational fix is to stop treating assessment as an event that happens at purchase.
What Evidence to Demand Instead of Attestations
Replace assertions with artifacts. A vendor who has done the work produces these in a week.
- A per-feature AI inventory or AI Card: models, providers, purpose, data categories, region, retention, and where a human stays in the loop.
- The training prohibition as a contract term in the DPA or an AI addendum — not a line on a trust page that can be edited on a Tuesday.
- The subprocessor list and the notification term: days of notice, and whether you may object.
- Dated configuration evidence: a screenshot or API response showing retention settings on your tenant.
- An adversarial testing summary: dates, coverage across jailbreak, injection and data leakage, severity counts, remediation status, retest date. A sentence claiming they red team their models is not evidence.
Rebuilding Third-Party AI Risk Management
1. Tier by data exposure and autonomy, not by spend
A cheap tool with write access to your ticketing system outranks an expensive dashboard.
2. Trigger on detection, not on the calendar
A new AI provider destination in traffic from an approved tool reaches your SIEM months before the renewal.
3. Re-verify the two defaults that flip
Retention and training use, with dated evidence, at every renewal.
Where Cranium AI Fits
Vendor AI risk is an evidence problem running in both directions — you need it from suppliers, and your customers are about to ask you for the same thing. The AI Trust Loop runs Discover, Observe, Govern, Secure, and Prove:
- Discover The Shadow AI view classifies activity from your connected SIEM into AI Provider Access, AI-Enabled SaaS, AI Web App Usage and Dev App Embedded AI, with events, unique users and data transferred. That is how an unannounced AI feature surfaces before the renewal.
- Observe Guardian records interactions between the AI applications you operate and the LLMs they call — sessions, events, signal results, token and cost — so your own systems have a behavioral record.
- Govern The AI System Manager holds each AI system with an owner and its documentation as the system of record, with compliance scoring against the EU AI Act, NIST AI RMF and ISO/IEC 42001.
- Secure Profiles set every Guardian signal to block, modify or pass, independently on input and output, and Cranium Arena™ tests the models you control adversarially.
- Prove The Cranium AI Card™ is a living, exportable transparency report per AI system, mapped to the EU AI Act, NIST AI RMF and ISO 42001, with a request and respond workflow you run against suppliers. My Vendors aggregates what comes back.
Nobody audits from inside a vendor’s stack, ourselves included. Ask precisely, collect structured evidence, and detect when something moves — that is most of the distance between a questionnaire and an assessment.
Frequently Asked Questions
Is a SOC 2 report useless for assessing a vendor’s AI?
No. It is real evidence of a disciplined control environment. Read the system description to see whether the AI feature is inside the boundary, compare the period end date to the feature launch date, and check whether the model provider is carved out.
The vendor says its model provider does not train on API data. Is that enough?
Usually true, still not sufficient. The vendor chose that provider and can change it, and the default says nothing about what the vendor itself retains.
What if a vendor refuses to answer AI-specific questions?
That is an answer. Record it as an accepted risk with a named owner and an expiry date, restrict the data the tool can reach, and raise it at renewal.
How do we find embedded AI in tools we approved years ago?
AI-related egress in your SIEM, release note monitoring for your top vendors, and your own repositories, where developers wire vendor AI features into internal services before anyone documents them.
See Cranium AI in Action
See how AI Cards and My Vendors turn vendor AI assessment into structured, dated evidence—schedule a personalized demo: cranium.ai/get-a-demo/
