Login Get a Demo
01 Glossary · Governance and risk

What Is Model Validation?

What validators test, who should do the testing, how often, and how validation differs from red teaming and an AI audit.

The short answer

What is model validation?

Last updated
Read time
4 min read
From
Cranium AI
Share on LinkedIn

Model validation is the evaluation of whether a model performs as expected, including an assessment of its reliability and its limitations, by people independent enough to challenge it.

It examines the reasoning behind a model, compares its output with real outcomes and keeps checking after deployment, so errors and inappropriate uses surface early.

Key takeaways
  • The 2026 US guidance describes three activities: conceptual soundness, outcomes analysis and ongoing monitoring.
  • Validation generally occurs before a model’s first use, then continues as the model and its data change.
  • The PRA expects the validation function to operate independently from model developers and model owners.
  • Vendor models are validated too, even when the code and training data are proprietary.
02 Core checks

What does model validation test?

The April 2026 US guidance describes three activities. SR 11‑7 named the same three in 2011, which is why most validation teams are organized around them.

01

Conceptual soundness

Assessing and documenting the quality of a model’s design and construction, and whether that design suits the purpose the model is used for.

02

Outcomes analysis

Comparing model outputs with the real-world outcomes they were meant to estimate, to see how far and how often the two diverge.

03

Ongoing monitoring

Evaluating whether a model in production still performs as expected. Frequency and scope depend on the nature of the model, new data or methods, and materiality.

03 How it works

How does a validation run?

Depth and timing scale with a model’s purpose, methodology and materiality. The US guidance sets no fixed interval.

  1. Scope by materiality

    Rank the model by purpose and exposure. Higher materiality earns deeper testing and closer monitoring.

  2. Review the design

    Read the documentation, challenge the assumptions and examine the development data.

  3. Test the outputs

    Compare outputs with actual outcomes and probe the conditions where the model is weakest.

  4. Report limitations

    Record errors and limitations, clarify appropriate use and say whether corrective action is warranted.

  5. Keep monitoring

    Track performance in production and validate again when the model, its data or its use changes.

04 Compare

Model validation vs. red teaming vs. AI audit

Three reviews that are often run by different teams and answer different questions.

Model validationAI red teamingAI audit
QuestionDoes the model perform as expected, and where does it fail?Can an adversary make the system misbehave?Does the organization meet the criteria it is held to?
Performed byValidators independent of the model’s developers and ownersSecurity testers, in-house or externalInternal audit or an external body
OutputFindings on errors, limitations and appropriate useAttack findings to fix and retestA report or opinion against stated criteria
Read moreThis pageAI red teamingAI audit
05 Keep going

Related reading.

06 Questions

Frequently asked questions.

Straight answers, kept current. Last updated .

Ask us anything →
Who should perform model validation?

People who can challenge the model without a stake in it. The PRA’s SS1/23 expects the validation function to operate independently from the model development process and from model owners. The 2026 US guidance stresses the rigor of the review over organizational structure, and asks for expertise, enough independence to stay objective and the standing to bring about change.

How often should a model be validated?

The 2026 US guidance sets no fixed interval. It says the timing, nature and frequency of validation vary with a model’s purpose, its methodology, the frequency and scope of model changes and data limitations, and that validation generally occurs before first use. The OCC adds that its guidance does not require community banks to perform annual model validation.

Can you validate a vendor model you cannot see inside?

Yes, within limits. The US guidance accepts that vendors may withhold code, data or methodology, and still expects banks to understand a vendor model’s conceptual soundness, design, development data and performance, and to run ongoing monitoring and outcomes analysis. SS1/23 likewise expects firms to satisfy themselves that vendor models meet their internal validation standards.

Does Cranium AI perform model validation?

No. Cranium AI does not run statistical validation or record effective-challenge sign-off. It produces evidence validators can use: AI‑BOMs generated from code repositories, Arena adversarial test results, model file scan findings and Cranium Guardian runtime records. For generative and agentic AI, which the 2026 US guidance leaves out of scope, that evidence is often where a review starts.

07 See it for yourself

From the definition
to the evidence.

See how Cranium AI discovers, observes, governs, secures and proves every AI model, agent and vendor in your enterprise.