Conceptual soundness
Assessing and documenting the quality of a model’s design and construction, and whether that design suits the purpose the model is used for.
What validators test, who should do the testing, how often, and how validation differs from red teaming and an AI audit.
Model validation is the evaluation of whether a model performs as expected, including an assessment of its reliability and its limitations, by people independent enough to challenge it.
It examines the reasoning behind a model, compares its output with real outcomes and keeps checking after deployment, so errors and inappropriate uses surface early.
The April 2026 US guidance describes three activities. SR 11‑7 named the same three in 2011, which is why most validation teams are organized around them.
Assessing and documenting the quality of a model’s design and construction, and whether that design suits the purpose the model is used for.
Comparing model outputs with the real-world outcomes they were meant to estimate, to see how far and how often the two diverge.
Evaluating whether a model in production still performs as expected. Frequency and scope depend on the nature of the model, new data or methods, and materiality.
Depth and timing scale with a model’s purpose, methodology and materiality. The US guidance sets no fixed interval.
Rank the model by purpose and exposure. Higher materiality earns deeper testing and closer monitoring.
Read the documentation, challenge the assumptions and examine the development data.
Compare outputs with actual outcomes and probe the conditions where the model is weakest.
Record errors and limitations, clarify appropriate use and say whether corrective action is warranted.
Track performance in production and validate again when the model, its data or its use changes.
Three reviews that are often run by different teams and answer different questions.
| Model validation | AI red teaming | AI audit | |
|---|---|---|---|
| Question | Does the model perform as expected, and where does it fail? | Can an adversary make the system misbehave? | Does the organization meet the criteria it is held to? |
| Performed by | Validators independent of the model’s developers and owners | Security testers, in-house or external | Internal audit or an external body |
| Output | Findings on errors, limitations and appropriate use | Attack findings to fix and retest | A report or opinion against stated criteria |
| Read more | This page | AI red teaming | AI audit |
Straight answers, kept current. Last updated .
Ask us anything →People who can challenge the model without a stake in it. The PRA’s SS1/23 expects the validation function to operate independently from the model development process and from model owners. The 2026 US guidance stresses the rigor of the review over organizational structure, and asks for expertise, enough independence to stay objective and the standing to bring about change.
The 2026 US guidance sets no fixed interval. It says the timing, nature and frequency of validation vary with a model’s purpose, its methodology, the frequency and scope of model changes and data limitations, and that validation generally occurs before first use. The OCC adds that its guidance does not require community banks to perform annual model validation.
Yes, within limits. The US guidance accepts that vendors may withhold code, data or methodology, and still expects banks to understand a vendor model’s conceptual soundness, design, development data and performance, and to run ongoing monitoring and outcomes analysis. SS1/23 likewise expects firms to satisfy themselves that vendor models meet their internal validation standards.
No. Cranium AI does not run statistical validation or record effective-challenge sign-off. It produces evidence validators can use: AI‑BOMs generated from code repositories, Arena adversarial test results, model file scan findings and Cranium Guardian runtime records. For generative and agentic AI, which the 2026 US guidance leaves out of scope, that evidence is often where a review starts.
Primary sources for the figures and frameworks cited on this page. Last verified .
See how Cranium AI discovers, observes, governs, secures and proves every AI model, agent and vendor in your enterprise.
Cookie Policy · Privacy Policy
Global Privacy Control detected. Opt-out request honored: marketing tags stay off in this browser.