ATLAS—7THE VALIDATION ROOM
← Back to overview
An intricate titanium model core with an illuminated copper heart
INDEPENDENT THINKING. UNDER PRESSURE.

Every model
has a blind spot.

Find it before it becomes everyone's problem.

A credit model is days from launch. You have 40 hours to investigate it, challenge its creators, and make the call that matters.

Enter the validation room ↗Explore the evidence ↗
FIG. 01 / MODEL COREATLAS–7212 FEATURES · ONE CRITICAL DECISION
AN INTERACTIVE RISK SIMULATIONSCROLL TO INVESTIGATE ↓
05Days to investigate
40hYour entire budget
11Potential defect areas
01Decision with consequences
THE SYSTEM UNDER REVIEW

Trace the decision.

Follow the journey from application data to a life-changing lending decision.

ILLUSTRATIVE REVIEWMonOpening validation file VR-2026-114
01 · The week

The clock is running.
The pressure is real.

The full test plan needs more time than you have, so every run is a choice about where to look. Nine hours a day, five days, and an inbox that does not care about your plan.

Mon12 Oct

Brief and first tests

Interview the developers for leads, then pick the tests most likely to break the model's core claim.

Tue13 Oct

The business leans in

From the Head of Consumer LendingLaunch is booked for Monday. I need a preliminary green light today.
Wed14 Oct

The vendor offers help

From BureauVecSign our NDA and we will share the internal model summary.
Thu15 Oct

The regulator is coming

From the CRO's officeHorizontal review of ML credit models next quarter. Anything to escalate?
Fri16 Oct

Challenge and opinion

The lead developer contests your top three findings. Then you approve, approve with conditions, or send it back.

02 · The evidence

A number is a clue.
Evidence is the answer.

Every test produces a simulated exhibit from the case data. Explore a test, change its reading, and see how the evidence changes. Thresholds are assumptions for this training scenario.

A sculptural landscape of metal data fins and a copper thread
THE ART OF INDEPENDENT CHALLENGE

Find the signal.
Question the certainty.

Every exhibit is a test of judgment.

FIG. 02 / SIGNAL & NOISE

The hard part is the middle band. A breach is easy to call. A watch-band reading could be a mild defect or noise, and the game will not tell you which.

Within tolerance

Clear it and move on, or spend the hours elsewhere.

Watch band

Raise it, clear it, or pay one to three hours for a follow-up that settles it.

Breach

Raise it. The question is how severe, and whether you hold that line on Friday.

False alarms cost you. Every finding that turns out to be noise loses precision points and credibility with the committee.

03 · The cases

A new case.
A different truth.

Each case deals eleven areas into severe defects, mild defects, false alarms and clean results. Some cases have no High-severity defects, so sending everything back is not a strategy.

Case file----
0Severe
0Mild
0False alarms
0Clean

You see the mix. Which area is which is what the week is for.

Play this case →
04 · The challenge

Hold what the evidence supports.

On Friday the lead developer answers your strongest findings. Some replies carry evidence you can check. Some carry only confidence. Decide which one should make you withdraw.

HighMRA-02 · Disparate impact on a protected classAdverse impact ratio 0.84 · watch band
05 · The outcome

Your decision.
Its consequences.

Your findings become a committee memorandum in the format a Model Risk Committee would receive. Then the answer key, and six months of consequences for whatever you missed.

The committee room at dusk, ready for the final decision
THE MODEL RISK COMMITTEE

Your name is
on the decision.

The final memo carries your findings, your limitations, and your opinion.

Take your seat ↗
Model Risk Management · Validation memorandum

ATLAS-7 consumer credit model

To
Model Risk Committee
From
Lead Validator
Date
16 October 2026
File
VR-2026-114 · sample case

Opinion

Return to developer

Findings

  1. High Target leakage from post-decision data

    Evidence: Feature timing review, 3 post-decision features. SR 11-7 conceptual soundness · EU AI Act Art. 10.

  2. High Unstable adverse action reasons

    Evidence: Reason-code stability test, top-reason flip rate 31%. Reg B §1002.9 · EU AI Act Art. 86.

  3. Medium Applicant population has shifted

    Evidence: Population stability index 0.31. SR 11-7 outcomes analysis.

Scope limitations

Not performed within the review window: Calibration by risk decile; Documentation review.

How the 100 points split
50Findings found and rated at the right severity
10Precision. Three points off per false alarm
10Developer challenges answered correctly
20A validation opinion that fits what was really there
10Conduct: independence under pressure, early escalation
Examiner-grade90+
Strong second line75–89
Gaps the regulator will find55–74
Rubber stampbelow 55
06 · The rules

The disciplines behind
the decisions.

The scenario uses model-risk, fair-lending and AI-governance themes. References provide training context; the simulation is not an authoritative statement of current requirements.

Federal Reserve / OCCSR 11-7 · OCC 2011-12

Conceptual soundness, outcomes analysis, ongoing monitoring, implementation and vendor products. The spine of the test plan.

CFPBECOA · Regulation B §1002.9

Principal reasons on adverse action notices. Why unstable reason codes are a High, not a footnote.

Fair lending practiceFour-fifths screen · BISG

Adverse impact ratios, proxy estimation and the search for a less discriminatory alternative.

European UnionAI Act · Annex III 5(b)

Creditworthiness scoring as high-risk AI, with Articles 10, 11, 15 and 86 on data, documentation, accuracy and explanation.

Interagency, 2023Third-party risk guidance

Why a bundled embedding with no development evidence is your problem, not the vendor's.

Industry conventionPSI · AUC · calibration

The stability, discrimination and calibration thresholds every exhibit is read against.

07 · Who it is for
Model validators

Practice effective challenge on a model that fights back, without a production deadline on the line.

MRM and risk leaders

Calibrate a team. Run the same case code across a cohort and compare the memos.

Developers and data scientists

See your model the way second line sees it, and learn which arguments survive a committee.

Audit and examiners

Rehearse what a defensible validation file looks like, and where weak ones break.

The file is open

ATLAS-7 decides 31,000 loans a month from Monday.

Unless you find a reason it should not.

Enter the simulation ↗

Or play the case you dealt above: ----