AI Model Inventory KRIs: How to Spot Governance Drift Across Your AI Use Cases

TL;DR

• AI governance programs are only as good as the inventory they’re built on — and most inventories start drifting within months of their first build
• OCC Bulletin 2026-13 and the Treasury FS AI RMF explicitly require comprehensive model inventories covering vendor AI, not just in-house models
• Governance drift shows up in five measurable ways: incomplete coverage, stale owners, untiered use cases, overdue validations, and shadow AI
• KRIs for inventory governance drift give you leading indicators before an exam surfaces the gap as a finding

Most AI governance programs fail at the foundation, not the framework. The NIST AI RMF documentation is complete. The policy is signed. The escalation path is documented. But the inventory the entire program is built on has forty-three models — and sixty-one are actually running in production.

Nobody made a decision to let the inventory drift. It happened the way most governance decay happens: a product team launched an AI feature without going through the new model intake process. A vendor updated their fraud scoring model and nobody received a change notification. An owner left the company and the inventory field was never reassigned. Over eighteen months, the distance between the documented inventory and the deployed AI estate grew from a few percent to something that would take a week to reconcile.

When the examiner asks “show me your AI model inventory,” they’re not just asking whether you have a list. They’re asking whether the list is accurate. And governance drift KRIs are how you know the answer before they do.

Why Inventories Drift — and Why It Matters Regulatorily

Model inventories are not self-maintaining. They require active ownership, intake processes that catch new use cases before deployment, vendor notification procedures, and a periodic validation cycle that confirms what’s in the inventory matches what’s in production. Most programs build the first version of the inventory and then declare success. The drift starts the next day.

OCC Bulletin 2026-13, issued April 17, 2026 to replace SR 11-7, is explicit: model risk management governance workflows — approval processes, change management procedures, validation scheduling, board reporting — all depend on the inventory as their source of record. If the inventory is incomplete, every downstream governance function is running on bad data.

The Treasury’s Financial Services AI Risk Management Framework (FS AI RMF), released March 2026 with 230 control objectives, includes model inventory requirements under its AI Governance domain. Covered organizations are expected to maintain inventories that include: AI use case description, risk tier, deployment status, owner, applicable regulatory frameworks, validation schedule, and whether the model is in-house or vendor-supplied.

That last element is where most inventories have the largest gaps.

The Vendor AI Inventory Problem

The majority of AI use at most financial institutions is not in-house models built by data science teams. It’s vendor AI: fraud scoring APIs, credit underwriting platforms, customer service chatbots, document processing tools, AML transaction monitoring systems. These tools contain AI models that are updated by the vendor — sometimes with notice, sometimes without.

Governance drift in vendor AI entries is particularly dangerous because:

• Vendors may update underlying models without triggering your change management process
• Risk tiers assigned at vendor onboarding may not reflect capability expansions (a fraud tool that adds AI-powered behavioral profiling changes the risk profile)
• Audit rights and explainability obligations may not be in the original vendor contract

Examiners are now specifically asking whether vendor AI is in the model inventory. If the answer is “we track our in-house models but vendor AI is managed through TPRM,” that’s not an acceptable answer — it’s a gap.

The Six AI Model Inventory KRI Categories

1. Inventory Completeness

The foundational KRI. If your governance program tracks 40 models and 60 are in production, every other metric in your governance report is built on a 33% gap.

Measuring completeness requires a reference source independent of the inventory itself: deployment tooling, code repositories, procurement records, vendor contract lists. The inventory can’t validate itself.

2. Stale Owner Fields

Ownership is the linchpin of AI governance. Every inventory entry should have a named model owner responsible for ongoing monitoring, validation scheduling, issue escalation, and board reporting inputs. When owners leave or change roles, those entries go dark.

Stale owner rate is one of the best early indicators of broader governance drift. Owners who aren’t actively engaging with their inventory entries aren’t running the ongoing monitoring requirements — they’re letting models operate without active oversight.

3. Untiered and Incomplete Risk Tier Assignments

Risk tier determines validation frequency, monitoring requirements, and escalation thresholds. A high-risk model mis-classified as medium because the intake process was rushed — or because nobody updated the tier when the use case expanded — can sit on an inadequate monitoring cadence for years.

Tier reviews should happen whenever a model’s use case, customer exposure, or regulatory applicability changes — not just on an annual calendar cycle. A customer service bot that gains credit-recommendation functionality mid-year should be retested for tier assignment immediately, not at next year’s review.

4. Overdue Validations

OCC Bulletin 2026-13 requires that validation schedules be maintained and followed, with frequency calibrated to risk tier. The inventory KRI here isn’t about validation results — it’s about whether validations are happening at all.

The overdue validation KRI is a leading indicator of the model drift and performance degradation risks that develop between validation cycles. A model that’s two years past its scheduled validation and actively making credit decisions is carrying unquantified risk. The inventory KRI surfaces that before the model produces an adverse outcome.

5. Shadow AI Detection Rate

Shadow AI — models and tools deployed outside the governance process — can’t be measured directly in the inventory because it isn’t in the inventory. The proxy KRIs are signals of an active discovery process.

Organizations that find zero new models during a discovery exercise should not treat that as good news. They should ask whether the discovery methodology is actually effective.

6. Vendor AI Change Notification Lag

This KRI measures whether the process for receiving and acting on vendor model updates is functioning. When a vendor updates their underlying AI model, the inventory entry should be reviewed and updated — the risk tier may change, the validation schedule may need adjustment, and relevant business lines may need notification.

If your AI vendor contracts don’t require change notification, this KRI can’t function — you won’t receive the signal. The contract clause coverage metric is itself a governance drift indicator: it tells you whether your onboarding process is building the monitoring hooks you’ll need later.

The Governance Drift Cascade

These KRIs don’t fail in isolation. Stale owner fields lead to missed validation schedules. Missed validation schedules lead to undetected drift. Shadow AI avoids the intake process and therefore has no owner, no risk tier, and no validation schedule. Vendor AI without change notification clauses silently updates without triggering any governance review.

The cascade is what makes governance drift hard to detect with a point-in-time inventory review. You check the inventory, it looks reasonable, and you miss the downstream failures because you weren’t measuring the inputs that predict them.

An AI governance framework for financial services gives you the structure. An AI governance policy with employee and vendor obligations gives you the rules. These KRIs give you the measurement layer that tells you whether the structure and rules are actually functioning.

So What? Before the Examiner Finds the Gap

OCC Bulletin 2026-13 and the Treasury FS AI RMF are explicit about what a defensible AI governance program looks like, and a comprehensive, accurate model inventory is at the foundation of both. If an examiner asks to see your AI model inventory and the inventory doesn’t match what’s running in production, everything downstream — your validation reports, your monitoring dashboards, your bias testing results — is built on a gap you can’t explain away.

Run the inventory completeness ratio today. Count your documented models and count your deployed models from a source that’s independent of the inventory. If those numbers are more than five percentage points apart, you have a governance drift problem regardless of how complete the rest of your AI governance program looks.

Then build the KRI infrastructure that keeps the gap from growing back.

The KRI Library (132 Key Risk Indicators) includes pre-built AI risk KRI templates with thresholds, data sources, and escalation triggers across model drift, bias, hallucination, inventory governance, and vendor AI monitoring. Get started here.

Sources:

• OCC: OCC Issues Updated Model Risk Management Guidance (Bulletin 2026-13)
• NIST AI Risk Management Framework
• The CRI FS AI RMF: What 108 Financial Institutions Agree AI Risk Management Actually Requires
• Lowenstein Sandler: Financial Services AI Risk Management Framework — Operationalizing the 230 Control Objectives
• Qualys: From Shadow Models to Audit-Ready AI Security