AI Incident KRIs: Output Errors, Customer Harm, Near Misses, and Remediation Aging

In August 2022, the CFPB fined Oportun—operator of the Hello Digit savings app—$2.7 million and required $68,145 in consumer redress. The violation: Hello Digit’s algorithm repeatedly miscalculated when customers could “safely” transfer money to savings. The miscalculations triggered thousands of overdraft fees. When customers complained, the company failed to honor its promise to cover those fees.

Two years later, in October 2024, the CFPB ordered Apple and Goldman Sachs to pay a combined $89.8 million for failures in the Apple Card program. Among the violations: an automated dispute routing system failed to forward tens of thousands of consumer disputes to Goldman Sachs for investigation. Hundreds of thousands of consumers were affected. Goldman Sachs was banned from launching new credit card products unless it presents an adequate compliance plan.

In both cases, the algorithm didn’t break—it ran exactly as designed. The design had errors, the monitoring didn’t catch the volume of harm, and the remediation response failed. Those are AI incidents. AI incident KRIs are built to surface patterns like these before a regulator does.

TL;DR

• AI incidents are different from general operational incidents—they require specific metrics calibrated to AI failure modes: output errors, customer harm, near misses, and model drift.
• The most undercounted category is near misses. If your program only tracks incidents that reached the customer, you’re missing the leading signal that your controls are barely holding.
• Remediation aging is the KRI most likely to draw examiner attention—a stack of unresolved AI incident findings signals a governance failure, not just a technical one.
• NIST AI RMF, OCC 2026 model risk guidance, EU AI Act Article 73, and the FS AI RMF all require some form of AI incident monitoring and reporting.

Why AI Incidents Need Their Own KRI Category

Most operational incident KRI programs weren’t built for AI. They capture system outages, processing errors, and customer service failures. They’re not designed to detect an AI that produces plausible-sounding but incorrect outputs, or a model whose performance has quietly degraded to the point where its decisions are no longer reliable.

AI failures have a distinct signature:

• They’re often invisible until they accumulate. A model that’s wrong 3% of the time across a million decisions can generate 30,000 errors before anyone notices a pattern.
• They can be hard to attribute. A customer who received a wrong credit decision may not know—or may not connect—the denial to an algorithmic failure.
• They produce near misses that traditional incident tracking systems don’t capture because the bad output was caught before it left the system.
• They require different remediation than a software bug—model retraining, data reprocessing, retroactive review of affected decisions.

Building AI incident KRIs on top of your existing operational incident framework means missing the category entirely. They need their own metrics, their own thresholds, and their own escalation paths.

The 8 AI Incident KRIs

These metrics span the full incident lifecycle: detection, harm measurement, near-miss visibility, containment speed, and remediation health.

KRI 1: AI Output Error Rate

This is the foundational measurement for AI incident monitoring. For each deployed AI system, track the rate of verified errors per 1,000 outputs. “Verified” is the key word—this KRI requires a review methodology that actually catches errors, whether that’s human sampling, automated output testing, complaint attribution, or downstream outcome tracking (e.g., credit decisions later overturned on appeal).

Establish a baseline in the first 60 to 90 days of production. Set amber at 25% above baseline and red at 50% above baseline—or at an absolute threshold if the error type carries direct regulatory consequence (e.g., any systematic underwriting error affecting protected classes should be amber at first identification regardless of rate).

The breakdown by system matters. A rising error rate on your fraud model is different from a rising error rate on your marketing recommendation engine. The former demands faster escalation.

KRI 2: Customer Harm Incidents

A customer harm incident is an AI output that caused actual customer impact: a loan denial that was incorrect and not remediated, an overdraft caused by an algorithmic miscalculation, a chatbot that provided inaccurate information leading to a financial decision the customer wouldn’t have made otherwise.

Track by severity (financial harm, regulatory reporting obligation, complaint filed) and by affected population count. One customer harm incident is amber. Three in a quarter—or any incident that triggers a regulatory reporting requirement—is red.

The Hello Digit enforcement action illustrates why this metric needs its own category. The harm wasn’t a system outage. It was thousands of individual algorithmic miscalculations, each small, that accumulated into an enforcement action. The KRI is designed to catch that pattern before it compounds.

KRI 3: Near-Miss Count

Near misses are events where the AI produced a harmful or incorrect output but a control caught it before it reached the customer—a human reviewer flagged a suspicious credit decision, a downstream validation rule blocked an anomalous output, a quality check process identified an error before delivery.

Most organizations don’t track near misses because they don’t result in customer harm. That’s exactly why this is a leading indicator. A healthy near-miss count suggests your safeguards are working. A declining near-miss count can mean one of two things: your AI is getting better, or your review layer is catching fewer errors because it’s underfunded, overwhelmed, or being bypassed.

If near-miss count drops significantly quarter over quarter while customer harm incidents hold steady or rise, the near-miss drop is not good news—it’s a signal that your detection layer is breaking down.

KRI 4: Human Review Bypass Rate

Many AI governance programs require human review of certain AI outputs before they reach the customer: high-stakes credit decisions, adverse actions, chatbot responses involving sensitive topics. The bypass rate measures how often that review requirement is skipped in practice.

Bypass often happens for operational reasons—review queues back up, workarounds develop, the policy isn’t enforced consistently. A 3% bypass rate might be operationally understandable. A 12% bypass rate means your control isn’t functioning.

Review this KRI weekly for high-risk applications. The data source is your production system logs cross-referenced against your review workflow records.

KRI 5: Time to Contain

From when an AI incident is detected to when it’s contained—meaning the erroneous output is no longer reaching customers and affected decisions are under review—how long does it take?

Containment speed matters because AI incidents can compound quickly. A model producing systematic errors in real time is generating more affected customers with every passing hour. Twenty-four hours is the amber threshold for high-risk applications; seventy-two hours is red.

Track time to contain separately from time to full remediation. Containment is “the bleeding has stopped.” Full remediation is “we’ve reviewed all affected cases, corrected errors, and updated the model.” Both matter; they require different KRI tracks.

KRI 6: Remediation Aging

This is the KRI most likely to draw examiner attention. The count of open AI incident remediation items that have exceeded their target completion date—particularly items more than 30 days past target—signals a governance failure.

A healthy AI incident program generates findings, assigns remediation owners, sets timelines, and closes items with evidence. An unhealthy one generates findings, assigns owners who don’t have time to fix them, and accumulates a growing stack of overdue items that appear in every risk committee report but never close.

The threshold logic: three items more than 30 days overdue is amber. One item more than 60 days overdue without an approved extension and documented rationale is red. The extension and rationale requirement is important—it forces accountability rather than passive aging.

KRI 7: Repeat Root Cause Count

If your AI incident program is working, each incident generates a root cause analysis and a corrective action that prevents recurrence. If it’s not working, you see the same root cause—insufficient output validation, inadequate training data, human review bottleneck—appear in multiple incident reports across consecutive quarters.

Track root cause categories and flag any that appear in more than one incident per quarter, or that have appeared in two consecutive quarters. Repeat root cause is a signal that your corrective actions aren’t actually correcting anything.

KRI 8: Unresolved High-Severity Incident Count

The count of open severity-1 (immediate customer harm or regulatory reporting obligation) and severity-2 (significant operational impact, no immediate customer harm) incidents at any given time. A high-severity AI incident that has been open for more than 30 days without resolution is amber; more than 60 days or an escalation to a regulator is red.

This KRI provides the board-level view. Most committee members don’t need to know about every AI incident. They need to know whether any high-severity incident is in unacceptable aging territory.

The Regulatory Mandate for AI Incident Monitoring

The requirement to monitor AI incidents is no longer implicit.

OCC Bulletin 2026-13 (issued April 17, 2026, jointly with the Federal Reserve and FDIC) is the revised interagency model risk management guidance that replaced SR 11-7. It requires ongoing monitoring of model performance and adverse outcomes, with documentation requirements that apply to AI systems. An AI that produces systematic errors and whose errors weren’t monitored is a model risk management finding. One important note: the 2026 guidance explicitly excludes generative AI and agentic AI from scope—the agencies have signaled a forthcoming separate RFI for those systems. For institutions using GenAI or agentic AI in production, this creates a governance gap. Until dedicated guidance arrives, AI incident KRIs are the primary mechanism for documenting that you’re managing what the interagency guidance doesn’t yet cover.

The NIST AI RMF 1.0 MANAGE function explicitly includes “near-misses” as a required monitoring signal—the actual NIST playbook language requires maintaining “a database of reported errors, near-misses, incidents and negative impacts” and sharing information about near-misses with incident databases and stakeholders. NIST published AI 800-4 in March 2026, the first federal report focused specifically on post-deployment AI monitoring challenges, including the lack of standardized guidance on AI incident sharing across organizations.

EU AI Act Article 73 requires providers of high-risk AI systems to report serious incidents to national market surveillance authorities within 15 days (standard), 2 days (widespread/serious), or 10 days (involving death). The Digital Omnibus agreement (May 2026) extended compliance deadlines—standalone high-risk AI systems now have until December 2, 2027—but the incident reporting obligations apply from the moment a covered system is deployed.

The U.S. Treasury’s FS AI RMF (February 2026) includes incident management across its 230 control objectives. The Colorado AI Act (effective January 1, 2027) requires deployers of high-risk AI to conduct annual bias reviews and mandates developer disclosure of known risks within 90 days of discovery.

Near Misses Are the Most Undercounted Category

Nearly every AI governance program tracks customer harm incidents. Very few track near misses systematically.

This is backwards from a risk management perspective. Near misses are the signal that arrives before the harm. The review process that caught the bad credit decision before it was delivered to the customer is the same process that will catch the next one—if it’s adequately staffed, consistently applied, and measured.

Near misses also tell you where your AI is failing most often. A model that generates 200 near misses per month in one output category is showing you exactly where to focus validation and retraining resources. That’s information a customer harm metric can’t give you because customer harm only counts what got through.

Building a near-miss tracking system requires defining what counts as a near miss for each AI application. For a credit decisioning model: any output flagged by a human reviewer as potentially erroneous before delivery, any decision that triggered an exception process, any case where the model output diverged significantly from manual review. For a chatbot: any output that triggered a human escalation flag, any response that was modified before delivery, any session where the AI’s answer was overridden.

So What?

AI incident KRIs are harder to build than most KRI categories because the data sources aren’t always clean. Production AI systems don’t always have error logging built in. Human review processes don’t always generate audit trails. Near misses don’t always get documented.

The most common failure mode isn’t the absence of data—it’s the absence of accountability. Someone needs to own each AI incident KRI: the data source, the threshold calibration, the escalation path, and the reporting cadence. Without that, the metrics exist on paper but don’t function as a monitoring program.

For teams building out AI KRI monitoring, the companion resources that connect to this framework:

• AI Risk KRIs: Drift, Bias, Hallucination, and Override — the broader AI risk KRI framework this sits within
• Generative AI KRIs: Hallucination Rates, Sensitive Data Exposure, and Escalation Failures — GenAI-specific metrics
• AI Model Inventory KRIs: How to Spot Governance Drift — tracking whether the governance program itself is functioning

If you’re building a comprehensive KRI library that spans operational, AI, vendor, and compliance domains, the KRI Library (132 Key Risk Indicators) includes pre-built AI and compliance KRIs with thresholds, data sources, and escalation triggers calibrated for financial services.

External references:

• CFPB Enforcement Action Against Oportun (Hello Digit), August 2022 — $2.7M penalty for algorithmic overdraft miscalculations
• CFPB Enforcement Action Against Apple and Goldman Sachs, October 2024 — $89.8M for automated dispute system failures affecting hundreds of thousands of consumers
• NIST AI RMF Playbook — MANAGE Function — near-miss monitoring, incident database, and lessons-learned requirements
• OCC Bulletin 2026-13 — Revised Interagency Model Risk Management Guidance — ongoing monitoring requirements (note: GenAI/agentic AI explicitly excluded pending separate guidance)
• EU AI Act Article 73 — Serious Incident Reporting — 15-day/2-day/10-day reporting timelines for high-risk AI providers
• U.S. Treasury FS AI RMF — 230 control objectives including incident management
• NIST AI 800-4 (March 2026) — challenges to monitoring deployed AI systems