Disparate Impact Testing Techniques: Statistical Methods Examiners Actually Accept

TL;DR

• OCC Bulletin 2025-16 (July 2025) and FDIC FIL-41-2025 (August 2025) removed disparate impact from federal fair lending exams, and the CFPB’s Regulation B amendment took effect July 21, 2026 — but the Fair Housing Act, state law (New Jersey’s December 2025 rules, state AG cases like Massachusetts v. Earnest), and private litigation still create live exposure
• The four statistical methods regulators and plaintiffs have historically used: Adverse Impact Ratio (four-fifths rule), regression analysis, Fisher’s exact test/standard deviation analysis, and BISG proxy methodology
• Each method answers a different question; no single test is sufficient for a defensible AI model testing program
• Documentation is the exam deliverable — undocumented testing is the same as no testing when an examiner or plaintiff asks

The OCC’s July 2025 bulletin removing disparate impact from its Fair Lending examination booklet generated exactly the reaction compliance teams expected: some departments immediately asked whether they could deprioritize statistical testing. The answer is no — and the reasoning matters more than the conclusion.

Federal de-emphasis of disparate impact doesn’t extinguish the risk. It redistributes it. Massachusetts AG Andrea Joy Campbell settled with Earnest Operations for $2.5 million in July 2025 over AI underwriting models alleged to disparately harm Black, Hispanic, and non-citizen applicants — after the federal rollback had already begun. New Jersey’s Division on Civil Rights adopted disparate impact rules under the Law Against Discrimination in December 2025, expressly covering financial lending. New York’s FAIR Business Practices Act (Chapter 708 of 2025, effective February 2026) extends the state AG’s General Business Law § 349 authority from deceptive practices to unfair and abusive ones — a broader hook for credit-model harms, independent of federal posture.

The statistical testing discipline hasn’t become optional. It’s become more complex — because now you’re managing exposure across jurisdictions with different standards, not just one federal framework. What follows is the practitioner’s guide to the four methods that actually matter and how to document them.

Why “No Federal Disparate Impact” Doesn’t Mean “No Disparate Impact Risk”

Understanding your residual exposure requires mapping it specifically. As of October 2026:

Federal banking agencies: OCC Bulletin 2025-16 (July 14, 2025) removed disparate impact references from the OCC’s Fair Lending booklet and told examiners they “should no longer examine for disparate impact.” The FDIC followed with FIL-41-2025 on August 29, 2025, stating it will evaluate ECOA and FHA discrimination only through evidence of disparate treatment. Both agencies kept disparate treatment reviews, including HMDA analysis.

CFPB / Regulation B: The CFPB’s final Regulation B rule, published April 22, 2026 and effective July 21, 2026, provides that ECOA does not authorize disparate-impact liability (the “effects test”) and deletes the contrary language from § 1002.6(a) and its commentary. The same rule notes creditors remain subject to other laws, such as the FHA and state laws, that may impose disparate-impact liability.

State law: New Jersey’s Division on Civil Rights rules (N.J.A.C. 13:16, effective December 2025) apply disparate impact to financial lending. The Mortgage Bankers Association sued to block them in federal court on September 3, 2026 (D.N.J. No. 2:26-cv-11418) — check the docket before relying on either outcome. Massachusetts has shown it will use state consumer protection law against AI underwriting outcomes. Institutions operating in multiple states are not covered by the federal shift.

Fair Housing Act: The Supreme Court’s Texas Department of Housing v. Inclusive Communities (2015) held that the FHA supports disparate impact claims, which reaches residential mortgage lending. HUD has proposed removing its own discriminatory-effects regulation at 24 CFR 100.500 (January 2026, with a supplemental proposal whose comment period closed October 9, 2026) and leaving the question to the courts. A HUD rule change doesn’t overturn a Supreme Court statutory holding.

Private litigation: The ECOA limitations period is five years from the violation. Disparities building now can be litigated by private plaintiffs under state law or the FHA for years after federal enforcement contracts.

The practical posture: maintain statistical testing capability, but adapt documentation to reflect which legal framework each test is designed to satisfy.

The Four Statistical Methods Examiners and Plaintiffs Use

1. Adverse Impact Ratio (AIR) — The Screening Test

The adverse impact ratio is the starting point for any fair lending statistical review. It answers one narrow question: are two groups being approved at materially different rates?

How to calculate it:

AIR = Approval rate for protected class ÷ Approval rate for comparison group (highest rate)

The four-fifths rule, from the Uniform Guidelines on Employee Selection Procedures (UGESP), treats an AIR below 0.80 (80%) as a signal of adverse impact. This was designed for employment selection, but practitioners and litigants have borrowed the same threshold in credit contexts. Read the fine print the agencies wrote themselves: the Q&As call the rule a “rule of thumb” that is not a legal definition, say differences smaller than 20% can still be adverse impact when they are statistically and practically significant, and warn against assuming adverse impact when numbers are so small that chance could explain the gap.

Worked example: A personal loan model approves 1,800 of 3,000 applicants proxied as non-Hispanic White (60.0%) and 470 of 1,000 applicants proxied as Hispanic (47.0%). AIR = 47.0 ÷ 60.0 = 0.78. That fails the four-fifths screen, so you move to significance testing and regression. Now flip it: 54.0% vs. 60.0% gives AIR = 0.90, which passes — but with groups that large, a two-proportion z-test puts the gap well past 1.96 standard deviations. Under the UGESP logic, a passing AIR doesn’t end the inquiry when volumes are high.

Practical limits: AIR is a screening test, not a finding. A group with a lower credit profile will show a lower approval rate even under a perfectly calibrated model. AIR below 0.80 doesn’t prove discrimination — it triggers the next layer of analysis. Also, AIR is unstable when sample sizes are small; you need at least 30 applicants per group before the ratio is meaningful.

At which decision points to run AIR: Application approval, pricing (APR bracket), product assignment (if multiple products have meaningfully different terms), and adverse action.

2. Regression Analysis — Controlling for Legitimate Factors

Regression analysis answers the question AIR can’t: after controlling for the legitimate underwriting variables, does a protected class characteristic still predict outcomes?

The standard approach for mortgage and consumer credit is logistic regression (binary outcome: approved/denied) or OLS regression (continuous outcome: APR). Include all legitimate underwriting variables the model actually uses:

If a racial or ethnic group coefficient remains statistically significant (p < 0.05) after including all legitimate controls, you have unexplained disparate impact — the model’s structure is producing differential outcomes that can’t be attributed to credit risk factors.

For AI models specifically: The regression approach faces a challenge when the model uses hundreds of variables. Including all model inputs as controls essentially replicates the model’s logic, which can mask disparate impact. Some practitioners use a two-stage approach: run the model on a test dataset, then regress the model’s outputs against protected class status and a limited set of legitimate factors. The residual gap is what you’re measuring.

Examiner expectation: Regression doesn’t disappear with disparate impact. Federal examiners still review for disparate treatment, including HMDA-based pricing and underwriting comparisons, and a protected-class coefficient that survives legitimate controls is exactly the kind of result that prompts a comparative file review. Under either theory, regression is how you show you looked for unexplained disparities and addressed them.

3. Fisher’s Exact Test and Standard Deviation Analysis

When sample sizes are small, Fisher’s Exact test is the statistically appropriate tool for evaluating whether differences in approval rates between groups are statistically significant or likely due to random variation.

Standard deviation (sigma) test: For larger samples, the standard deviation test computes the z-score of the difference in selection rates. A z-score above 1.96 (p < 0.05) indicates the difference is statistically significant at the 95% confidence level. A z-score above 2.0 has historically been treated as a presumptive trigger for further investigation in fair lending contexts.

Fisher’s Exact: Produces a p-value directly from a 2x2 contingency table (protected class vs. comparison group, approved vs. denied). Use when group counts are below 30.

Neither test tells you why the difference exists — only that it’s unlikely to be random. Statistical significance is a necessary precondition for a disparate impact claim, not a sufficient one.

4. BISG Proxy Methodology — When You Don’t Have Race Data

Most credit models don’t contain race or ethnicity as an input field — and shouldn’t. But you can’t test for disparate impact without knowing the demographic composition of your applicant pool.

Bayesian Improved Surname Geocoding (BISG) estimates the probability that an individual belongs to each race/ethnicity category by combining:

• Surname analysis (Census Bureau surname frequency data by race/ethnicity)
• Geocoding (census tract demographic composition from ACS data)

The result is a probability vector for each applicant: e.g., P(White) = 0.72, P(Hispanic) = 0.18, P(Asian) = 0.06. These probabilities are used in weighted regression analysis rather than hard-coded classifications.

What the validation data shows. The CFPB’s 2014 paper Using Publicly Available Information to Proxy for Unidentified Race and Ethnicity tested BISG against mortgage applicants who self-reported race and ethnicity. Correlations between proxy probability and reported group:

BISG limitations to document:

• Surname alone is weak for Black applicants (0.40); combining geography is what makes BISG usable for that group, so geocoding quality matters most there
• Near-zero predictive power for American Indian/Alaska Native and multiracial applicants — say so rather than reporting results for those groups
• Does not distinguish between subgroups within broad categories
• Probabilities vs. thresholds: document whether you used probability weights or assigned applicants above a cutoff, because the choice changes your counts
• The CFPB said in that paper that its supervision and enforcement staff rely on BISG for non-mortgage fair lending analysis, where Regulation B generally bars collecting race data

Alternative: Disclosed data where available. For HMDA-covered loans, you have actual race/ethnicity data for most applicants. Use BISG to fill gaps for applicants who declined to disclose, not as a substitute for disclosed data.

Choosing the Right Method: A Decision Framework

The LDA Obligation: What Happens When Testing Finds Disparity

A less discriminatory alternative (LDA) is an alternative model or decision rule that serves the same business purpose — predicting credit risk — with less adverse impact. Under traditional disparate impact doctrine, demonstrating adverse impact doesn’t automatically create liability; it shifts the burden to the institution to show business necessity. If a plaintiff can then identify an available LDA, liability may follow.

Proactively searching for LDAs during model development — before deployment — is the documented best practice. The search doesn’t require testing every conceivable alternative. It requires documenting what alternatives were considered, why they were rejected on business grounds, and whether the current approach represents the least discriminatory path to serving the business purpose.

A basic LDA analysis log covers:

• The current model’s adverse impact ratios by protected class
• Alternative specifications tested (different variable weightings, thresholds, feature sets)
• The AIR and business performance metrics (accuracy, default prediction) of each alternative
• The rationale for selecting the current model over alternatives with lower adverse impact

This documentation is the difference between a proactive compliance program and a reactive litigation response.

Documenting Testing for Exam Readiness

The question isn’t whether your testing will satisfy a federal examiner today — it’s whether it will satisfy a state examiner or plaintiff tomorrow. Documentation requirements are the same regardless of the reviewing body:

Required artifacts:

1. Testing protocol memo — which groups, which decision points, which methods, which thresholds
2. AIR calculations — by group, by decision point, with sample sizes
3. Regression output tables — model specification, coefficients, p-values, R²
4. BISG methodology documentation — if used, explain the approach, data sources, and limitations
5. Findings memo — what was found, what the finding means, what action was taken
6. LDA search documentation — what alternatives were evaluated and why the current approach was retained
7. Monitoring plan — how often testing repeats, who reviews results, what triggers remediation

The absence of documentation — even when testing was actually conducted — is an independent finding. “We ran the numbers but didn’t write it up” is not a defensible response to a regulatory inquiry or a discovery request.

So What? Actions for Compliance and Model Risk Teams

The federal disparate impact shift creates a false sense of security for teams that conflate “OCC won’t examine for it” with “we have no exposure.” The exposure is real — it’s just more fragmented and harder to predict.

Right now:

• Map your jurisdiction footprint. If you lend in New Jersey, or in states whose AGs have brought algorithmic lending cases (Massachusetts) or expanded unfair-practices authority (New York), you have state-law exposure regardless of federal posture. Residential mortgage lending carries FHA exposure everywhere.
• Update policies and model documentation that still cite § 1002.6(a)‘s pre-July 2026 effects-test language or the old OCC and FDIC exam procedures.
• Confirm which of the four statistical methods your current testing program uses and whether each decision point is covered.
• Check whether your BISG methodology documentation is current and addresses known limitations.

Before next model deployment:

• Run all four methods on the pre-deployment test dataset.
• Conduct and document an LDA search.
• Get the findings memo signed by the model owner and compliance officer before launch.

Ongoing:

• Schedule post-deployment disparity monitoring at least quarterly for high-volume, high-impact models.
• Set re-testing triggers: any model update, training data refresh, or >10% change in approval volume by product.

The statistical work isn’t going away. The regulatory audience for it has just gotten more complicated.

Need a structured pre-deployment AI risk assessment with built-in bias testing documentation? The AI Risk Assessment Template & Guide includes an AI Use Case Inventory with auto-tiering, bias testing scorecard, and worked examples across eight use cases mapped to the current regulatory landscape.

Also see: AI Bias Testing for Fair Lending: Methodologies Every Risk Team Needs, AI and Fair Lending: UDAAP Risk in Algorithmic Decisioning, and AI Risk Assessment Template: Pre-Deployment Checklist for Financial Services.