BIA Quality Assurance: Challenge Outliers and Calibrate Scores Across Business Units

TL;DR

• Business impact analysis quality assurance happens after collection and before approval. It tests whether business-unit answers are complete, comparable, evidenced, and operationally possible.
• Flag outliers; do not auto-correct them. Every challenge needs a question, evidence, owner response, disposition, and reviewer.
• Calibrate the assumptions behind scores—time horizons, units, customer-impact logic, peak periods, and dependency scope—before debating the final color or tier.
• Reconcile desired RTOs and RPOs with actual systems, vendors, staffing, and test results. A capability mismatch is a governance decision, not a spreadsheet cleanup.

Your BIA workbook has 86 processes and 86 confident answers. Forty-three are “critical.” Twelve need a 15-minute recovery time. Every department says its busiest day is the worst possible day for an outage.

That is not a finished analysis. It is unchallenged testimony.

Business impact analysis quality assurance turns business-unit submissions into one defensible enterprise view. The QA reviewer looks for missing scope, incomparable assumptions, score compression, unsupported loss estimates, impossible recovery objectives, and dependencies that contradict each other. Then the reviewer preserves how each challenge was resolved.

This is distinct from writing a BIA. Existing guidance explains what the analysis should produce. QA asks whether the collected answers can be trusted together.

What authoritative guidance gives you—and what it leaves to your method

The 2019 FFIEC Business Continuity Management booklet places the BIA inside the BCM risk-management process. The Federal Reserve issued it through SR 19-13 on November 14, 2019, and the OCC announced it in Bulletin 2019-57. Those sources establish the supervisory artifact and its enterprise context; they do not hand you a cross-business-unit outlier query.

NIST SP 800-34 Rev. 1, section 3.2 is more explicit about the analytical chain. It says the BIA characterizes components, supported business processes, and interdependencies; correlates the system with critical processes; and sets contingency requirements and priorities. It describes three steps: determine processes and recovery criticality, identify resource requirements, and identify recovery priorities for system resources. Section 3.2.1 also says the coordinator should work with management and internal and external contacts to identify and validate the processes supported by the system.

That validation obligation is where a practical QA protocol belongs.

Freeze the submitted population before reviewing it

Do not run QA on a workbook that business units are still editing. Create a submission snapshot and record:

• BIA cycle and version
• functions expected, received, excluded, and overdue
• process owner and business-unit approver
• data-collection method and submission date
• applicable impact horizons
• scoring methodology version
• evidence links
• prior-cycle function ID and score

Then reconcile the population against sources outside the BIA: organization chart, process inventory, application inventory, critical-vendor register, product list, regulatory obligations, and prior BIA. This catches the most dangerous omission—the function nobody submitted.

A practical completeness test is bidirectional:

1. Every BIA function maps to a current owner and at least one product, obligation, customer service, or internal service.
2. Every critical product, regulated obligation, critical application, and critical third party maps back to at least one BIA function.

The second test finds orphan dependencies. If the vendor register labels a payment processor critical but no BIA function depends on it, either the vendor tier is wrong or the BIA is incomplete.

For dependency structure, use the critical-service dependency crosswalk rather than expecting one free-text “dependencies” cell to carry the analysis.

Run seven outlier tests before the calibration meeting

Outlier rules are screening devices. They should generate review rows, not automatic score changes.

Avoid universal percentage cutoffs. A flag threshold that works for a 300-function bank may be noisy for a fintech with 18 functions. Start by showing distributions, medians, extremes, duplicates, and changes from the prior cycle. Calibrate formal rules after reviewing one or two cycles of internal history.

An anti-gaming check matters here: reconcile the number of “critical” functions with recovery plans and exercise coverage. If 43 functions are truly top-tier, the recovery strategy, call trees, vendor requirements, and annual exercise program should reflect 43 top-tier commitments. A label unsupported by resources is not conservatism; it is false precision.

Challenge the assumption, not the color

A meeting that asks “should this be red or amber?” usually gets political. Ask about the inputs instead.

Financial impact

Require a unit, time horizon, source, and calculation. “High revenue impact” is not evidence. Better:

Average daily interchange revenue for the affected product, derived from Finance report FIN-22 for the last completed quarter; excludes transactions expected to queue and settle after restoration.

The caveat matters. Gross daily revenue is not automatically loss. Some transactions defer; some abandon; some create remediation expense. QA should distinguish delayed cash flow, permanently lost revenue, direct expense, and potential penalties rather than adding them into one dramatic number.

Customer impact

Ask how many customers are affected at each horizon, what they cannot do, whether funds or essential services are inaccessible, and what manual alternative exists. “Customer-facing” does not establish identical criticality for password reset, account closure, wire release, and marketing preferences.

Regulatory and contractual impact

Name the obligation, deadline, service commitment, or approval condition. A filing function may be deferrable on most days and critical near a deadline. Preserve that time sensitivity instead of forcing one context-free score.

Recovery objectives

Separate business need from current capability:

• required RTO/RPO based on impact;
• currently achievable recovery based on architecture, staffing, contracts, and tests;
• gap between the two;
• interim workaround or risk acceptance;
• funded action and accountable owner.

The RTO and RPO guide explains the objectives themselves. QA adds the challenge record proving they were not accepted on assertion.

Calibrate comparable functions across business units

Calibration is not forcing the same score on every similar-looking process. It is making sure teams used the same language and evidence standard.

Build peer groups before the meeting:

• payment initiation, processing, settlement, and reconciliation;
• customer access and servicing;
• fraud, sanctions, transaction monitoring, and case operations;
• regulatory reporting and finance close;
• identity, access, and security operations;
• employee and internal support functions.

For each peer group, compare impact onset, peak-period behavior, manual workarounds, customer population, legal deadlines, shared systems, shared vendors, minimum staffing, and stated recovery sequence.

Realistic hypothetical: Card dispute intake has a four-hour RTO; ACH returns processing has a 48-hour RTO. That difference may be correct, but “both are Operations” tells you nothing. The challenge record should identify each process’s customer consequence, network or legal timing, backlog behavior, and workaround. Calibration tests the reasoning, not organizational symmetry.

Use this meeting sequence:

1. BCM shows the distribution and flagged records without naming “bad departments.”
2. Process owners explain the underlying facts and evidence.
3. Technology and vendor owners validate capability and dependency claims.
4. Risk challenges inconsistent interpretations of the methodology.
5. The chair records retain, revise, conditionally accept, or escalate.

Sales and Operations will push back if calibration looks like BCM lowering their priority to save money. Keep need and capability separate. A required two-hour RTO can remain valid even when current recovery takes eight hours; the honest output is a six-hour gap requiring investment or accepted risk.

Use four dispositions in the challenge log

Every flagged item should end with one of these outcomes:

A useful challenge note reads:

Field challenged: RTO, Customer Payments. Reason: Two-hour target conflicts with eight-hour recovery demonstrated in DR test DR-2026-04 and vendor SLA. Owner response: Two-hour business need retained due to same-day settlement exposure; manual release covers priority wires only. Disposition: Conditionally accept target; record six-hour capability gap. Technology owns failover remediation by Q2 2027. Operational Risk Committee approved interim risk on August 20, 2026.

That note preserves need, reality, workaround, gap, owner, date, and authority. “RTO confirmed” preserves almost nothing.

Reconcile QA results into the approved BIA

Do not let the challenge log become a side file that disagrees with the final workbook. Before approval:

• apply every approved revision;
• link retained outliers to their evidence;
• create issue IDs for conditional acceptances;
• route escalations to the correct authority;
• rerun population and outlier tests;
• lock the final version;
• retain the pre-QA submission, change history, challenge log, evidence, and approval record.

Then test downstream consistency. Highest-priority functions should appear in recovery plans, technology recovery sequencing, critical-vendor requirements, exercises, and management reporting. The BIA technology-dependency mapping guide is the right follow-on when process priorities and system recovery order disagree.

A two-week BIA QA operating plan

For a mid-size program, use this as a starting sequence and adjust it to scope:

These are workflow targets, not regulatory deadlines. A smaller institution may finish faster; a complex group may need several rounds. The control is that the QA gate closes before approval, not that every program uses fourteen days.

So What?

Run one query today: list every function whose desired RTO is shorter than the latest demonstrated recovery time of its slowest critical dependency.

That report will usually produce more useful management decisions than another polished heat map. It exposes where the business need, technology reality, vendor commitment, and approved risk do not match.

Business impact analysis quality assurance is successful when those disagreements become visible, owned decisions—with evidence—not when every score looks tidy.

The Business Continuity & Disaster Recovery Kit includes editable BIA, dependency-mapping, recovery, exercise, action-tracking, and board-reporting artifacts that can support this QA workflow after tailoring.