Test sensitivity and document the decision

A ranking is sound only when its possible reversals are known, outstanding disagreements remain visible and the final choice has an accountable owner.

Dr Youssef Trardi·Updated 7 September 2026

Reading

Treat the ranking as a hypothesis to be tested

The pivot scenario summarises measurements and preferences. It does not establish that the first-ranked option will remain first when evidence, weights or constraints change. Sensitivity analysis searches for those reversals.

Uncertainties

Do not mix what is unknown with what is disputed

Measurement range does not mean the same thing as preference disagreement. Both are tested, but they retain distinct owners and treatments.

Source	What varies	Treatment
Measurement	Data, population, error or extrapolation	Justified low, central and high values
Appraisal	Different readings of the same evidence	Separate scenarios and reason for disagreement
Preference	Relative value of the swings	Weight sets defended by the authorities
Context	Budget, horizon, baseline or regulation	Named scenario and new version where required

Measurement

What varies: Data, population, error or extrapolation
Treatment: Justified low, central and high values

Appraisal

What varies: Different readings of the same evidence
Treatment: Separate scenarios and reason for disagreement

Preference

What varies: Relative value of the swings
Treatment: Weight sets defended by the authorities

Context

What varies: Budget, horizon, baseline or regulation
Treatment: Named scenario and new version where required

Protocol

Stress-test the ranking in a reproducible order

Each test changes one identified assumption and retains the rest of the model. Plausible variations are documented before their effect is interpreted.

Check dominance — remove options worse on every relevant criterion
Vary one input — isolate the effect of every uncertain value
Vary weights — replay preferences that remain defensible
Calculate the switch — find the smallest change that reverses two ranks
Combine scenarios — test coherent changes in context
Recheck gates — ensure no admissibility has become unknown

Switch

Name the assumption that governs the decision

A switching value expresses the evidential effort or preference change required to reverse the ranking. If it is impossible, the tested assumption is not decisive; if it is close, it becomes the next question to examine.

Verdict

Qualify robustness without decorative language

The verdict states what was tested and what was not. It never claims to cover scenarios absent from the record.

Robust in the tested frame: Same first rank, gates passed and no reversal within selected ranges
Conditional: The rank depends on identified evidence, weight or scenario
Indeterminate: Evidence or preferences do not distinguish the options
Outside ranking: A gate is unknown or failed

Decision

Separate analytical result from the act of deciding

The committee receives the ranking, its conditions and limitations. The authority may select another option, but it must state the reason, consequences and follow-up control.

Result — pivot rank, scenarios, intervals, dominance and gates
Decision — option selected, rejected, deferred or returned for examination
Rationale — accepted trade-off and determining assumptions
Conditions — required evidence, control, budget or milestone
Authority — name, role, date and any dissenting view
Review — event that requires the frame to be replayed

Output

Allow a third party to reconstruct the choice

The record retains inputs, value functions, weights, scripts or calculation files, scenario results, divergences, signed decision and versions. A decimal without this trail is not evidence.

Sources and references

UK Government — practical multi-criteria analysis guidance (new tab)
Value functions, swing weighting, decision conferencing, sensitivity and ranking robustness.
UK Government — Green Book 2026 (new tab)
Swing weighting, sensitivity analysis, switching values and the warning against simple weighted scoring.
NIST AI RMF 1.0 (new tab)
Govern, map, measure and manage the risks of an AI system.
ISO/IEC 42005:2025 (new tab)
Assessing the impacts of an AI system on individuals, groups and society.

Frequently asked questions

What is a switching value?

It is the smallest change in a datum, value function or weight that reverses a ranking or changes the decision.

When is a ranking robust?

When it remains unchanged across the plausible variations actually tested, all gates pass and its limitations are documented.

Can the committee decide against the ranking?

Yes. It must then record the selected option, accepted trade-off, determining assumptions, conditions and the authority taking responsibility for the decision.

Robustness

Which change would reverse your current priority?

Present the ranking, assumptions and disagreements. We identify the switches to test before commitment.

Stress-test the ranking