AI portfolio arbitration
Apply decision gates before the score
A prohibited, intolerable or unknown condition does not become acceptable because an average is high. It must be resolved before any ranking.
Principle
Separate admissibility from preference
A gate asks whether an option may enter the arbitration. A criterion asks which admissible option is preferred. Mixing the two allows a secondary benefit to offset a limit that was never meant to be compensated.
Specification
Write every gate as an executable control
A general phrase such as “acceptable risk” cannot be tested. The gate names the object, condition, evidence, person qualifying it and authority deciding the effect.
Minimum fields for a decision gate
| Field | Question | Requirement |
|---|---|---|
| Condition | Which fact opens or closes the gate? | Observable state and scope |
| Evidence | Which item supports the conclusion? | Source, date, version and limitation |
| Authority | Who may accept or refuse? | Name, role and delegation |
| Effect | What happens if it fails? | Stop, defer or reframe |
| Reopening | What permits reassessment? | Expected evidence and owner |
Condition
- Question
- Which fact opens or closes the gate?
- Requirement
- Observable state and scope
Evidence
- Question
- Which item supports the conclusion?
- Requirement
- Source, date, version and limitation
Authority
- Question
- Who may accept or refuse?
- Requirement
- Name, role and delegation
Effect
- Question
- What happens if it fails?
- Requirement
- Stop, defer or reframe
Reopening
- Question
- What permits reassessment?
- Requirement
- Expected evidence and owner
States
Three states, never pass by default
Missing evidence is not a pass. It preserves uncertainty and prevents the option from receiving a ranking presented as decision-ready.
- Passed
- The condition is met for the examined version and scope
- Unknown
- Evidence is inconclusive; the option remains outside the ranking
- Failed
- The condition is breached; the specified effect is applied
- Expired
- Old evidence or a change requires fresh qualification
Families
Adapt gates to the legal and operating context
The following families structure the examination without creating universal thresholds. Their wording depends on the use, sector, affected people and approved tolerance.
| Family | Example condition | Possible effect |
|---|---|---|
| Admissibility | Prohibited use or undefined purpose | Stop or reframe |
| Impacts | Residual risk outside tolerance | Stop or reduce scope |
| Data | Right to use critical data not established | Defer |
| Operation | No practicable fallback capability | Defer or reframe |
| Resources | Mandatory cost or time constraint not met | Remove from the current frame |
Admissibility
- Example condition
- Prohibited use or undefined purpose
- Possible effect
- Stop or reframe
Impacts
- Example condition
- Residual risk outside tolerance
- Possible effect
- Stop or reduce scope
Data
- Example condition
- Right to use critical data not established
- Possible effect
- Defer
Operation
- Example condition
- No practicable fallback capability
- Possible effect
- Defer or reframe
Resources
- Example condition
- Mandatory cost or time constraint not met
- Possible effect
- Remove from the current frame
Execution
Put every option through the same protocol
Order and evidence level remain the same for every candidate. The log retains initial states, examined evidence and every exception.
Reopening
Reassess on new evidence, never on new pressure
An option returns when the expected evidence exists, the condition is reassessed and the authority confirms the new status. Renaming the option, artificially narrowing the risk or increasing another score does not reopen anything.
Output
Produce a register that explains every exclusion
The register connects option, gate, status, evidence, authority, effect, rationale and reopening condition. A committee can distinguish a preference choice from a legal, ethical, technical or operating impossibility.
Sources and references
- UK Government — practical multi-criteria analysis guidance (new tab)
Value functions, swing weighting, decision conferencing, sensitivity and ranking robustness.
- UK Government — Green Book 2026 (new tab)
Swing weighting, sensitivity analysis, switching values and the warning against simple weighted scoring.
- NIST AI RMF 1.0 (new tab)
Govern, map, measure and manage the risks of an AI system.
- ISO/IEC 42005:2025 (new tab)
Assessing the impacts of an AI system on individuals, groups and society.
Frequently asked questions
How does a gate differ from a criterion?
A gate establishes whether an option may enter the ranking. A criterion compares options that are already admissible. A failed gate cannot be offset by a better value elsewhere.
What does unknown evidence mean?
The condition is neither satisfied nor disproved. The option remains outside the ranking until the expected evidence is produced and qualified.
Can a gate be reopened?
Yes, on new evidence defined in advance, followed by a fresh qualification and a decision by the competent authority — never through sponsor pressure alone.
Decision gates
What are you currently trying to offset with a high score?
Present the mandatory constraints and available evidence. We separate what can be compared from what must be resolved first.