AI portfolio arbitration
A grid becomes defensible when every value denotes an observable state, the options are comparable and two readers can reconstruct the same reasoning.
The same option may be useful for learning and unsuitable for industrialisation. Before measurement, the mandate fixes the decision, objective, horizon, scarce resources, affected population and baseline option.
A criterion enters the model only if it describes a useful difference between options. We search for missing effects, overlaps and causal dependencies before writing a scale.
A value function translates performance into the language of the decision. It may be linear, contain a threshold or reflect diminishing returns; its form must be justified, not selected to improve a ranking.
| Element | Control question | Record |
|---|---|---|
| Raw measure | What is observed before transformation? | Unit, source, population and date |
| Low anchor | What is the least desirable state relevant to this choice? | Value 0 and rationale |
| High anchor | What is the most desirable state relevant to this choice? | Value 100 and rationale |
| Shape | Does one raw unit create the same value everywhere? | Function, breakpoints and assumptions |
The additive model assumes that preferences on one criterion remain interpretable regardless of the level taken by the others. If the effect of one criterion structurally depends on another, aggregation can tell a false story.
Two assessors independently appraise calibration options. The comparison improves the scale, clarifies the evidence and separates disagreement about facts from disagreement about preferences.
Swing weighting compares the value of moving from the low to the high anchor on each criterion, over the range actually observed between options. An important-sounding criterion may carry little weight if all candidates are equivalent on it.
| Stage | Decision required | Control |
|---|---|---|
| Reference | Which swing creates the most value? | Assign it 100 comparison points |
| Comparison | What is every other swing worth against the first? | Document pairwise judgements |
| Normalisation | What share does each swing represent? | Weights sum to 1 |
| Scenarios | Which disagreements must remain visible? | Test alternative weight sets separately |
The calibration record assembles the frame, criteria tree, raw measures, value functions, calibration options, initial appraisals, calibration decisions and weight scenarios. Every object carries a version and owner.
Primary sources
Last documentary review: 7 September 2026.
Value functions, swing weighting, decision conferencing, sensitivity and ranking robustness.
Swing weighting, sensitivity analysis, switching values and the warning against simple weighted scoring.
Govern, map, measure and manage the risks of an AI system.
Assessing the impacts of an AI system on individuals, groups and society.
Before you write to us
Because the numbers remain arbitrary until 0, 100 and intermediate states are tied to observable performance and a justified value function.
We connect each criterion to a distinct effect, examine causal dependencies and remove or restructure criteria that describe the same change.
The relative value of moving between the low and high anchors over the range considered. It does not measure the abstract importance of the criterion name.
Criteria and anchors
Present the decision, options and current grid. We identify overlaps, ambiguous anchors and missing evidence.
Review the grid