2.2. Scoring of indicators with multiple dimensions
Most indicators have a number of separate dimensions, each of which must be assessed separately. The overall score for an indicator is based on the scores for the individual dimensions. The scores for multiple dimensions are combined into the overall score for the indicator using either the Weakest Link (WL) method or the Averaging (AV) method. Each indicator specifies the method to be used.
1. Weakest link method: M1 (WL). This method is used for multidimensional indicators where poor performance on one dimension is likely to undermine the impact of good performance on other dimensions of the same indicator. In other words, this method is applied where there is a “weakest link” in the connected dimensions of the indicator. The steps in determining the aggregate indicator score are as follows:
- Each dimension is initially assessed separately and given a score on the four-point calibration scale.
- The aggregate score for the indicator is the lowest score given for any dimension.
- Where any of the other dimensions score higher, a “+” is added to the indicator score. Note: It is NOT acceptable to choose the score for one of the higher-scoring dimensions and add a “-” for any lower scoring dimensions.
2. Averaging method: M2 (AV). The aggregate indicator score awarded using this method is based on an approximate average of the scores for the individual dimensions of an indicator, as specified in a conversion table (table 1). Use of this method is prescribed for selected multidimensional indicators where a low score on one dimension of the indicator does not necessarily undermine the impact of a high score on another dimension of the same indicator. Though all dimensions of an indicator fall within the same area of the PFM system, in certain areas progress on some individual dimensions can be independent of the others. The steps in determining the aggregate indicator score are as follows:
- Each dimension is initially assessed separately and given a score on the four-point calibration scale.
- Refer to the conversion table for indicator scores using the averaging method (table 1) and find the appropriate section of the table—that is, whether there are two, three, or four dimensions for the indicator.
- Identify the row in the table that matches the scores for each dimension of the indicator; the ordering of the dimension scores does not matter.
- Enter the corresponding overall score for the indicator.
The conversion table applies to indicators using M2 (AV) scoring methodology only. Using it for indicators designated for M1 (WL) will result in an incorrect score. The conversion table is intended for use on individual indicators only and is not suitable for aggregating scores across the full set, or subsets, of indicators. No standard methodology has been developed for aggregation across indicators because each indicator measures a different subject and has no standard, quantitative relationship with other indicators.
