Use of Program Rules vs Program Indicators
All scores are implemented as Program Rules, rather than as Program Indicators.
The rationale is simple: as assessment lists evolved over time, questions are added and removed, and their weight could change. We do not want to change a historical value, which was calculated based on the questions present at the time of a given evaluation. Hence we use Program Rules to store the calculated values at the time of the evaluation. These values are stored on dedicated DEs that are created as part of the list set-up.
Assessment scores in HNQIS are calculated using a weighted arithmetic mean of all the answered questions grouped by Composite Indicators (Also referred to as Feedback Orders).
The basic formula for the score in the following section:
The Assessment Tree illustration below ‘Exmaple 1’ shows the structure of a H2 Checklist, which includes the Scores, Sub-scores and their retrospective questions.
Scoring follows a hierarchy based on how the Assessment is structured, the most important part here is that Score formulas are defined from the lowest level of the hierarchy to the top (from the innermost part of the Assessment Tree to the exterior).
As shown in the illustration above ‘Example 1’, the first 'Score' formulas that will be defined are;
Score 1.1
Score 1.2
Score 2.1
Score 3
Also, every score is split into two parts: Numerator and Denominator.
In HNQIS, ‘Scores’ are divided into ‘Sub-scores’, however, ‘Sub-scores’ are not used to calculate the ‘Scores’, for example, scores 1.1 + 1.2 would not define score 1.
‘Scores’ are calculated from the results of the ‘Q' questions within ‘Sub-scores’, not the 'Sub-score’ themselves.
This is the reason why every score is split between Numerator and Denominator so that those values can be re-used in the Scoring Formulas of all the ‘Scores’.
Calculating a Score Containing Only Questions
In this example, we use the ‘Scoring Formula’ to obtain the values from Sub-scores 1.1 and 1.2.
To simplify this example, every question will have a Numerator and Denominator with a value of 2.
Here we separate the main ‘Scoring Formula’ to define the Numerator and Denominator formulas.
Numerator and Denominator Explained:
Numerators and Denominators are also separated into two parts: Critical and Non-Critical.
The sum of the Critical Numerator and Non-Critical Numerator equals the Numerator of the score, the same applies to the Denominator of the score.
This separation of Critical and Non-Critical is mainly used for the Assessment Score and the Competency Class calculations, every other score uses the combination of Critical and Non-Critical questions.
The use of this distinction will be further explained in the Calculating the Critical and Non-Critical Scores.
In this example, we will assume that all questions are ‘Critical’
Score 1.1 which contains questions Q1, Q2 and Q3 would have the following set of formulas:
Score 1.2 containing questions Q4, Q5, Q6 and Q7 the set of formulas would be the following:
The same process would be repeated for Score 2.1 and Score 3, which also only contain questions.
Calculating a Score Containing Multiple Sub-scores
In order to define the value of Score 1, which contains multiple sub-scores, the Numerators and Denominators of each Sub-score are used instead of calculating the average of each section:
This means that, if a score contains more than two Sub-scores, the formula would make use of the Sub-scores Numerators and Denominators as shown below;
This formula applies even if a Sub-score contains more Sub-scores, there are no limits for the number of Sub-scores.
Calculating the Assessment Score
The Assessment Score (also referred to as Global Score) is determined by the score of the Critical Questions in a checklist.
If a H2 Checklist does not contain at least one Critical Question, the score will be determined by the Non-Critical Questions.
The Assessment Score Formula is calculated in the same way as a Score containing multiple Sub-scores.
Please note you should give consideration that the following formula only includes Critical Questions (while the other ones included both Critical and Non-Critical questions).
Remember that if a checklist contains only Non-Critical Questions, the Assessment Score will be based on those questions instead of the Critical Questions.
The Numerators and Denominators for each score in the example are the following:
Now, using the formula defined in the previous section,the Assessment Score of this example ends up being:
The formula for the Assessment Score in this example ended up including all the questions in the checklist due to the assumption made at the beginning of this section where we stated all questions all ‘Critical’.
As another example, if the only Critical Questions were Q1, Q2, Q3, Q8 and Q9; the Assessment Score formula would be as follows:
Please note the PCA manages all the Critical and Non-Critical Questions Calculations automatically.
Below you see how the final equation looks like for DHIS2:
( ( ( (#{_S1Q1}*2+#{_S1Q2}*2+#{_S1Q3}*2)+ (#{_S3Q1}*2+#{_S3Q2}*2) ) ) ) / (((d2:countIfZeroPos('_S1Q1')*2+d2:countIfZeroPos('_S1Q2')*2+d2:countIfZeroPos('_S1Q3')*2)+(d2:countIfZeroPos('_S3Q1')*2+d2:countIfZeroPos('_S3Q2')*2)))
Calculating the Critical and Non-critical Scores
As previously stated, all questions in our examples are classed as 'Critical, a special condition was presented to demonstrate the Assessment Score calculation. This condition was to assume that every question in the assessment was Critical, but in most cases, HNQIS Checklists contain a combination of Critical and Non-critical questions.
Define Scores and defined critical scores
Please Note: The Assessment Score will always be defined by the Critical Score unless the Assesment does not contain any Critical Questions.
The formula will always return an Assessment Score, even if it is not required for the Assessment, it is required for the Competency Class Calculation. In the event there are no Critical Questions, the formular will fall back to the non-critical scores.
In the event that there are no Critical Questions in an Assessment, the Assessment Score will be defined by the Non-critical Score, even if the Non-critical score is not required for the Assessment Score calculation, it's still required for the Competency Class Calculation.
Now, to define both formulas, the formula for Scores with multiple sub-scores is used as the base:
Then, taking into account that both the Numerator and Denominator can be split into Critical and Non-critical as briefly described previously, the Critical Score and Non-critical Score are defined as follows:
In the Calculating the Assessment Score section, a new example was proposed where questions Q1, Q2, Q3, Q8 and Q9 were Critical and the others were Non-critical. This would result in the following Critical and Non-critical scores:
This surely is confusing, but don't worry, the PCA got you covered! Just keep in mind that all the other scores don't make a distinction between Critical and Non-critical questions, this is only used for the Assessment Score and the Competency Classification.
The key to understand how these calculations work is to notice that all the formulas are basically the same. If a question is Critical, it'll be grouped with the Critical Numerators and Denominators, and if a question is Non-critical, it'll be grouped with the Non-critical Numerators and Denominators. Everything else is still the same.