Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 12 Next »

Assessment scores in HNQIS are calculated using a weighted arithmetic mean of all the answered questions grouped by Composite Indicators (Also referred to as Feedback Orders).

The basic formula for the score in the following section:

The formula is applied for every score and its nested sub-scores. To explain this further let's create the structure of an example H2 Checklist.

In H2 you're able to nest scores inside other scores as well as combine those nested scores. Scoring follows a hierarchy based on how the Assessment is structured, the most important part here is that score formulas are defined from the lowest level of the hierarchy to the top (from the innermost part of the Assessment Tree to the exterior).

Assessment Tree

Name of the structure of the Assessment, it's called Tree because the branching structure resembles a tree.

As shown in the illustration above ‘Example 1 H2’, the first score formulas that will be defined are Score 1.1, Score 1.2, Score 2.1 and Score 3. Also, every score is split into two parts: Numerator and Denominator. This is due to the following reason.

In HNQIS, ‘Scores’ are divided into ‘Sub-scores’, however, ‘Sub-scores’ are not used to calculate the ‘Scores’, for example, scores 1.1 + 1.2 would not define score 1.

‘Scores’ are calculated from the results of the ‘Q' questions within ‘Sub-scores’, not the 'Sub-score’ themselves.

This is the reason why every score is split between Numerator and Denominator so that those values can be re-used in the Scoring Formulas of all the ‘Scores’.

Use of Program Rules vs Program Indicators

All scores are implemented as Program Rules, rather than as Program Indicators.

The rationale is simple: as assessment lists evolved over time, questions are added and removed, and their weight could change. We do not want to change a historical value, which was calculated based on the questions present at the time of a given evaluation. Hence we use Program Rules to store the calculated values at the time of the evaluation. These values are stored on dedicated DEs that are created as part of the list set-up.

Calculating a Score Containing Only Questions

In this example, we use the ‘Scoring Formula’ to obtain the values from Sub-scores 1.1 and 1.2.

To simplify this example, every question will have a Numerator and Denominator with a value of 2.

Here we separate the main ‘Scoring Formula’ to define the Numerator and Denominator formulas.

Numerator and Denominator Explained:

Numerators and Denominators are also separated into two parts: Critical and Non-Critical.

The sum of the Critical Numerator and Non-Critical Numerator equals the Numerator of the score, the same applies to the Denominator of the score.

This separation of Critical and Non-Critical is mainly used for the Assessment Score and the Competency Class calculations, every other score uses the combination of Critical and Non-Critical questions.

The use of this distinction will be further explained in the Calculating the Critical and Non-Critical Scores.

In this example, we will assume that every question is ‘Critical’

Score 1.1 which contains questions Q1, Q2 and Q3 would have the following set of formulas:

Score 1.2 containing questions Q4, Q5, Q6 and Q7 the set of formulas would be the following:

The same process would be repeated for Score 2.1 and Score 3, which also only contain questions.

Calculating a Score Containing Multiple Sub-scores

In order to define the value of Score 1, which contains multiple sub-scores, the Numerators and Denominators of each Sub-score are used instead of calculating the average of each section:

This means that, if a score contains more than two Sub-scores, the formula would make use of the Sub-scores Numerators and Denominators as shown below;

This formula applies even if a Sub-score contains more Sub-scores, there are no limits for the number of Sub-scores.

Calculating the Assessment Score

The Assessment Score (also referred to as Global Score) is determined by the score of the Critical Questions in a checklist.

If a H2 Checklist does not contain at least one Critical Question, the score will be determined by the Non-Critical Questions.

The Assessment Score Formula is calculated in the same way as a Score containing multiple Sub-scores.

Please note you should give consideration is that this formula only includes Critical Questions (while the other ones included both Critical and Non-Critical questions).

Remember that if a checklist contains only Non-Critical Questions, the Assessment Score will be based on those questions instead of the Critical Questions.

The Numerators and Denominators for each score in the example are the following:

Now, using the formula defined in the previous section the Assessment Score of this example ends up being:

The formula for the Assessment Score in this example ended up including all the questions in the checklist due to the assumption made at the beginning of this section. Imagine for a moment that the only Critical Questions in the checklist were Q1, Q2, Q3, Q8 and Q9; in that case, the Assessment Score formula would be the following:

The PCA will automatically manage all the Critical and Non-critical Questions, so you don't have to worry about dealing with those tedious calculations. This is what the last equation looks like for DHIS2:

((((#{_S1Q1}*2+#{_S1Q2}*2+#{_S1Q3}*2)+(#{_S3Q1}*2+#{_S3Q2}*2)))) / 
(((d2:countIfZeroPos('_S1Q1')*2+d2:countIfZeroPos('_S1Q2')*2+d2:countIfZeroPos('_S1Q3')*2)+(d2:countIfZeroPos('_S3Q1')*2+d2:countIfZeroPos('_S3Q2')*2)))

Calculating the Critical and Non-critical Scores

Previously, a special condition was presented to demonstrate the Assessment Score calculation. This condition was to assume that every question in the assessment was Critical, but in most cases, HNQIS Checklists contain a combination of Critical and Non-critical questions.

Define Scores and defined critical scores

The Assessment Score will always be defined by the Critical Score unless there are no Critical Questions in the Assessment. In the case that there are no Critical Questions, the Assessment Score will be defined by the Non-critical Score. Even if the Non-critical score is not always needed for the Assessment Score calculation, it's still used for the Competency Class Calculation.

Now, to define both formulas, the formula for Scores with multiple sub-scores is used as the base:

Then, taking into account that both the Numerator and Denominator can be split into Critical and Non-critical as briefly described previously, the Critical Score and Non-critical Score are defined as follows:

In the Calculating the Assessment Score section, a new example was proposed where questions Q1, Q2, Q3, Q8 and Q9 were Critical and the others were Non-critical. This would result in the following Critical and Non-critical scores:

This surely is confusing, but don't worry, the PCA got you covered! Just keep in mind that all the other scores don't make a distinction between Critical and Non-critical questions, this is only used for the Assessment Score and the Competency Classification.

The key to understand how these calculations work is to notice that all the formulas are basically the same. If a question is Critical, it'll be grouped with the Critical Numerators and Denominators, and if a question is Non-critical, it'll be grouped with the Non-critical Numerators and Denominators. Everything else is still the same.

  • No labels