Routine Data Quality Assurance (RDQA)

Image Banner

PSI Product Owners:

PSI Support:

Reader Audience: M&E technical advisors, DHIS2 system administrators, DHIS2 developers, RDQA end users

Summary: RDQA is a metadata package used in DHIS2 to assess the quality of routine data.

RDQA Space Table of Contents

Introduction

High-quality data is essential for making evidence-based decisions when managing health programs. Assessing and improving overall data quality in a health system goes beyond implementing data collection validations. PSI collects and uses data to monitor and evaluate its programs; estimate the health impact of specific interventions; assess value for money and improve intervention cost-effectiveness; and evaluate the health of the markets PSI works to strengthen. Thus, high-quality data is essential for accurately monitoring and measuring the health impact of our interventions.

High-quality data is dependent on strong and robust monitoring and evaluation systems. Routine data quality assessments are therefore necessary for verifying the quality of the reported data and assessing the underlying data management and reporting systems for key indicators. These assessments are necessary to ensure that collected data is within expected ranges and that the data flows between paper forms, registers, and information systems are timely and accurate.

Routine Data Quality Assessment Tool

PSI employs an approach to assessing the quality of routine monitoring data and their related systems called Routine Data Quality Assessment or RDQA. In 2007, PSI developed the RDQA tool based on a data quality assessment tool developed by MEASURE Evaluation, designed to be used by external agents as an evaluation tool. The MEASURE tool is available as a series of MS Excel templates in the MEASURE Evaluation data quality tools collection.

The PSI RDQA tool is a five-step process designed to be used by internal personnel who are part of the organization to continuously monitor the quality of routine data of a data collection site. For PSI, a 'data collection site' typically means a service delivery provider (clinic), but the tool can be adapted for different contexts.

Objectives of the RDQA

Improved data quality allows for better decision-making at all levels of health systems. At the service level, routine data quality assessments are a critical activity to:

Verify the quality of reported data against source documents.
Identify causes of poor-quality data by assessing the ability of the data management system to collect, manage and report quality data.
Support the development of data quality improvement plans to strengthen the data management and reporting system to improve data quality.

To this end, the objectives of the RDQA tool are:

To rapidly verify the quality of reported data for key indicators at service delivery sites, including health facilities, outreach interventions e.g., peer educators, community health workers, etc.
To rapidly assess the ability of data management systems to collect, manage and report high-quality data.
To implement corrective measures with action plans for strengthening the data management and reporting system to improve data quality.

Dimensions of data quality

The data quality dimensions addressed by RDQA tools include accuracy, availability, completeness, timeliness, integrity, confidentiality and precision.

Data Quality Dimension	Operational Dimension

Data Quality Dimension	Operational Dimension
Accuracy	The degree to which the data correctly reflects what they were intended to measure. It is also known as validity. Accurate data correctly measure actual events, cases, units, etc.
Availability	The extent to which data and its supporting documentation are available. Supporting documentation include documents that contain the data e.g. client in-take forms or registers, patient cards, summary reports, etc.
Completeness	The extent to which the document contains all the required entries of the indicator as appropriate i.e. are completely filled in.
Timeliness	The extent to which data is up-to-date (current) and is made available on time.
Integrity	The extent to which data is protected from unauthorized changes or manipulation.
Confidentiality	The extent to which clients’ personal information is protected and kept secure.
Precision	The extent to which data is collected with the level of detail required to measure the indicator e.g. disaggregation by age, sex, etc.

Components of the RDQA tool

The RDQA tool was designed mainly for the purpose of conducting routine data quality checks as part of ongoing supervision at the service delivery level. Network members are encouraged to adapt this tool and fit it into their local program context. There is a need for members to prioritize, allocate resources, and establish an independent and objective reporting line for RDQAs. The RDQA tools consist of:

Instructions: This section provides general instructions on how to complete the tool.
Data verification: This section facilitates the assessment of the quality of data for the selected indicator. It involves the review of documents for accuracy, completeness, timeliness, etc. This sheet also includes a dashboard section where the scores and
ratings for each data quality dimension are displayed.
Accuracy_Monthly data sheet: This section is used to enter or tally monthly or periodic
data under verification
System assessment: The system assessment checklist enables the qualitative
assessment of the strengths and weaknesses of the functional areas of the data management and reporting system. This helps to identify potential risks to data quality in the system.

Both the Data verification and System assessment sheets include recommendations and action plan sections that outline actions required, persons responsible, and timelines.

While it is recommended that both parts of this tool (Data Verification and System Assessment) be used to assess data quality fully, the system assessment component could be conducted less often. This can be done during the initial assessment (baseline) and can be triggered when the site scores low on the data verification component.

Steps for conducting an RDQA

Prior to conducting an RDQA there are several steps you must complete before starting the assessment.

Identity the project and program area to be assessed. Then identify key indicators to be included in the assessment as well as gather reporting documents for the slected period that will be assessed.
Understand the data flow and select the reporting sites to be included in the assessment (all or a representative sample)
Prepare and conduct site visits. Sites should be notified prior to the visit to ensure relevant staff are present to complete the checklist in the tool, facilitate access to the relevant documents and receive feedback following the assessment.

Factors to consider when planning RDQAs

Ideally, RDQAs should be conducted routinely across all sites, programs, indicators, and M&E reporting levels as part of ongoing supervision. However, this may not be possible or necessary for different network members, given their local context. Some factors to consider when determining the scope and frequency of conducting RDQAs include the following:

Factors	Consideration for RDQA
Number and size of programs/projects	Members with fewer and/or smaller programs may be able to conduct RDQAs more frequently and across all programs than those with more and/or larger programs. Larger programs may have priority over smaller ones when planning for RDQAs.
Indicators and period for assessment	Different programs report different indicators to programs or donors. Indicators to be assessed can be selected based on their importance to programs or donors, number of partners reporting on them, and those with suspected data quality issues. The reporting period under assessment can vary by program or indicator from one to several months.
Number and type of service delivery sites and client volumes	Members with fewer service delivery sites may be able to conduct RDQAs more frequently and across all service delivery sites than those with many sites. RDQAs should be conducted across the different types of delivery sites. Sites with high client volumes may have priority over those with low client volumes.
Resources available	Conducting RDQAs require resources (e.g. human, financial, time, etc.). Members with adequate resources may conduct RDQAs more frequently and across all programs than those with fewer resources. In some cases, RDQAs may be outsourced where feasible.
Quality of data	Programs or service delivery sites where the quality of the data is low may be given priority over those with high quality data when planning for RDQAs. RDQAs may also be conducted in preparation for a formal data quality assessment.

Approaches to Conducting RDQAs

Different approaches may be used to conduct RDQAs. RDQAs may be conducted across all or key program areas and across all or a sample of service delivery sites. The frequency of conducting RDQAs may vary for each program area depending on need and a mix of the above factors. RDQAs can be done on a monthly to an annual basis.

Conduct RDQA on all sites across all or key program areas:

As the RDQA is designed to support quality improvement of data reported by service delivery sites, it can be included in already planned supervision visits at the sites. Service delivery sites would be assessed on the quality of their data and their underlying data management and reporting systems. Their performance would inform the level of support needed for data quality improvement and can be used to determine the timing of their next assessment or support visit. Where resources allow, each service delivery site and outreach team should be assessed at least once a year at the minimum. Additional assessments can be prioritized based on the quality of data or other factors.

Conduct RDQAs on a sample of sites across all or key program areas:

Given the various factors determining the scope for conducting RDQAs, it may not be possible to assess all service delivery sites for a given program area. Members may choose to use random sampling techniques to select a representative sample of sites to obtain a reasonable estimate of data quality for a given program. However, a purposive sample may also be selected based on the purpose of the RDQA. Precise estimates require a large number of sites though this may not be practical in terms of resources. Where the data flow includes intermediate reporting or aggregation levels (e.g., districts, regions, or provinces), sites may be sampled based on these levels (cluster sampling).

Using DHIS2 to conduct an RDQA

While data assessment teams can use the RDQA tool offline, PSI wanted to support on-the-ground M&E advisors and facility supervisors with an efficient and accurate means of conducting these assessments that would support rapid assessment collection and feedback.

Since 2011, PSI has used DHIS2 as its global enterprise-level management information system. DHIS2 is used widely across countries, programs, and projects; therefore, PSI developed an RDQA metadata package to be used in DHIS2.

The RDQA DHIS2 metadata package provides an out-of-the-box DHIS2 configuration that allows DHIS2 system administrators efficiently establish the RDQA components within DHIS2. System administrators can configure the metadata package for use on a web browser (laptop or computer) or an Android device (phone or tablet).

The RDQA modules allow data assessors to capture inputs and provide outcomes and analysis in real-time using standardized DHIS2 dashboards, allowing for improved data quality and decision-making at facility, district, and country levels.

As of 2023, PSI has implemented the RDQA metadata package in over ten countries across several health areas, including HIV, VMMC, TB, SRHR, IMCI, Safe Motherhood, Malaria, DREAMS, and WASH.

RDQA Confluence Space

Within the RDQA Confluence Space, you will find technical documentation and user manuals to help you to install and implement the RDQA tool.