An Assessment of the Application of Quality Improvement ... · PDF fileAn Assessment of the...

AN ASSESSMENT OF THE APPLICATION OF QUALITY

IMPROVEMENT CONCEPTS BY HEALTH CARE WORKERS

TRAINED UNDER THE AIDSFREE PROJECT IN

ZIMBABWE

AIDSFREE ZIMBABWE MAY 2017

AN ASSESSMENT OF THE APPLICATION OF QUALITY

IMPROVEMENT CONCEPTS BY HEALTH CARE WORKERS

TRAINED UNDER THE AIDSFREE PROJECT IN ZIMBABWE

AIDSFREE ZIMBABWE MAY 2017

This publication is made possible by the generous support of the American people through the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) with the U.S. Agency for International Development (USAID) under the Cooperative Agreement Strengthening

High Impact Interventions for an AIDS-free Generation, number AID-OAA-A-14-00046. The information provided does not necessarily reflect the views of USAID, PEPFAR, or the U.S. Government.

AIDSFree

The Strengthening High Impact Interventions for an AIDS-free Generation (AIDSFree) Project is a five-year cooperative agreement funded by the U.S. Agency for International Development under Cooperative Agreement AID-OAA-A-14-00046. AIDSFree is implemented by JSI Research & Training Institute, Inc. with partners Abt Associates Inc., Elizabeth Glaser Pediatric AIDS Foundation, EnCompass LLC, IMA World Health, the International HIV/AIDS Alliance, Jhpiego Corporation, and PATH. AIDSFree supports and advances implementation of the U.S. President’s Emergency Plan for AIDS Relief by providing capacity development and technical support to USAID missions, host-country governments, and HIV implementers at the local, regional, and national level.

Recommended Citation

Mutede Blessing, Chivanga Roy, Mponda Juliet, Chinaka Eugenia, Musarandega Reuben, and Mahomva Agnes. 2017. An Assessment of the Application of Quality Improvement Concepts by Health Care Workers Trained under the AIDSFree Project in Zimbabwe. Arlington, VA: Strengthening High Impact Interventions for an AIDS-free Generation (AIDSFree) Project.

JSI Research & Training Institute, Inc. 1616 Fort Myer Drive, 16th Floor Arlington, VA 22209 USA Phone: 703-528-7474 Fax: 703-528-7480 Web: aidsfree.usaid.gov Email: [email protected]

http://aidsfree.usaid.gov/

mailto:[email protected]

Acknowledgments

AIDSFree would like to acknowledge the Zimbabwe Ministry of Health and Child Care Quality Assurance and Quality Improvement (QA/QI) Department and the HIV QI improvement team for providing overall direction and the project framework. We would also like to thank the provincial medical directorates for Manicaland and Masvingo and their respective district medical officers for facilitating implementation of this project and the subsequent post-training assessment. The same gratitude goes to the direct beneficiaries of this project, as well as the health care workers for their dedication to duty and application to task.

The following persons and organizations made outstanding contributions to this project and its assessment: Dr. Agnes Mahomva, Dr. Blessing Mutede, and Mr. Reuben Musarandega provided continuous critical oversight and direction, from project design and implementation through the assessment and evaluation stages; Ms. Juliet Mponda, Ms. Eugenia Chinaka, Mr. Roy Chivanga, and the rest of the AIDSFree team in Zimbabwe worked tirelessly to ensure that the implementation of the project as well as this post-project implementation assessment protocol was conducted efficiently and on time.

Last but not least, we would like to acknowledge the support the AIDSFree Zimbabwe country team received from EGPAF’s global AIDSFree team as well as from JSI, the global AIDSFree Project prime. Finally, we would like to acknowledge USAID Zimbabwe mission for availing financial support for this project

vii

CONTENTS Acronyms .......................................................................................................................................................... ix

Executive Summary ........................................................................................................................................ xi

Introduction ...................................................................................................................................................... 1

Background ................................................................................................................................................................... 1

AIDSFree Overview .......................................................................................................................................... 3

AIDSFree QI Technical Assistance in Zimbabwe ............................................................................................. 3

The Training Assessment ............................................................................................................................... 5

Objectives ...................................................................................................................................................................... 5

Methods and Materials ............................................................................................................................................ 5

Ethical Considerations .............................................................................................................................................. 9

Results .............................................................................................................................................................. 11

General Findings ...................................................................................................................................................... 11

Pre- and Post-Training Changes in QM/QI and Indicators ..................................................................... 18

Discussion ........................................................................................................................................................ 21

Limitations .................................................................................................................................................................. 22

Conclusions and Recommendations ........................................................................................................ 23

Recommendations .................................................................................................................................................. 23

Annex 1. Facilities with Health Care Workers Trained on QM/QI ..................................................... 25

Annex 2. National QI Indicators ................................................................................................................. 27

Annex 3. Semi-Structured Interview Questionnaire for Health Care Workers Trained in Quality Improvement under the AIDSFree Mechanism ..................................................................................... 31

Annex 4. Focus Group Discussion Guide for Quality Management Committee Members ........ 37

Annex 5. Extent of Establishment of QM Structures Within the Capacitated Facilities .............. 41

MOHCC Organizational Quality Assessment Tool ...................................................................................... 41

Annex 6. Consent Form ................................................................................................................................ 57

ix

ACRONYMS AIDSFree Strengthening High Impact Interventions for an AIDS-free Generation

ANC antenatal care

ART antiretroviral therapy

EGPAF Elizabeth Glaser Pediatric AIDS Foundation

FGD focus group discussion

HEI HIV-exposed infant

HCW health care worker

MNCH maternal, newborn, and child health

MOHCC Ministry of Health and Child Care

MRCZ Medical Research Council of Zimbabwe

OA organizational assessment

OI opportunistic infection

PEPFAR U.S. President’s Emergency Plan for AIDS Relief

PM performance measurement

QA/QI quality assurance/quality improvement

QM quality management

QMC quality management committee

QMP quality management program

QM/QI quality management and quality improvement

USAID U.S. Agency for International Development

xi

EXECUTIVE SUMMARY

The goal of the HIV quality improvement (QI) program in Zimbabwe is to provide care that meets or exceeds clients' expectations and aligns with national HIV prevention, care, and treatment guidelines. Implementation of the HIV QI program is guided by the National Quality Assurance/Quality Improvement (QA/QI) Policy and the National Quality Improvement Strategy. Between 2015 and 2016, the U.S. Agency for International Development- and U.S. President’s Emergency Plan for AIDS Relief-funded Strengthening High Impact Interventions for an AIDS-free Generation (AIDSFree) Project supported Zimbabwe Ministry of Health and Child Care (MOHCC) with technical assistance to build the capacity of facilities and health care providers in QI programming. Through this assistance, a total of 59 health care workers (HCWs) from 38 health facilities received formal training in HIV QI, followed by systematic post-training coaching and mentoring. In November 2016, AIDSFree conducted a post-training assessment to determine the extent to which trained HCWs applied the knowledge and skills they had gained three months after the training. The team also assessed the extent to which providers' respective facilities implemented quality management/quality improvement (QM/QI) activities and changes in QM/QI indicators before and after training.

AIDSFree conducted a cross-sectional study covering all the HCWs trained between May and July 2016 and the facilities from which they came. The assessment used the standard organizational assessment (OA) tool used in the national program and semi-structured interviews to collect data on the extent of QM/QI implementation and application of knowledge and skills, respectively. The team conducted retrospective health facility record review to determine changes in QM/QI indicators before and after training; and conducted focus group discussions (FGDs) to explore barriers and enablers and provide program context.

There was a good retention and response rate (93%) among the trained HCWs. Most (96.3%) of the trained HCWs were intensely involved in facility-level QM/QI programming. Almost two-thirds (64.6%) found the training very useful and 78 percent were able to correctly match the QI tool with the corresponding processes of QI implementation.

The extent of QM/QI implementation ranged from a mean domain score of 0.97 to 1.94 of a possible five across the nine QM/QI domains. This was a significant improvement from mean domain scores ranging from 0.14 to 1.04 before the training (p<0.05). Eight of fourteen QI indicators tracked in the national QI program significantly improved after the training, but six did not change. These differences were amplified after stratifying by the availability of post-training support to the facility. Availability of support from the leadership and QI coaches, time, and teamwork were enablers for application of QM/QI knowledge and skills; lack of staff and equipment were notable barriers. Most of the respondents had adaptive strategies to mitigate these barriers.

Facility-level HCWs should receive post-training support after formal QI training to sustainably improve the application of learned skills and knowledge, and thus QM/QI implementation.

1

INTRODUCTION

Background The Zimbabwe Ministry of Health and Child Care (MOHCC) developed the national HIV quality management/quality improvement (QM/QI) program in 2013 with support from the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF-Zimbabwe) and HealthQual International, among other partners. The goal of the HIV quality improvement (QI) program in Zimbabwe is to provide care that meets or exceeds clients' expectations and complies with national HIV prevention, care, and treatment guidelines. Implementation of QI is based on guidance from the Zimbabwe National Quality Improvement and Quality Assurance (QI/QA) Policy and the Zimbabwe National Quality Improvement Strategy. The policy and the strategy will guide quality programming for the entire health services sector for five years (2016–2021).

Based on the policy and strategy, the HIV QI program in Zimbabwe distinguishes broader and all-encompassing QM from the actual QI actions, which are more HIV program-oriented. A well-developed quality management program (QMP) is recommended for QI to occur.

Quality management in the HIV program comprises the establishment of frameworks, structures, and plans at all levels of the health care delivery system, from facility to national levels. A mature QMP operates effectively in the following major domains (see Box 1).

QM programs comprise seven major domains, described below. 1) Quality management structures:*

• Senior leadership creates an environment that supports a focus on improving the quality of care in the hospital. • The hospital program has an effective quality committee to oversee, guide, assess, and improve the quality of hospital

services. • The hospital has a comprehensive quality plan that is actively used to oversee QI activities.

2) Workforce engagement: Clinicians and staff routinely engage in QI activities and are provided training to enhance knowledge, skills, and methodology needed to fully implement QI work on an ongoing basis.

3) Data measurement, analysis, and use: The hospital or health facility routinely measures program performance and uses data to improve it.

4) Quality improvement initiatives: The hospital identifies and conducts QI initiatives using methodology to assure high levels of performance over long periods of time.

5) Consumer/patient involvement: Patients are effectively engaged and involved in the HIV QM program. This entails systematic collection of consumer views on the care they receive. Analysis of the information collected allows providers to modify the systems and processes in HIV care to make the program more acceptable to the intended beneficiaries.

6) Quality program evaluation: A process is in place and is used to evaluate the hospital’s QMP and related activities, processes, and systems to ensure attainment of goals, objectives, and outcomes.

7) Achievement of outcomes: The HIV program monitors patient outcomes and uses data to improve care.

*The QM structures domain is a broad domain with three subdomains, i.e., enabling environment, QM committee, and quality plan. These domains, for the purpose of assessing the advancement of QM/QI work in a facility, are assessed and reported separately, effectively leading to nine QM/QI domains, as shown in Figure 1.

Box 1. Quality Management Program Domains

2

Quality improvement encompasses the actual activities conducted at facilities to close observed service gaps defined by the performance measurement (PM) framework and key problems that health facilities implementing HIV care have prioritized. These are normally organized into improvement projects by teams of three to four individuals working on specific, clearly defined service gaps over a period of six months. Currently, 14 nationally accepted service delivery areas are prioritized for improvement through QI (see Annex 2). Health facilities participating in the HIV QI program can choose up to three projects, depending on the availability of staff.

The Zimbabwe HIV QI Strategy recommends that health facilities have annual organizational assessments (OAs) of progress toward full establishment and implementation of QMPs in the domains described above. The strategy emphasizes coaching and mentoring health providers and peer learning platforms to build their capacity for QM/QI implementation; sustain initiatives; improve programmatic efficiency; and create a culture of routine QI in health services.

Training, coaching, and mentoring build the capacity of health care providers to continuously and systematically improve the quality of their programs at all levels, and is a key feature of the HIV QI program in Zimbabwe. Coaching and mentoring for QI goes beyond clinical skills transfer. Such support requires training HCWs to gain a deep understanding of their program context, structure, and dynamics to identify opportunities for improvement. This knowledge is used to improve the program's effectiveness and efficiency by modifying the systems and processes underlying HIV care. The coaching and mentoring also provides districts and sites with the capacity to document systematic improvement lessons for collective learning within the program.

Peer learning: To the extent possible, service providers are given the opportunity to learn what works and to adapt lessons/best practices from other care settings to their own. In the Zimbabwean HIV QI program, this entails convening facilitated inter-facility or -district meetings where QI implementers review progress. Participants review and share successful program implementation strategies so they can be replicated in other facilities. An external facilitator often moderates these sessions. Virtual peer learning meetings via conference calls and email are also implemented; these mainly take place at national and international levels, though some facilities participate.

3

AIDSFREE OVERVIEW The Strengthening High Impact Interventions for an AIDS-free Generation (AIDSFree) Project is a five-year initiative, funded by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) through the U.S. Agency for International Development (USAID) to improve the quality and effectiveness of high-impact, evidence-based HIV and AIDS interventions. The project is led by JSI Research & Training Institute, Inc. in partnership with the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF), Jhpiego, Abt Associates Inc., EnCompass LLC, and IMA World Health. In late 2015, USAID/Zimbabwe made funding available to the AIDSFree Project for technical assistance on QI interventions, provided to the MOHCC through EGPAF-Zimbabwe at the national and district levels in selected USAID-supported health care facilities. The AIDSFree Zimbabwe QM/QI activity ended in December 2016.

AIDSFree QI Technical Assistance in Zimbabwe The goal of the AIDSFree Zimbabwe QM/QI technical assistance activity was to use QI strategies to improve the survival and quality of life of HIV-positive persons through early identification, initiation, and retention on antiretroviral therapy (ART). Although the scope of work for the project was much broader, this assessment focuses on two key activities: creation of an enabling environment for QI; and building HCWs’ skills in QM/QI implementation. These were central to the achievement of the activity’s objectives.

Creating an enabling environment for QI: AIDSFree Zimbabwe supported the revision of QI training materials and other normative documents, and trained health care providers at the national, provincial, and district levels to support full implementation of the national HIV QI program. Led by consortium partner EGPAF’s Zimbabwe office, AIDSFree supported revision and update of QI training materials, standard operating procedures, and job aids. This preliminary work to guide QI implementation was competed between January and March 2016.

AIDSFree supported the districts and health facilities to establish quality management committees (QMCs) to develop QM plans and client feedback mechanisms based on guidance from the Zimbabwe National Quality Assurance and Quality Improvement Policy and the QI Strategy. Each district facility supported for QI had the target of fully establishing a QM infrastructure within the seven QM domains described above.

Capacity building for QI: AIDSFree recruited four district-level QI trainers or coaches at the beginning of the project (December 2015) to train, coach, and mentor district- and facility-level HCWs so that they could implement QI activities in their own facilities. The trainings were based on the national QI curriculum, which combines PM and QI components. The training included PowerPoint® presentations, facilitated plenary discussions, practical data abstraction, project planning sessions, and group work to ensure participatory learning. The training content comprised principles of PM for QI; service delivery problems prioritization and QI planning; implementation; and documentation based on the plan-do-study-act model. Pre- and post-test examinations were administered to assess knowledge levels before and after the training. Upon returning to their facilities, the trained HCWs were expected to

4

lead the implementation of QI activities. After formal trainings, participants received intensive coaching and mentoring. By the end of August 2016, AIDSFree had trained 115 HCWs, covering 87 percent of the QI facilities targeted for the year in USAID-supported districts.

Program changes: Due to realignment of roles and responsibilities between EGPAF and other USAID implementing partners in early 2016, not all the HCWs who were trained under the AIDSFree QI Technical Assistance Activity between May and July received follow-up site-level coaching and mentoring from EGPAF. Some facilities whose health care workers (HCWs) had been trained by AIDSFree transitioned to other implementing partners for facility-level QI implementation. EGPAF was not able to follow through with these facilities with QI coaching in the post-training period according to the AIDSFree workplan.

5

THE TRAINING ASSESSMENT

As part of the AIDSFree end-of-project report, an assessment of the completed QI trainings to determine the extent to which trained HCWs were able to apply their new knowledge and skills in their health facilities was conducted. This assessment took place after a three-month post-training period.

The assessment sought to answer:

1. To what extent did the HCWs trained by the AIDSFree Project develop QM structures in their facilities?

2. To what extent did trained HCWs implement QM/QI processes and strategies in their facilities? 3. What were the barriers/facilitators to implementation of QM/QI processes and strategies within

the trainees' facilities?

Objectives The project's goal was to determine the extent to which HCWs applied QI knowledge gained from the combined PM and QI training at their health facilities. Specific objectives included:

1. Assess the extent of QM implementation at the trainees' health facilities against the seven nationally recommended QM/QI domains (aspects).

2. Determine the extent to which participants applied the knowledge they received at the formal PM/QI training to implement initiatives in their own facilities.

3. Explore facility-level barriers and enablers that participants encountered in applying the knowledge gained during the training.

4. Review the strategies that participants used to address facility-level barriers to QM/QI implementation.

5. Review and document changes in selected QI indicators at participants' facilities before and after trainings.

Methods and Materials

Study Design The assessment used a cross-sectional study design that assessed the QM/QI outcomes of HCW trainings implemented between May and July 2016.

Setting: The assessment took place in health facilities with HCWs who were trained in QM and QI. These included rural and urban sites ranging in size from clinics to hospitals.

Populations: There were two main study populations in this assessment:

• HCWs trained under the AIDSFree mechanism between May and July 2016. This cohort had the longest time to implement QI initiatives.

• Health facilities from which the HCWs trained under the AIDSFree mechanism were drawn from.

6

Sample Size

Facility sampling: All 38 health facilities that had at least one staff member trained between May and July 2016 were included in the assessment. Annex 1 shows the list of these facilities.

HCW sampling: All 59 HCW trainees were included in this assessment.

Study outcomes and variables: The main study outcomes were:

• The extent that trained HCWs were involved in QM/QI processes within their health facilities, measured on a three-point ordinal scale (not involved, minimally involved, and heavily involved).

• The extent of QM implementation in facilities with trained HCWs determined on a 5-point ordinal scale, where 0 represents no implementation and 5 a fully implemented/mature QM program.

Table 1 shows the study variables measured in the assessment and their respective data collection tools.

Table 1. Study Variables and Their Data Sources

Domain to be Measured Variables Data Collection Method/Tool

Objective 1: Assessing the extent of QM implementation at the trainees' health facilities against the seven nationally recommended QM/QI domains (aspects)

1. Leadership involvement* Facilities’ mean score for the creation of an enabling environment for QI by senior leadership

OA tool

Facilities’ mean score for the provision of leadership and oversight by the quality committee

Facilities’ mean score for the active use of a comprehensive plan to oversee QI activities

Facilities’ mean score for the engagement of staff in QI activities

2. Measurement, analysis and use of data to improve program performance

Facilities’ mean score for routinely conducting PM and use of data for program improvement

3. QI initiatives Facilities’ mean score for identifying and conducting QI to sustain high performance

4. Patient involvement Facilities’ mean score for engaging and involving patients

5. Quality program evaluation Facilities’ mean score for routinely evaluating their QM programs

6. Achievement of outcomes Facilities' mean score for monitoring patient outcomes in the HIV program

7


Objective 2: Determining the extent to which participants applied the knowledge they received at the formal PM/QI training to implement QI initiatives in their own facilities

Extent of involvement of trained HCWs in facility-level QM/QI implementation

Number and proportion of trained HCWs who sit on QM committees

Structured interviews

Number and proportion of trained HCWs involved in at least 3 aspects of QM/QI planning within their facilities Number and proportion of trained HCWs involved in all the steps of QI planning

Understanding of QI tools and processes

Number and proportion of trained HCWs who applied the correct process at each step of QI implementation Number and proportion of trained HCWs who used correct and relevant tools at each step of QI implementation Number and proportion of trained HCWs that reported ease-of-use of the relevant QI tool for each step

Usefulness of QI training and availability of post-training support

Number and proportion of trained HCWs who found concepts learned in the training useful and relevant to QI implementation in their facilities Number and proportion of HCWs mentioning at least three aspects of the training that they found useful in QI implementation Number and proportion of trained HCWs who received any post-training support

Objective 3: Exploring facility-level barriers and enablers that participants encountered in applying the knowledge gained during the training

Facility-level barriers and enablers for QI implementation

Number and proportion of HCWs reporting barriers to QM/QI implementation

Semi-structured interviews, FGDs

Number and proportion of HCWs reporting facilitators to QM/QI implementation

Number and type of barriers identified

Number and type of facilitators identified

Objective 4: Reviewing the strategies that participants used to address facility-level barriers to QM/QI implementation

Strategies to address barriers Number and proportion of HCWs that had strategies to improve QM/QI implementation

Semi-structured interviews FGDs

8


Number and types of strategies used

Objective 5: Reviewing and documenting changes in selected QI indicators at participants' facilities before and after trainings

QM domains Difference in mean QM/QI domain scores EGPAF QI database facility records review

QI Difference in mean uptake proportions for QI indicators

EGPAF QI database facility records review

*The leadership domain is broken into 3 subdomains for the purpose of assessing each health facility, effectively giving 9 QM/QI domains instead of 2 as contained in Box 1.

Data Collection Methods

Organizational assessment: A facility checklist (Annex 5) adapted from the MOHCC OA tool was used to measure the extent of QM implementation in participating facilities. The checklist included QI strategic plans; minutes of QI meetings; existence of QI corners; evidence of generation and use of PM data to track and improve service delivery; and availability of client feedback mechanisms developed by the facility’s QI team.

Structured interviews: Structured interviews were conducted with the trained HCWs using interviewer-administered questionnaires (Annex 3). The interviews reviewed the HCWs' understanding of PM and QI processes; assessed their involvement in the implementation of QI initiatives within the health facility; and explored their perceptions of facility-level barriers or facilitators of QI implementation.

Focus group discussions: FGDs were held with the health facility’s QMCs, whose members oversaw QI implementation. The FGDs explored facility-level barriers and/or enablers to QM/QI implementation. FGDs also solicited managers’ perspectives on facility-level QM/QI implementation. An FGD guide for the members of the QMC was used for this purpose (Annex 4).

Health facility records review: The technical assistance team conducted a desk review of data from organizational assessments done before AIDSFree began the HCW training and other activities. The review used the same tools and methods used in this assessment, and stored the data in the EGPAF-Zimbabwe QI database. QI PM data available within the same database were also downloaded and reviewed.

Data Collection Process

Two teams collected the data. Each team comprised of four people drawn from EGPAF-Zimbabwe and MOHCC. Each team visited 9 to 10 health facilities per week over a two-week period, including travelling between facilities. HCWs were notified in advance of the data collection to maximize the yield. Each group member was assigned specific aspects of the data collection in each team to optimize the efficiency of data collection and reduce variations among observers.

9

Training of Data Collectors and Project Staff

All data collectors, trainers, and management staff received training on research ethics before starting data collection activities. The whole data collection team was trained on the data collection procedures and processes, including using the structured questionnaire to collect quantitative data; administering the OA tool; and conducting FGDs by AIDSFree and EGPAF staff. The research team from the AIDSFree/EGPAF global office provided support to the in-country team throughout the assessment.

Data Management and Analysis

Data quality assurance: Before departure from each health facility, the most senior member of each data collection team verified the forms and questionnaires and addressed any inconsistency in the collected data at the site. Trained data collectors entered the data directly into a specially prepared, password-protected access database. Unique identifiers were created for each completed questionnaire to facilitate tracking of entries when reviewing data accuracy. Identifiable data such as name and contact details of the HCWs were not captured.

Quantitative data entry and analysis: Quantitative data from the structured interviews and the OAs were entered into Microsoft Access® and Microsoft Excel® databases. Aggregate data from past OA and PM reports were downloaded from the EGPAF QI database. The data were summarized using means and proportions as follows:

1. The extent of QM/QI implementation was determined by calculating the mean domain scores for each domain and comparing this with the expected results for full implementation of a QI program on a continuous validated 0–5 scale.

2. The extent of involvement of the trained HCWs was determined semi-quantitatively on a three-point ordinal scale by determining the proportion of respondents reporting involvement in a predetermined number of steps of QM/QI implementation.

3. Changes in QM and QI indicators were determined by comparing mean scores obtained by the facilities before and after AIDSFree QI implementation (formal training and coaching), and testing the paired differences for significance using Student’s t-test.

Qualitative data: Qualitative data were manually sorted into predetermined themes and analyzed using thematic content analysis to determine enablers and barriers encountered and strategies to address them.

Ethical Considerations The MOHCC conducted an institutional review before granting permission to conduct the study. Ethical approval was also obtained from the Medical Research Council of Zimbabwe (MRCZ), approval number MRCZ/E/151.

No clients or patients were interviewed for this assessment; only HCWs were interviewed. All respondents were granted the right to withdraw or opt out from any component of the data collection in which they were involved.

10

Consent process: Written informed consent was obtained from all participating HCWs. A signed consent form adapted from the MRCZ was used for all respondents (Annex 6). Personally leading and identifying data were collected during the consent process from those interviewed but the signed consent forms were delinked from the questionnaires and were separately filed in a lockable cabinet.

Personally identifying health information: No personally identifiable data were used during the analysis or in the report.

Confidentiality: Individual study participants and participating health facilities are not identified in the report. This will protect facilities that have not yet implemented any QI strategies from being regarded as “underperforming” and thus facing censure.

11

RESULTS

General Findings

Response Rate

The assessment covered all participating districts and sites, but some respondents were not present on the day of the assessment. Table 2 shows the level of response achieved for the three elements of the assessment.

Table 2. Distribution of Responses by Data Collection Component

Study Data Component Target Achieved Percentage Completed

1. Organizational assessments 38 37 97 2. Semi-structured interviews 59 55 93 3. Focus group discussions 38 30 79

Profile of Health Facilities and Interviewees

The assessment included both urban and (mainly) rural facilities, ranging from clinics to large hospitals. Table 3 profiles the 38 participating facilities.

Table 3. Profile of Health Facilities

Variable Category Frequency (%)

Type of Facility Clinic 23 (60.5) Med–large hospital (>20 beds) 12 (31.6) Small hospital (<20 beds) 3 (7.9)

Setting Rural 28 (73.7) Urban 10 (26.3)

Nurses from these facilities were the primary respondents for this assessment. Table 4 shows the profile of the 54 respondents interviewed.

Almost two-thirds of the HCWs were female. The majority (44.4%) of respondents and of HCWs trained by AIDSFree were in the 36- to 45-year age range. More than half (55.6%) were working in either the maternal, newborn, and child health (MNCH) and maternity or opportunistic infection (OI)/ART departments; these were mostly respondents from medium-sized to large hospitals. Over one-third of respondents from clinics had work that cut across several departments (OI/ART, MNCH, and maternity).

12

Table 4. Profile of Interview Respondents

Variable Characteristics n (%)

Age

<35 16 (29.6) 36–45 24 (44.4) 46–55 10 (18.5) >55 4 (7.4)

Sex Male 19 (35.2)

Female 35 (64.8)

Department

MNCH (including maternity) 15 (27.8) OI/ART 15 (27.8)

Multi (includes OI/ART, MNCH and maternity)

19 (35.2)

General (OPD or IPD)* 5 (9.3)

Designation Nurse with administrative role 8 (14.8)

Nurse with midwifery 9 (16.7) Nurses (QI implementers) 37 (68.2)

Duration in Post** <2 years 8 (14.8) 2–5 years 12 (23.1) >5 years 32 (61.5)

* OPD: outpatient department; IPD: inpatient department. ** 2 data values missing.

QM/QI Implementation among Facilities and Participants

Figure 1 shows the average scores for implementation of QM/QI activities in the 38 participating facilities. Scores are based on a 0–5 scale across 9 domains, with 5 representing a mature program. Scores ranged from 0.97 to 1.94 across all domains.

13

Figure 1. Mean QM/QI Domain Scores for Facilities

Involvement of Trainees in QM/QI Planning and Implementation in Their Facilities

Table 5 indicates participants' involvement in QM/QI, based on the three-point scale described previously. Most of the respondents in the assessment (79.6%) participated in structures that directly manage HIV QI. This increases to more than 96 percent if participants who participate in other quality-related committees are added. About half of respondents (53.7%) were involved in between 5 and 10 aspects of HIV QM/QI (significant involvement); and 22.2 percent were involved in 10 or more aspects. In terms of the actual QI implementation, almost two-thirds of respondents were involved in all facility-level steps of QI implementation; only one-fifth were minimally involved (participating in three of the six steps).

Table 5. Proportion of Respondents Involved in QM/QI Implementation

Involvement Variable Criteria Classification Frequency (%)

1. Involvement in a QM/QI committee (based on a 3-point scale )

Structure directly related to HIV QM/QI or QMC + QI Team

Intensively involved 43 (79.6)

Quality-related structure not directly related to HIV QI, e.g., MNCH+ IPC

Involved 9 (16.7)

1.1

0.41 0.24

0.55 0.48

0.17 0.41

0.14

0.66

1.94

1.64

1.28

1.54 1.51 1.35

1.24

0.97

1.32

0

0.5

1

1.5

2

2.5QUALITY MANAGEMENT DOMAIN

BaselineMeanDomainscore

AssessmentMeanDomainScore

14

Involvement Variable Criteria Classification Frequency (%)

Does not sit on any quality-related committee

Minimally involved 2 (3.7)

2. QM/QI aspects that the respondent was involved in (of a possible 11)

10–11 aspects Intensively involved 12 (22.2) 5–9 aspects Involved 29 (53.7) <5 aspects Minimally involved 13 (24.1)

3. Steps of QI implementation in which the respondent was involved (of a possible 6)

5–6 steps Intensively involved 35 (64.8) 3–4 steps Involved 8 (14.8) <3 steps Minimally involved 11 (20.4)

Figure 2 shows respondents' involvement in the steps of the QM/QI process. Two-thirds or more were involved in most steps, but fewer than half were involved in setting up QI teams or implementing peer learning platforms.

Figure 2. Distribution of Respondents by QM/QI Aspect in Which They Were Involved

FGD respondents in hospitals reported that QI committees comprise mainly department heads. Those in clinics mentioned that junior employees, such as nurse aides, community-based health workers, and health center committee members, also sit on the QMC. Only HCWs who are members of the QI committee took part in FGDs. Almost all QI meetings took place at the health facilities at least once a month, in the afternoons when site staff is less busy, and lasted for between 30 minutes and one hour. Attendance was said to be good. The most frequent reasons for non-attendance mentioned were attending workshops or being on leave.

39% 48%

65% 67% 69% 70% 74% 76% 76% 76%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Prop

ortio

n of

Res

pond

ents

QM/QI Aspect

15

Figure 3. Respondents' Involvement in QI Steps

Usefulness of QI Training and Availability of Post-Training Support

All but 3 of the 54 participating HCWs commented on the usefulness of the QI training, whether they disseminated their knowledge to their peers, and whether they received any form of post-training support (Table 6). A large majority of respondents found the training useful or very useful, and nearly all were able to disseminate their new knowledge to peers at their facilities. Post-training support was available to most of the respondents.

Table 6. Trainees' Responses on QI Training and Availability of Post-Training Support

Variable Category Frequency (%)

Found the training useful

Very useful 35 (64.8)

Useful 15 (27.8)

Somewhat useful 1 (1.9)

No response 3 (5.6)

Disseminated learned knowledge to peers Yes 53 (98.2)

No 1 (1.8)

Received some form of post-training support Yes 48 (88.9)

No 6 (10.1)

64.8%

51.9%

66.7% 59.3%

66.7%

57.4%

0%

10%

20%

30%

40%

50%

60%

70%

Setting priorities Identifyingprojects

Establishing theteams

Understandingprocessesunderlying

systems of care

Making changes Measuring effectof change

Prop

ortio

n of

Res

pond

ents

QI Steps

16

QI Tools Used by Trainees

Figure 3 shows the QI tools used by the trained HCWs and the relative frequency of use for each.

Almost 80 percent of the respondents were able to correctly mention use of appropriate QI tools for the different steps of the QI process, but a significant proportion (22%) could not. The most common correctly identified tool was the Fishbone, followed by brainstorming and then flowcharting. A few respondents (6%) had gone as far as requesting and using run charts.

Figure 4. QI Tools Used by Trained HCWs

Facility-Level Barriers and Enablers to Applying New Knowledge

Barriers to QI Implementation

Of the 53 trained HCWs who went back and implemented QI, 39 (73.6%) reported challenges. The most commonly reported challenges were:

• Lack of resources to implement QI (cited by 30.2% of respondents). The most common shortages were:

o Insufficient staff so the trained cadre had insufficient time to systematically implement QI along with other duties.

16%

10%

46%

6%

22%

Brainstorming

FlowChart

FishBone

Run Charts

Inappropriate

17

o Insufficient infrastructure including limited space for improving client flow; power cuts, which affect data capture and management; equipment breakdown (CD4 machine); and lack of EDTA tubes for outsourcing of some HIV testing services; ART services were also mentioned.

• Resistance from other staff members (cited by 13.2% of respondents).

Five respondents (9%) mentioned lack of confidence in applying knowledge of QI or being deployed to departments where opportunities to implement HIV QI were unlikely.

During FGDs with QI committees, most discussants said that they did not have adequate time to implement QI due to work overload, followed by lack of both human and material resources. A few committees said that they did not know what elements to support or how to support them, since they received irregular mentoring from their coaches. Some had last received support as far back as 2015. Respondents also mentioned that different partners with different approaches and expectations had negative effects on QI implementation.

More than half of the respondents (52.8%) were able to report on some positive experiences in the implementation of QI.

Enablers of QI Implementation

The 46 (85.1%) respondents who identified enablers of QI implementation mentioned the following:

• Availability of support from the senior leadership within the facility and the district health executive.

• Availability of a coach who supports QI implementation processes at the health facility. • Presence of other staff members trained in QI within the facility. • Finding time to fulfill QI processes and requirements.

Four respondents (7.5%) failed to identify enablers and confused them with strategies to overcome the challenges. Seven respondents (13%) did not identify any enablers for QI implementation.

Strategies Used by Participants to Address Barriers

HCWs used different strategies to address facility-level barriers to QM/QI implementation. • Nineteen respondents (36%) mentioned creating viable and dynamic teams in their facilities. In

these teams, effective communication and clear definition of roles were mentioned as being important.

• Fourteen respondents (26.9%) mentioned helping facility staff to understand and appreciate QI. Trainees used several strategies to help peers understand and support QI implementation: examining the working environment through applying the 5 “S” principles;1 developing the facility QI plan/roadmap; and conducting facility self-assessment reviews for selected key or core care processes.

1 Sort, Set in order, Shine, Standardize, Sustain: a workplace organization approach originally developed in Japan.

18

• Eleven (20.8%) respondents mentioned using QI coaches to facilitate processes to resolve facility-level barriers, and also to help the teams navigate particularly difficult challenges.

• Seven respondents (13.2%) used standard, appropriate implementation tools to implement QI programs. This included the use of visuals, such as establishing QI corners displaying implementation progress in graphic format, and pictures showing "before and after scenarios." This was also a motivating factor for staff.

• Other respondents (9%) mentioned creating quality-focused structures and ensuring that they met frequently.

The strategies highlighted above were not employed singly, but in different combinations.

Pre- and Post-Training Changes in QM/QI and Indicators After receiving training, participating HCWs led the implementation of QI initiatives within their health facilities. Table 7 below shows the differences in the mean QM/QI domain scores for these facilities before and after formal QI training.

Table 7. Changes in Mean QM/QI Domain Scores before and after Formal QI Training

Domain Mean Domain Score Paired

Difference P-Value

Endline Baseline

1. Enabling environment 2.04 1.04 1.00 0.0001* 2. Presence of quality committee 1.79 0.43 1.36 0.0001* 3. Availability of comprehensive quality plan 1.43 0.25 1.18 >>0.0001* 4. Workforce engagement 1.61 0.50 1.11 >>0.0001* 5. Performance measurement 1.64 0.46 1.18 0.0002* 6. QI initiatives 1.52 0.19 1.33 >>0.0001* 7. Patient involvement 1.57 0.64 0.93 0.0002* 8. Quality program evaluation 1.66 0.47 1.19 >>0.0001* 9. Achievement of outcomes 1.18 0.14 1.04 0.0001* * p<0.05

All the mean domain scores for QM/QI improved significantly after formal QI training under the AIDSFree mechanism. These differences are accentuated when these facilities are stratified according to availability of post-training follow-up support (see Figure 5). There was an incremental effect in the domain scores when the sites were stratified by those that received QI training only and those that received QI training plus QI coaching and mentoring.

19

Figure 5. Comparison of Baseline Mean Domain Scores against Scores for Sites Receiving QI Training Only and Sites Receiving QI Training and Coaching

Table 8 shows a comparison of QI indicators before and after the training. Some of these sites were implementing QI with post-training support from AIDSFree coaches. Improvements occurred in 8 of the 13 quality indicators. In 6 of these indicators, improvements were statistically significant at p<0.05. Although retention on ART and CD4 monitoring improved, the improvements were not statistically significant. The rest of the indicators did not change significantly by the time of the assessment.

Table 8. Comparison of QI Indicators Before and After QI Training at 38 Facilities

Indicator Mean Proportion %

Paired Difference %

P-Value,

Pr (T >t) Baseline Endline

Retention on ART (general clients) 81 89 08 0.16

Adherence assessment 76 91 15 0.03*

CD4 monitoring 13 17 04 0.20

DNA-PCR sample collection from HIV-exposed infants (HEI) <2 months

51 76 25 0.011*

HEI with DNA-PCR collected and had results returned within a month

09 46 37 0.0001*

0

0.5

1

1.5

2

2.5

Enablingenvironment

Qualitymanagement

committee

Qualityimprovement

plan

Workforceengagement

Regularperformancemeasurement

QI Initiatives Patientinvolvement

Programevaluation

Achievementof outcomes

Mea

n Sc

ore

QM/QI Domain

Baseline (All sites n= 38)

Endline (sites trained but no follow up coaching and mentoring, n=20)

Endline (sites trained and followed through with coaching and mentoring, n=18)

20

Indicator Mean Proportion %

Paired Difference %

P-Value,

Pr (T >t) Baseline Endline

Retention on ART (mothers) 0.0 36 36 0.0002*

HIV retesting 25 64 39 0.0002*

Pediatric cotrimoxazole initiation 52 75 23 0.02*

ART initiation in antenatal care (ANC) 58 47 -11 0.76

Tuberculosis screening 97 96 -1 0.78

Pediatric ART for <15 years 41 20 -21 0.96

ART initiation for HIV-positive infants 47 32 -15 0.64

Retention on ART (babies) 47 33 -14 0.90

* p<0.05

The FGDs showed that the improvements noted in Tables 7 and 8 and Figure 5 were mainly due to the establishment of QI committees and teams in trainees' facilities. During these meetings, the most commonly discussed issues were:

• Review of comments from the suggestion boxes. • Review of QI projects being implemented using various forms of data, focusing on:

o Completion of registers and importance of documentation. o HIV case management and disease-related issues (CD4, DBS sample collection, client follow-

up). o Operational challenges encountered at the clinic. o Feedback from HIV trainings and new approaches.

• Infection prevention and control. • Review of service delivery in other programs, such as malaria and Expanded Program of

Immunisation.

21

DISCUSSION The assessment achieved response rates above 90 percent for the OAs and semi-structured interviews. Considering that all the health facilities were assessed and almost all HCWs trained between May and July interviewed, these results represent precise parameters describing these study populations and changes occurring within them. Although FGDs had a lower response rate, most qualitative findings of the study from the FGDs tended to occur in themes and were therefore captured in the assessment, since these were done from each level and setting of the facilities.

The facilities supported by AIDSFree were mainly rural clinics and big district and mission hospitals. In these public health facilities, limited staff availability is a significant restricting factor in the delivery of services and implementation of health programs, including QM/QI. The majority of the respondents in the semi-structured interviews and FGDs reiterated this observation.

Nurses between 36 and 45 years of age, mostly without administrative functions, were the majority of trainees and direct beneficiaries of this QI project; thus they were the core implementers of subsequent QI programs in their facilities. Older nurses represented a minority of those trained, and were mostly the nurse managers. This is in line with the aims of the technical assistance, which was to capacitate the “shop floor workers” with QI skills, and to build a sustainable culture of quality improvement within the health facilities by providing some sensitization and training to nurse managers.

Scores on the extent of QM/QI implementation ranged from 0.97 to 1.94 on a 5-point continuous scale across all the QM/QI domains. Most of these facilities had not even begun QI implementation before the 2016 training began. At baseline, the mean domain scores were nearly all less than 1, and for some domains, 0. The facilities significantly improved the extent of QM/QI implementation in the post-training phase across all the QM/QI domains. Provision of systematic post-training support in 18 facilities significantly enhanced this improvement. This indicates that participating facilities started QI implementation in earnest and now have defined and identifiable structures and processes for implementing QI. AIDSFree was not able to follow each trained HCW with coaching and mentoring for QI because some of the nurses were from facilities that were not allocated to AIDSFree for sustained QM/QI support.

Only 3 percent of respondents were not able to use the knowledge and skills they obtained in the formal trainings upon returning to their facilities. Most respondents were moderately to intensely involved in QI activities within their health facilities, participating and providing input in quality-related structures; involving themselves in various QM/QI programming elements; and participating in QI implementation steps. Virtually all respondents (98%) found the training useful or very useful for implementing QI work, and were able to disseminate the knowledge gained in the trainings to their peers. This facilitated their leading roles in facility QI implementation and provided a basis for creating functional QI structures and teams within the health facilities. Most of the trained nurses were able to support all five of the steps in QI planning and implementation as described in the Zimbabwe QI guide and presented during the trainings. They were also able to identify the appropriate tools for

22

each step. This forms the core of QI implementation, and to a large extent explains the positive gains observed in the QM/QI and uptake indicators.

Ninety percent of HCWs reported receiving some post-training support. The type of on-the-job training provided to some facilities through AIDSFree is important in ensuring sustainable skills transfer and maximizing efficiencies in service delivery. FGD participants echoed the individual HCWs' reports.

The sole purpose of the HIV QI initiative is to improve patient outcomes among people living with HIV. This evaluation shows that service uptake improved in 8 of 13 QI indicators, with significant improvements in 6 indicators. Not all indicators improved, most likely due to the short implementation time before the post-training assessment. Though the assessment generally reflects the distribution of QI projects within participating facilities, it is important to note that some facilities had not started actual QI implementation by the time of the evaluation, and some did not receive post-training support from AIDSFree because MOHCC revised the allocation of sites to implementation partners early in the year. The difference in the extent of QM/QI implementation was clear when facilities were stratified by availability of post-training support; implementation was much stronger in facilities that received post-training support and mentoring.

Limitations Although program records were reviewed, the assessment also depended on data from retrospective self-reports, which are subject to measurement bias. Triangulation of these data with client reports would have strengthened the evaluation findings. Also, the review included facility-based records whose data quality is known to be suboptimal.

The measurement method used to test for differences in uptake before and after QI trainings was in line with the six-month fixed review periods recommended by the MOHCC, which are too wide and not very sensitive.

23

CONCLUSIONS AND RECOMMENDATIONS AIDSFree's QI capacity-building project was designed to build the capacity of 59 HCWs from 38 facilities to develop and implement QI activities within their HIV programs. The majority of HCWs trained through the project were still working within their original facilities after the intervention, in departments that enabled them to use the knowledge and skills they gained during the training. Consequently there were significant improvements in the extent of QM/QI implementation in these facilities. These improvements were more marked in the subset of facilities that received structured support and mentorship from the AIDSFree coaches. Overall, these sites now have structures and processes for implementing QI; however, they require more time and support to fully implement these measures.

There were substantial improvements in 8 of 13 QI indicators after QI training. Statistically significant improvements occurred in six of these indicators: adherence assessment; DNA-PCR sample collection from HEIs; DNA-PCR testing for HEI and returning of results; retention on ART in mothers; HIV retesting in ANC; and pediatric cotrimoxazole initiation. Indicators that did not improve—ART initiation in ANC; tuberculosis screening of ART clients; pediatric ART for <15 years; ART initiation for HIV-positive infants; and retention on ART of babies—require more focus and time.

Shortages of staff limited trainees' subsequent application of QI skills and knowledge. This also made focusing on QI work difficult. Different post-training implementation and support approaches also limited improvements.

Recommendations • The MOHCC should formally train all facility-level HCWs in QI using standard QI implementation

and coaching and mentoring tools, followed by systematic and structured post-training QI coaching and mentoring to accelerate and receive the full benefits of QI implementation. The MOHCC should explore and implement strategies to address staff shortages. This is now an absolute bottleneck in service delivery and patient outcomes. Partners need to support the MOHCC in mobilizing resources to support sustained QI coaching following formal training.

• Facility-level HCWs should meet frequently in QI committees to plan and direct QI work, build functional and effective teams, and give staff time to review health service provision.

25

ANNEX 1. FACILITIES WITH HEALTH CARE WORKERS TRAINED ON QM/QI

Province District Facility Number of trained HCWs

Manicaland

Mutare

Mutare City 3 Mutare City Health Centre 1

Marange 2 St. Joseph’s 2 Zvipiripiri 1

Mutasa Old Mutare 2

Buhera Murambinda 2

Munyanyi 1 Birchnough Bridge Hospital 2

Chimanimani

Biriwiri 2 Chakohwa 1

Chimanimani Hospital 1 Mutambara Hospital 1

Chipinge St. Peters 2

Chipinge Town 1 Mt. Selinda 2

Makoni Mayo 2 1

Masvingo

Bikita Odzi 1

Bikita Rural 1 Chikuku 1

Chiredzi

Chiredzi Hospital 2 Hippo Valley 2

Chizvirizvi 1 Chikombedzi 2

26

Province District Facility Number of trained HCWs

Chivi Nyahombe 1

Ngundu 2 Chirogwe 1

Gutu Cheshuro 1 Matizha 1

Masvingo

Masvingo Provincial 3 Mucheke 2

Mapanzure 1 Musvovi 1 Bondolfi 1

Mwenezi Matibi 3 Nehanda 1

Zaka Bota 2

Chinyabako 1

Total 13 38 59

27

ANNEX 2. NATIONAL QI INDICATORS Se#

Indicator Type Indicator Eligible Population [Program Standard

of Care]

Denominator [Sample Size]

Numerator Data Source

ART indicators

1 Retention of patients on ART

Proportion of patients retained on ART at this site

ART patients who visited the clinic in 6 mos. prior to the review period [Standard: 100% of ART patient seen for follow-up at least once per 6 mos.]

Number of patients who visited clinic in the 6 mos. prior to the review period

Number of patients who visited the clinic at least once during 6 mos. review period

ART register Patient file

2 Adherence assessment

Proportion of patients who had an ART adherence assessment at the most recent visit

ART patients seen during review period [Standard: 100% of ART patients have adherence assessment performed at each clinic visit]

Number of patients on ART with at least one visit during the review period

Number of patients with ART adherence assessment at the most recent visit


3 CD4 monitoring

Proportion of patients who had a CD4 test performed

Patients on ART seen during review period [Standard: 100% of ART patients who have CD4 test performed every 6 mos.]

Number of patients on ART due for 6 mo. CD4 test during the review period

Number of patients with a CD4 test performed during the review period


4 TB screening

Proportion of ART patients screened for TB at most recent visit

Patients on ART seen during the review period [Standard: 100% of ART patients

Number of patients on ART with one visit during the review period

Number of patients on ART who were screened for TB at the most recent visit (5-question assessment)


28

Se#


of Care]



screened for TB at each clinic visit]

5 Pediatric ART

Proportion of HIV+ and eligible children (<15 yrs.) who were initiated on ART during the reporting period

HIV+ (confirmed) and eligible children 0–15 [Standard: 100% of HIV+ eligible clients initiated on ART e.g., all HIV+ <5 yrs.]

Number of HIV+ children who are eligible for ART with one visit during the review period

Number of eligible children initiated on ART

Pre-ART OR pediatric register Patient file

1 ART initiation in ANC

Proportion of pregnant women who are HIV+ and initiated on ART on the same day of identification

Pregnant women who are known to be HIV+ but not on ART [Standard: 100% of HIV+ pregnant women not on ART initiated on ART]

Number of known HIV+ pregnant women not on ART at booking

Number of HIV+ pregnant women who are initiated on ART on the same day as booking date

ANC register (columns 12, 13, 14, 15, 31, and 32)

2a

Early infant HIV diagnosis

Proportion of infants born to HIV+ women who had a DNA-PCR sample within 2 mos. of birth

All HIV-exposed infants (HEIs) <2 mos. [Standard: 100% of HEI have DNA-PCR sample collected before 2 mos.]

Number of HEI <2 mos. who are enrolled in HIV-exposed infant follow-up during review period

Number of HEI ≤2 mos. who had DNA-PCR sample collected

Delivery and DNA-PCR registers

2b

Proportion of HEI who had a DNA-PCR sample collected within 2 mos. of birth and received result within 1 mo.

All HEI who had a DNA-PCR sample collected [Standard: 100% of HEI receive DNA-PCR test result within 1 mo. of collection]

Number of HEI who had a DNA-PCR sample collected

Number of HEI who had a DNA-PCR sample collected within 2 mos. of birth and received results within a month of collection

Exposed infant DNA-PCR and clinic registers

3 ART initiation for HIV+ children

Proportion of HIV+ children <2 yrs. initiated on ART

All HIV+ children ˂2 yrs. [Standard: 100% of confirmed HIV+ children <2 yrs. put on ART]

Number of HIV+ children ˂2 yrs. (confirmed HIV+)

Number of HIV+ children ˂2 yrs. initiated on ART within 21 days of caregiver receiving results

DNA-PCR and ART registers Patient file Pre-ART (to link

29

Se#


of Care]



DNA-PCR to ART register)

4a

Retention in care

Proportion of HIV+ women retained on ART 6 mos. after initiation in ANC

All HIV+ women who initiated ART in ANC in 6 mos. prior to the review period. [Standard: 100% of HIV+ women initiated on ART in ANC retained in care 6 mos. after initiation]

Number of HIV+ women who initiated ART in 6 mos. prior to review period

Number of HIV+ women who initiated ART in ANC in 6 mos. prior to review period and had a visit during last 3 mos. of the review period at the clinic


4b

Proportion of HIV+ children retained on ART 6 mos. after initiation

All HIV+ children who initiated ART in 6 mos. prior to the review period. [Standard: 100% of HIV+ children initiated on ART retained in care 6 mos. after initiation]

Number of HIV+ children who initiated ART in the 6 mos. prior to the review period

Number of HIV+ infants who initiated ART 6 mos. prior to review period and visited the clinic at least once during the last month of the review period


5a HIV retesting to detect incident infection

Proportion of pregnant women who tested HIV- during the 1st and 2nd trimester who were retested for HIV during the period 32 wks. to onset of labor

All HIV- pregnant women at 32-wk. gestation to onset of labor.[Standard: 100% of initially HIV- women retested for HIV in the 3rd trimester]

Number of pregnant women testing HIV- during the 1st and 2nd trimester and are currently between 32 wks. gestation and onset of labor

Number of pregnant women who tested negative for HIV in the 1st and 2nd trimester who were retested for HIV from 32 wks. gestation to onset of labor

ANC and HTC registers

30

Se#


of Care]



5b

Proportion of HIV- postpartum women tested for HIV and received result

All HIV- postpartum women [Standard: 100% of initially HIV- women retested for HIV in the postpartum period]

Number of HIV- postpartum women enrolled in postpartum care

Number of HIV- postpartum women retested for HIV

PNC and HTC registers

6 Cotrimoxazole initiation in HEIs

Proportion of HEI <2 mos. initiated on cotrimoxazole

All HEI ˂2 mos. [Standard: 100% of HEI <2 mos. initiated on cotrimoxazole]

Number of HEI ˂2 mos. enrolled in follow-up care during the past 6 mos.

Number of HEI ˂2 mos. enrolled in follow-up who have been initiated on cotrimoxazole

Delivery and infant dispensing registers

31

ANNEX 3. SEMI-STRUCTURED INTERVIEW QUESTIONNAIRE FOR HEALTH CARE WORKERS TRAINED IN QUALITY IMPROVEMENT UNDER THE AIDSFREE MECHANISM AIDSFree Zimbabwe

Questionnaire Number: …………. Date of interview: DD/MM/YY

1. Name of health facility: _____________________________________________________ 2. Type of health facility

[ ] Clinic [ ] Small hospital [ ] Med-Big hospital

3. Setting of the health facility: [ ] Urban [ ] Rural

Written consent obtained: [ ] Yes Consent number: …………….. [ ] No If written consent has not been obtained, thank the participant and stop at this stage As mentioned earlier in the consent process, my name is …………………… and my colleague’s name is………………………………. We are from the Ministry of Health and Child Care and the Elizabeth Glaser Pediatric AIDS Foundation. We are carrying out a study to determine the extent of QI implementation by health workers after the formal QI trainings done in May and June. Findings from this study will be used to inform planning for health care QI programming in Zimbabwe. In this session we are conducting semi-structured interviews with health workers trained on HIV quality improvement between May and June 2016 to assess the extent of their involvement in the quality improvement initiatives within their respective facilities. Your participation is voluntary and you are free to withdraw from this discussion at any stage of this process. The information you will provide will not be shared with anyone outside the research team. Individual respondents will not be identified by name in any analyses or reports. Responses will be aggregated and reported as summary statistics only.

32

Section A: Respondent Background Details

1. What is your age?:____________ (Record age of respondent in completed years) 2. (Observe and record the sex of respondent) [ ] male [ ] female 3. Which department do you work in? (Ask for respondents working at hospitals)

[ ] Administration [ ] OI/ART [ ] MNCH [ ] TB [ ] Maternity [ ] OPD [ ] General Wards [ ] Other

4. What is your designation?

[ ] District or Government Medical Officer [ ] District Nursing Officer [ ] Nurse in Charge [ ] Midwife (Non-manager)

[ ] Registered General Nurse [ ] Primary Care Nurse [ ] Health information personnel (HIO/HIA/DEC)

5. How long have you been employed in your current capacity at this facility? __________ (Record time in years)

6. When did you first train for PM or QI? _______/________/________ 7. Were you trained for QI between May and June? [ ] Yes [ ] No

Section B: Involvement of Respondent in Facility-Level QM/QI

i. Quality Management 1. Which quality-related committees teams do you sit on in this facility?

[ ] Quality Management Committee [ ] IPC Committee [ ] MNCH/Perinatal Mortality/Review [ ] QI team(s) for specific projects

[ ] Other 1b. If you check the QI teams box record the number of teams on which respondent sits ______ 2. Which aspects of QM/QI implementation have you been involved in? (Participant is allowed to

mention several aspects) [ ] Setting up and implementation of the Quality Management Committee [ ] Conducting the baseline Performance Measurement and Organizational Assessment [ ] Identifying priorities for improvement [ ] Setting up Quality Improvement teams [ ] Working in a QI team or teams on specific QI projects [ ] Setting up and implementing client feedback mechanisms [ ] Coaching and mentoring of other health staff on QM and QI (structured QI Coaching) [ ] Implementation of peer learning platforms [ ] Recording and documenting QI initiatives [ ] Reporting (Compiling QI reports)

33

ii. Quality Improvement 1. Which steps of implementation of QI projects have you been involved in this facility? (Participant is

allowed to mention several aspects) [ ] Setting priorities for QI/ identification of QI projects [ ] Defining performance measurement methods for QI project/s [ ] Establishing a QI team [ ] Understanding the process of underlying system of care [ ] Making changes to improve care [ ] Measuring the effect of changes

Section C: Understanding of QI Tools and Processes

1. Can you describe the tools and processes you used at the different steps of QI implementation? Break down and record these according to themes in the table below

Step Process Tools Used

Identifying QI projects

Which data/measures to use

Criteria to use

Establishing the QI team(s)

Deciding who goes into the different QI teams

Securing participation of the nominated members

Understanding the processes underlying the system of care

How was this done?

Making changes to improve care

Designing “tested changes”

Implementing the “tested changes”

Testing the effect of the changes

34

2. Which tools did you find most useful and easy to use? a. __________________________________________________________________________ b. __________________________________________________________________________ c. __________________________________________________________________________ d. __________________________________________________________________________ e. __________________________________________________________________________

Section C: Usefulness of Training and Availability of Post-Training Support

1. Did you find the training you had in the QI training between May and June useful in implementing QI at your facility? [ ] Not useful [ ] Somewhat useful [ ] Not sure [ ] Useful [ ] Very useful

2. Have you been able to disseminate the knowledge you obtained at the QI training? [ ] Yes [ ] No

2b. If yes please explain: a. ______________________________________________________________________ b. ______________________________________________________________________ c. ______________________________________________________________________ d. ______________________________________________________________________

3. Which aspects of the training did you find most useful? (Indicate the top three aspects mentioned by the respondent) [ ] Quality improvement principles [ ] Quality improvement steps [ ] Performance measurement in including QI indicators [ ] Abstraction process and use of the QI database [ ] Use of problem solving tools to identify root causes [ ] Designing and implementing tested changes [ ] Managing resistance to change [ ] QI documentation and reporting

4. What post-training support have you received? [ ] None [ ] Regular QI coaching and mentoring by the QI coach [ ] QI coaching by national level QI team [ ] Participation in peer learning sessions [ ] Health facility organized quality-focused meetings [ ] Other Please specify: _______________________________________________

35

Section D: Facility-Level Barriers and Enablers for QI Implementation

1. What has been most helpful to you when trying to/or applying your knowledge and learned techniques in implementing QI within the facility? a. ______________________________________________________________________ b. ______________________________________________________________________ c. ______________________________________________________________________ d. ______________________________________________________________________

2. Have you had any challenges in applying your knowledge and learned techniques to implement QI within the facility or in your department? [ ] Yes [ ] No If yes please explain: a. _______________________________________________________________________ b. _______________________________________________________________________ c. _______________________________________________________________________ d. _______________________________________________________________________

3. What strategies have you adopted in order to improve your application of knowledge and learned techniques implementation of QI within the facility? a. _______________________________________________________________________ b. _______________________________________________________________________ c. _______________________________________________________________________ d. _______________________________________________________________________

4. What has worked best in improving your implementation of QI within the facility? a. _______________________________________________________________________ b. _______________________________________________________________________ c. _______________________________________________________________________

Section E: Changes to Patients’ Care and Experiences As a Direct Result of QI Interventions

1. Do you have (any) instance(s) in QI work that you were involved in that lead to a positive change in patients care and/or experience? [ ] Yes [ ] No

1b. If yes, please describe at least 2 of these instances: a. ________________________________________________________________________

________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________

b. ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ Verify these stories with available documentation

End of Questionnaire Thank you!

37

ANNEX 4. FOCUS GROUP DISCUSSION GUIDE FOR QUALITY MANAGEMENT COMMITTEE MEMBERS AIDSFree Zimbabwe

Assessment of QI Trainings Conducted Under the AIDSFree Mechanism

1. Instructions • Please use this to guide the Focus Group Discussion (FGD) with members of the QI committees

at the selected health facilities. • Please introduce yourself and the study. • Please go through the informed consent process. • Please remind participants that they are under no obligation to answer any or all questions.

2. Basic information

Date District Facility Name Number of Participants

3. Introduction

My name is …………………… and my colleague’s name is………………………………. We are from the Elizabeth Glaser Pediatric AIDS Foundation. We are carrying out a study to determine the extent of QI implementation by health workers after formal QI training. Findings from this study will be used to inform planning for health care quality improvement programming in Zimbabwe. In this session we are conducting a Focus Group Discussion with you (members of the Quality Management Committee) to hear your views on quality improvement implementation is happening. Your participation is voluntary and you are free to withdraw from this discussion at any stage of this process. The information you will provide will not be shared with anyone outside the research team. We may capture specific statements during this discussion but if incorporated in the final report they will be captured without contributor’s names.

38

4. Guiding questions and prompts

Quality Management

1. Who sits in the quality management committee? List the offices/designations of the members of the quality management committee.

Probe for the following: Member of the DHE Member of Hospital Executive HOD within the facility Member of the community (Health Centre Committee, PLWHIV, CBHW)

2. As members of the Quality Management Committee for this facility, how have you been meeting? Probe for the following:

Average frequency of meeting in the past 6 months Venue for meetings Times for the meetings Length of meetings Attendance of meetings by the members Facilitation by QI coaches during the meetings Minuting of these meetings

3. What have you deliberated on during these meetings? Please tell us more. 4. Probe for the following:

Identifying key focus areas for improvement Development of annual Quality Management Plans Setting up of QI teams to work on specific QI projects Setting up of client feedback mechanisms Reviewing the work of the different QI teams Reviewing service delivery

o Case reviews o Case audits o Reviewing outputs from Quality Management committees

Other quality implementation issues

5. How would you rate your support for the QI initiatives within this facility?

Excellent Good Okay, but can be improved Not satisfactory We have not supported QI initiatives at all

39

6. What has worked well in your support to the implementation of QI initiatives within this facility? Probe for the following:

Prior training in QI Support/Facilitation from the QI coaches Availability of resources and time Participation in the QI peer learning sessions Being clear about priorities for QI within the facility Having an annual Quality Management plan Having supportive team with clear roles

7. What have been the biggest challenges to your ability to support implementation of HIV QI initiatives? Please tell us more. Probe for the following:

Not having enough time Not knowing what to support Not knowing how to support Not having resources Inadequate staff No clear reporting lines for QI work Unavailability of implementation level tools to track progress Lack of recognition

8. How would you rate your skills and knowledge as the Quality management committee of this facility? Follow up with the probes below.

Explore strengths with the committee (frequency of meetings, content of minutes) Do you feel you have adequate skills and knowledge to support quality improvement for HIV

care? Where do you feel you need support to improve your oversight role?

9. What changes would you like to see in the implementation of the HIV quality improvement initiative? Probe for the following.

Peer learning Coaching and mentoring Consumer involvement Performance measurement QI cycles

41

ANNEX 5. EXTENT OF ESTABLISHMENT OF QM STRUCTURES WITHIN THE CAPACITATED FACILITIES

MOHCC Organizational Quality Assessment Tool Adapted from HEALTHQUAL International

Purpose of the Organizational Assessment

Sustained improvement activities require attention to the organizational Quality Management Program (QMP), in which structures, processes, and functions support measurement and improvement activities. Development, implementation and spread of sustainable QI requires an organizational commitment to quality management. Organizational structure is fundamental to QI success, and involves a receptive health care organization, sustained leadership, staff training and support, time for teams to meet, and data systems for tracking outcomes. This structure supports quality initiatives that apply process improvement including: reliable measurement, root cause analysis and finding solutions for the most important causes identified.

This assessment identifies all of the important elements associated with a sustainable QMP. Scores from 0 to 5 are defined to identify gaps in the QMP and to set organizational priorities for improvement. The scoring structure measures program performance in specific domains along the spectrum of improvement implementation. When assigning a score of 0 to 5 for individual components, select the number that most accurately reflects organizational achievement in that area. You must meet all of the elements associated with a particular number in order to receive that score. If all of the boxes are not checked within one particular score section, then the score should be the number preceding that one. To score “2” for example, each box for the elements corresponding to that score section must be checked. If there is any uncertainty in assessing whether performance is closer to the statement in the next higher or next lower range, choose the lower score. Applied annually, this assessment will help a program evaluate its progress and guide the development of goals and objectives. Note that you may decide to check boxes for criteria in some of the higher scores and use that information to address gaps in the program that will help you meet the higher score.

42

The OA is implemented in two ways: 1) by an expert QI coach or 2) as a self-evaluation. The results are ideally used to develop a workplan for each element with specific action steps and timelines guiding the planning process to focus on priorities, setting direction and assuring that resources are allocated for the QMP. Whether performed by a QI coach or applied as a self-evaluation, key leadership and staff should be involved in the assessment process to ensure that all key stakeholders have an opportunity to provide important information related to the scoring.

Results of the OA should be communicated to internal key stakeholders, leadership and staff. Engagement of organizational leadership and staff is critical to ensure buy-in across departments, and essential for translating results into improvement practice.

Improvement activities should be aligned with National Quality Improvement strategies, where applicable.

Note: for small centers with few staff, a formal committee or project team may not be necessary to complete the functions described in this assessment. In these organizations, the entire staff should be considered the “committee” or the “team” that is involved in improvement activities.

A. Quality Management

GOAL: To assess how the organizational Quality Management Plan support a systematic process with identified leadership, accountability and dedicated resources.

Three components form the backbone of a strong sustainable QMP: Leadership, Quality Planning, and a Quality Committee.

Leadership

Senior leadership staff are defined by each organization since titles and roles vary among organizations. Clinical programs should include a clinical leader and an administrative leader. Larger programs may include additional leadership positions. There may be other informal leaders in the organization that support quality activities, but these are not included in this section. When reviewing the criteria for each score, consider the clinical or administrative leader who is responsible for the quality management program or is most closely associated it with it if there is no one officially designated for this function. Ideally, this person should be a hospital or health center senior leader who has the authority to convene committees and approve actions that are important to implement the quality management program.

Leaders establish a unity of purpose and direction for the organization and work to engage all staff, patients and external stakeholders in meeting organizational goals and objectives, this includes motivation that promotes shared responsibility and accountability with a focus on teamwork and individual performance. Organizational leaders should prioritize quality goals and improvement initiatives for the year, and establish accountability for performance at all organizational levels. The benefits of strong leadership include clear communication of goals and objectives, where evaluation, alignment and implementation of activities are fully integrated.

Evidence of leadership support and engagement includes establishment of clear goals and objectives, communication of program/organizational vision, creating and sustaining shared values, and providing resources for implementation.

43

Quality Committee

A quality committee drives implementation of the quality plan and provides high-level comprehensive oversight of the quality program. This involves reviewing performance measures, developing workplans, chartering project teams, and overseeing progress. Teams should be multidisciplinary and include a client when feasible. The committee should meet monthly, document their activities, and share meeting notes with committee members and other organizational staff and key stakeholders. For smaller organizations, the entire staff may be the QI committee and should be considered as such, as they perform all of the functions of the Quality Management Program.

Quality Plan

A quality management plan documents programmatic structure and annual quality program goals. The quality plan should serve as a roadmap to guide improvement efforts, and include a corresponding workplan to track activities, monitor progress and signify achievement of milestones.

A.1. To what extent does senior leadership create an environment that supports a focus on improving the quality of care in the organization?

Getting Started 0 Senior leaders are not visibly engaged in the quality of care program

Planning and Initiation

1

Leaders are: Primarily focused on reporting requirements Inconsistent in use of data to identify opportunities for improvement Not fully involved in improvement efforts Not fully involved in quality meetings Not supporting provision of resources for QI activities, including dedicated time for improvement

Beginning Implementation

2

Leaders are: Engaged in quality of care with focus on use of data to identify opportunities for improvement Somewhat involved in improvement efforts Somewhat involved in quality meetings Supporting resources for QI activities but not yet at optimal levels to support improvement

Implementation

3

Leaders are: Providing routine leadership to support the quality management program Providing routine and consistent allocation of staff or staff time for QI (depending on organization size) Actively engaged in QI planning and evaluation Actively managing/leading quality committee meetings Clearly communicating quality goals and objectives to all staff Recognizing and supporting staff involved in QI Routinely reviewing performance measures and patient outcomes to inform program priorities and data use for improvement. Attentive to national health care trends/priorities that pertain to the program

44

Progress Toward Systematic Approach to Quality

4

Leaders are: Supporting development of a culture of QI across the program, including provision of resources for staff participation in QI learning opportunities, seminars, professional conferences, QI story boards for distribution Supporting prioritization of quality goals based on data, and critical areas of care Promoting patient-centered care and patient involvement through the QMP Routinely engaged in QI planning and evaluation Routinely providing input and feedback to QI teams

Full Systematic Approach to Quality Management in Place

5

Leaders are: Actively engaged in the implementation and shaping of a culture of QI across the program, including provision of resources for staff participation in QI learning opportunities, seminars, professional conferences, QI story boards Encouraging open communication through routine team meetings and dedicated time for staff feedback Routinely and consistently engaged in QI planning and evaluation Routinely and consistently providing input and feedback to QI teams Encouraging staff innovation through QI awards and incentives Directly linking QI activities back to institutional strategic plans and initiatives

A.2. To what extent does the organizational program have an effective quality committee to oversee, guide, assess, and improve the quality of services?

Getting Started 0 A quality committee has not yet been developed or formalized or is not currently meeting regularly to provide effective oversight for the quality program


1

The quality committee: May review data triggered by an event or problem, or generated by donor or Ministry of Health urging Has not yet developed a systematic process for data use to identify and prioritize annual goals Has not yet defined roles and responsibilities for participating individuals


2

The quality committee: Has plans to hold regular meetings, but meetings may not occur regularly and/or do not focus on performance data Has been formalized, representing most institutional departments Has identified roles and responsibilities for participating individuals including the QI focal person Has not yet implemented a structured process to review data for improvement

Implementation

3

The quality committee: Is formally established and led by the organization’s director or manager as chair Represents most departments and disciplines The quality committee has established annual calendar of meeting dates Has defined roles and responsibilities as codified in the quality plan including the QI focal person Reviews performance data at each meeting Discusses QI progress and redirects teams as appropriate Introduces early stages of ground rule management and efficiency tools during meetings

45


4

The quality committee: Is formally established and led by an organizational director or manager as chair who actively oversees the work of the quality program with established annual meeting dates Represents all departments and disciplines Has established a performance review process to regularly evaluate clinical measures and respond to results as appropriate, including staff and patient satisfaction Communicates with non-members through distribution of minutes and discussion in regular staff meetings Actively utilizes a workplan to closely monitor progress of quality activities and team projects


5

The quality committee: Is a formal entity led by the organizational director or manager or by an individual as designated by the organization Represents all departments and disciplines Has defined roles and responsibilities as codified in the quality plan including a QI focal person Has established a systematic performance and review process, including structure, and process and outcomes measures. Is responsive to changes in treatment guidelines and external/national priorities, which are considered in development of indicators and choosing improvement initiatives Has fully engaged senior leadership who lead discussions during committee meetings Effectively communicates activities, annual goals, performance results and progress on improvement initiatives to all stakeholders, including staff and patients

A.3. To what degree does the organization have a comprehensive quality plan that is actively utilized to oversee quality improvement activities?

Getting Started 0 A quality plan, including elements necessary to guide the administration of a quality program has not been developed

Planning and Initiation 1

The quality plan: Is written but does not include the essential components necessary to direct an effective quality program (see level 3)


2

The quality plan: Is written for the HIV program only, and contains some of the essential components (see level 3) Is under review for approval by senior leadership, and includes steps for implementation Includes a designated point of contact to manage QM program communication within the organization and with the national program

Implementation

3

The quality plan: Is complete, defining all essential QI components. This includes goals and objectives, quality committee roles, responsibilities and logistics, performance measurement and review processes, annual goal identification and prioritization processes, QI methodology, communication strategy, patient involvement, and a program evaluation procedure. Includes a workplan/timeline outlining key activities of the quality program and improvement initiatives, including individuals accountable for each. The timeline is reviewed regularly by the quality committee and modified as necessary to achieve the identified goals. An organogram visually depicting the organizational quality management structure

46


4

The quality plan: Has been implemented and regularly used by the quality committee to direct the quality program Includes annual goals identified based on data generated through internal and external reviews, and engagement of the quality committee and staff to elicit priorities Includes a workplan/timeline outlining key activities in place and routinely used to track progress of performance measures and improvement initiatives, and is modified as needed to achieve annual goals Is routinely communicated to most stakeholders, including staff, patients, board members and the parent organizations, if appropriate Is evaluated annually by the quality committee to ensure that the needs of all stakeholders are met


5

The quality plan: Is written, implemented and regularly utilized by the quality committee to direct the quality program and includes all necessary components (see level 3) Includes regularly updated annual goals that were identified by the quality committee using data based on internal performance measures and externally required indicators through engagement of the quality committee and staff to identify priorities for improvement Includes the workplan/timeline outlining key activities in place Is routinely used to track progress on performance measures and improvement initiatives, and modified as needed to achieve annual goals Is communicated broadly to all stakeholders, including separate staff, patients, board members and the parent organizations, as appropriate Is evaluated annually by the quality committee and revised as needed to ensure that the needs of all stakeholders are met. Is adapted to changes in national policies and to ensure that the program continues to meet the changing needs of the patient as the evidence base and guidelines evolve

Opportunities/Gaps:

B. Workforce Engagement in the Quality Program GOAL: To assess awareness, interest and engagement of staff in quality improvement activities. Staff engagement in the quality management program at all organizational levels is central to the success of improvement activities. Engagement includes development and promotion of staff knowledge around organizational systems and processes to build sustainable quality management programs, such as internal management processes, operational barriers, patient interaction, and successful strategies and barriers to QI implementation. Ongoing training and retraining in QI methodology and practical skills reinforces knowledge and the building of workforce expertise around improvement. As staff progress along the continuum of QI sophistication, improvement is slowly integrated into routine work and practice, enhancing staff engagement in the process. Immediate access to improvement data for example, empowers staff to focus on key areas of care and build consensus around QI activities to improve patient outcomes. As QI becomes part of the institutional culture and team work progresses, staff embrace their respective roles and responsibilities, acquiring a sense of ownership and deeper involvement in improvement work.

B.1. To what extent are clinicians and staff routinely engaged in quality improvement activities and provided training to enhance knowledge, skills and methodology needed to fully implement QI work on an ongoing basis?

Getting Started 0

All of the staff (clinical and nonclinical) are not routinely engaged in QI activities and are not provided training to enhance skills, knowledge, theory or methodology or encouragement to identify opportunities for improvement and develop effective solutions

47


Engagement of core staff in QI (clinical and nonclinical): Is under development and includes training in QI methods and opportunities to attend meetings where QI projects are discussed

Beginning Implementation 2

Engagement of core staff in QI (clinical and nonclinical): Is underway and some staff have been trained in QI methodology Includes QI meetings attended by some designated staff

Implementation

3

Engagement of core staff in QI (clinical and nonclinical): Includes attendance in at least one training in QI methodology. Staff members are generally aware of Program QI activities (quality plan/priorities) Includes involvement in QI projects, project selection and participation in a QI committee Includes QI project development, where projects are discussed and reviewed during staff meetings Includes defined roles and responsibilities related to QI. Clinicians and staff are aware of the organizational quality management plan and priorities for improvement. Includes a formal process for regularly recognizing staff performance in QI via performance appraisals, public recognition during staff meetings, etc.


4

Engagement of core staff in QI (clinical and nonclinical): Is demonstrated by evidence that staff members are engaged and encouraged to use those skills to identify QI opportunities and develop solutions Involves a shared language regarding quality, which is evidenced in routine discussion Is described in the annual quality plan, and includes staff training and roles and responsibilities regarding staff involvement in QI activities Includes a formal process for recognizing staff performance internally. QI teams are provided opportunities to present successful projects to all staff and leadership.


5

Engagement of core staff in QI (clinical and nonclinical): Is defined by staff awareness of the importance of quality and continuous improvement, and their participation in identifying QI issues, developing strategies for improvement and implementing strategies Is evidenced by regular and continuous QI education and training in QI methodology Is reinforced by leadership who encourages all staff to make needed changes and improve systems for sustainable improvement including the necessary data to support decisions Involves formal and informal discussions where teamwork is openly encouraged and leadership shapes teamwork behavior Incorporates routine communication about new developments in QI, including promotion of QI projects both internally (e.g., quality conferences) and externally (e.g., national meetings) Includes a formal process for recognizing staff performance internally. QI teams are provided opportunities to present successful projects to all staff and leadership Includes opportunities for abstract development and submission to relevant professional conferences and authorship of related publications about development and implementation of institutional QM programs Involves clearly defined roles and responsibilities which are utilized to assess staff performance

C. Measurement, Analysis and Use of Data to Improve Program Performance

GOAL: To assess how the organization uses data and information to identify opportunities for improvement, develops measures to evaluate the success of change initiatives, to align initiatives with national priorities, and to monitor results; and to ensure that accurate, timely data and information are available to stakeholders throughout the organization to drive effective decision making.

48

The Measurement, Analysis and Use of Data section assesses how the organizational program selects, gathers, analyzes and uses data to improve performance. This includes how leaders conduct performance reviews to ensure that actions are taken, when appropriate, to achieve the organization’s program goals.

C.1. To what extent does the organization routinely measure performance and use data for improvement?

Getting Started 0 Performance measures: Have not been identified


1

Performance measures: Have been identified to evaluate some components of the organization’s program, but do not cover all significant aspects of service delivery Performance data: Collection is planned but has not been initiated


2

Performance measures: Are defined and used by staff in all applicable service delivery areas

Performance data: Analysis and interpretation of results on measures is in early stages of development and use Results are occasionally shared with staff and patients, but a structured process is not yet in place

Implementation

3

Performance measures: Are defined by the Ministry of Health or donor partner Are consistently used by staff in all applicable service delivery areas Performance data: Are longitudinally tracked, analyzed and reviewed with the frequency required to identify areas in need of improvement. A structured review process is used regularly by the leadership to identify and prioritize improvement needs and initiate action plans to ensure that goals are achieved. Are collected by staff with working knowledge of indicator definitions and their application Results and associated measures are routinely shared with staff and their input is elicited to make improvements Clinic has a process for checking the accuracy of its data occasionally but not systematically


4

Performance measures: Are tied to organizational goals and priorities Are defined and consistently used by staff in all applicable departments Performance data: Are reviewed for accuracy on all measures in all departments Are actively used to drive improvement activities Results and associated measures are frequently shared with staff to elicit their input and engage them in improvement processes aligned with organizational goals


5

Performance measures: Are selected using national/donor partner measures and organizational annual goals, with the intent to meet Ministry of Health requirements and the needs of stakeholders and patients Reflect organizational priorities and patients, in consideration of organizational & local issues Are defined for key component Are evaluated regularly to ensure that the program is able to respond effectively to internal and external changes quickly. Are linked to performance of key clinical outcomes

49

Performance data: Are reviewed for accuracy on all measures in all applicable departments Visible or easily accessible to ensure data reporting transparency throughout the organization Are arrayed in formats that enable accurate interpretation, such as run charts or simple bar graphs Results and associated measures are systematically shared with all key stakeholders, including staff and patients Are systematically reviewed through a Formal Data Quality Assurance program

Opportunities/Gaps:

D. Quality Improvement Initiatives

GOAL: To evaluate how the organization uses QI methodology and teamwork to achieve program goals and maintain high levels of performance over long periods of time.

The Quality Improvement Initiatives section examines how leadership and workforce use these methods and tools to conduct improvement initiatives with emphasis on identification of the exact causes of problems and designing effective solutions; determining program specific best practices and sustaining improvement over long periods of time. In high reliability organizations, robust process improvement methodology is routinely utilized for all identified problems and improvement opportunities to assure consistency in approach by all staff members.

D.1. To what extent does the organization identify and conduct quality improvement initiatives using QI methodology to assure high levels of performance over long periods of time?

Getting Started 0 Formal quality improvement projects have not yet been initiated in the organizational program


1

QI initiatives: Focus on individual cases without assessment of organizational performance or system level analysis of data. Reviews primarily used for inspection. Are not team-based Do not use specific tools or methodology to understand causes and make effective changes


2

QI initiatives: Are prioritized by the quality committee based on program goals, objectives and analysis of performance measurement data Involve team leaders and team members who are assigned by the quality committee or other leadership Begin to use specific tools or methodology to understand causes and make effective changes

Implementation

3

QI initiatives:

Are ongoing based on analysis of performance data and other program information, including external reviews and assessments

Focus on processes of care in which QI methodology is routinely utilized

Are regularly documented and provided to the Quality Improvement Committee

Involve staff on QI teams. Cross departmental/cross functional teams are developed depending on specific project needs. This would include laboratory, administrative and pharmacy staff where relevant.

50


4

QI initiatives: Are ongoing based on analysis of performance data and other program information, including external agency reviews and assessments Can be identified by any member of the program team through direct communication with program leadership Routinely and consistently reinforce and promote a culture of quality improvement throughout the program through shared accountability and responsibility of identified improvement priorities Are supported with appropriate resources, including people and time, to achieve effective and sustainable results Involve support of data collection with results routinely reported to QI project teams


5

QI initiatives: Are ongoing in core service categories Correspond with a structured process for prioritization based on analysis of performance data and other factors, such as patient surveys Are implemented by project teams. Further, physicians and staff can identify an improvement opportunity at any point in time and suggest a QI team be initiated Consistently and routinely utilize robust process improvement and multidisciplinary teams to identify actual causes of variation and apply effective, sustainable solutions Are guided by a team leader, and include all relevant staff depending on specific project needs Are regularly communicated to the Quality Committee, staff and patients Routinely involve patients on QI project teams Are presented in storyboard context or other formats and reported to the larger organization and/or placed in public areas for staff and patients (if relevant) Involve recognition of successful teamwork by senior leadership Are supported by development of sustainability plans

Opportunities/Gaps:

E. Patient Involvement

GOAL: This section assesses the extent to which patient involvement is formally integrated into the quality management program.

Patient involvement encompasses the diversity of individuals using the organization’s services and can be achieved in multiple ways, including solicitation of patient perspectives through focus groups, key informant interviews and satisfaction surveys; a formal patient advisory board that is actively engaged in improvement work; including patients as members of organizational committees; conducting patient needs assessments and including patients in specific QI initiatives. Ideally patients have a venue to identify improvement concerns and are integrated into the process to find solutions and develop improvement strategies. Overall, patients are considered valued members of the program, where patient perspectives are solicited, information is used for performance improvement and feedback is provided to patients. Patient experience is considered an important dimension of quality that is considered in determining improvement priorities and included as an important component of the quality management plan.

51

E.1. To what extent are patients effectively engaged and involved in the HIV quality management program?

Getting Started 0 There is currently no process to involve patients in HIV quality management program activities


Patient involvement is demonstrated by: Occasionally soliciting patient feedback, but no formal process is in place for ongoing and systematic participation in quality management program activities


Patient involvement is demonstrated by: Soliciting patient feedback, with development of a formal process for ongoing and systematic participation in quality management program activities, such as through patient satisfaction surveys

Implementation

3

Patient involvement is demonstrated by: Engagement with patients to solicit perspectives and experiences related to quality of care Formal involvement in quality management program activities through a formal patient advisory committee, satisfaction surveys, interviews, focus groups, storytelling and/or patient training/skills building. However, the extent to which patients participate in quality management program activities is not documented or assessed.


4

Patient involvement is demonstrated by: A formal process for patients to participate in quality management program activities, including a formal patient advisory committee, surveys, interviews, focus groups and/or patient training/skills building Three or more of the following activities: Sharing of performance data and discussing quality during formal patient meetings membership on the internal quality management team or committee training in quality management principles and methodologies engagement to make recommendations based on performance data results increasing documentation of how recommendations by patients are used to implement quality improvement projects Use of documented information gathered through the above activities to improve the quality of care. However, staff does not review with patients how their involvement contributes to refinements in quality improvement activities.


5

Patient involvement is demonstrated by: A formal, well-documented process for patients to participate in HIV quality management program activities, including a patient advisory committee with regular meetings, patient surveys, interviews, focus groups and patient training/skills building Quality improvement activities that include at least four of the items bulleted in E1#4 Information gathered through the above noted activities being documented, assessed and used to drive QI projects and establish priorities for improvement Review of changes by patients with program staff based on recommendations received with opportunities to offer refinements for improvements. Information is gathered in this process and used to improve the quality of care. Involvement on at least an annual basis in the review by the quality management team/committee of successes and challenges of patient involvement in quality management program activities, with the goal of enhanced collaboration between patients and providers engaged in improvement

Opportunities/Gaps:

52

F. Quality Program Evaluation

GOAL: To assess how the organization evaluates the extent to which it is meeting the identified program goals related to quality improvement planning, priorities and implementation.

Quality program evaluation can occur at any point during the cycle of quality activities, but should occur annually at a minimum. The process of evaluation should be linked closely to the quality plan goals: to assess what worked and what did not, to determine ongoing improvement needs and to facilitate planning for the upcoming year. The evaluation examines the methodology, infrastructure and processes, and assesses whether or not these led to expected improvements and desired outcomes. At a minimum, the evaluation should assess access to data to drive improvements, success of QI project teams; and effectiveness of quality structure. The evaluation is most effectively performed by program leadership and the program’s quality committee, optimally with some degree of patient involvement. Although external evaluations may be useful by peers or formal evaluators, the purpose of this assessment is focused on internal routine evaluation of the quality management program.

F.1. Is a process in place to evaluate the organization’s quality management plan and related activities, and processes and systems to ensure attainment of quality goals, objective and outcomes?

Getting Started 0 No formal process is established to evaluate the quality program


1 Quality program evaluation: To assess program processes and systems is exclusively external (National/donors/partners)


Quality program evaluation: Is part of a formal process and is integrated into annual quality management plan development, but has not been consistently employed

Implementation

3

Quality program evaluation: Occurs annually, conducted by the quality committee, and includes QM plan and workplan updates and revisions Involves annual (at minimum) revision of quality goals and objectives to reflect current improvement needs Results are used to plan for future quality efforts Includes a summary of improvements and performance measurement trends to document and assess the success of QI projects Results, noted above, are shared with patients and other key stakeholders


4

Quality program evaluation: In addition to the elements listed in F1.3, findings are integrated into the annual quality plan and used to develop and revise program priorities Is reviewed during quality committee meetings to assess progress toward planning goals and objectives Includes review of performance data, which is used to inform decisions about potential changes to measures Is used to determine new performance measures based on new priorities if they are identified Includes analysis of QI interventions to inform changes in program policies and procedures to support sustainability

Full Systematic Approach to Quality Management in Place 5

Quality program evaluation: In addition to the elements listed in F.1. 3 and 4, findings are integrated into routine program activities as part of a systematic process for assessing quality activities, outcomes and progress toward goals. Data and information from the evaluation are provided regularly to the quality committee. Is used by the quality committee to regularly assess the success of QI project work, successful interventions and other markers of improved care

53

Includes data reflecting improvement initiatives, and is presented to ensure comprehensive analysis of all quality activities Uses a detailed assessment process. The results of this assessment are utilized to revise and update the annual quality plan; adjust organizational program priorities; and identify gaps in the program. Includes an analysis of progress toward goals and objectives and QI program successes and accomplishments Describes performance measurement trends which are used to inform future quality efforts

Opportunities/Gaps:

G. Achievement of Outcomes GOAL: To assess HIV program capability for achieving excellent results and outcomes in areas that are central to providing high-quality HIV care. To determine whether a program is achieving excellence in HIV care, a system for monitoring and assessing clinical outcomes should be in place. This system should include routine analysis of an appropriate set of measures; trending results over time; stratifying data by high-prevalence populations and comparison of results to a larger aggregate dataset* used for programmatic target setting. A set of appropriate measures may be externally developed (national government, PEPFAR, WHO/UNAIDS) and/or internally developed based on program goals. Examples of outcome measures include viral load suppression, retention in care, mother-to-child transmission rates, and late diagnosis of HIV as measured by either CD4<200 or AIDS diagnosis at time of testing. At least one of these measures should be incorporated into the program’s set of clinical measures. *Possible data sets for comparison include national, provincial or partner network data sets.

G.1. To what extent does the HIV program monitor patient outcomes and utilize data to improve patient care?

Getting Started 0 No clinical performance results are routinely reviewed or used to monitor patient outcomes and guide improvement activities


Data: A clinical database is used to routinely measure performance of care (EMR, database, register) Some measures are routinely reviewed and used to guide improvement activities Trends for some measures are reported to determine improvement over time


Data: Results for most measures are routinely reviewed and used to guide improvement activities Trends for most measures are reported and many show improving trends over time

Implementation

3

Data: A listing of active patients is maintained and refreshed at least annually to remove those who have died, transferred or are lost to follow-up according to national definitions Results for all measures are routinely reviewed and used to guide improvement activities, including one of the following: viral load suppression (CD4 may be used as a proxy if viral load is not available), retention in care, late diagnosis, MTCT transmission rate Trends for all measures are reported and many show improving trends over time Results are compared to a larger aggregate data set for at least one outcome measure (see above) Comparison to a larger aggregate data set is used to set programmatic targets

54


4

Data: Results for all measures are routinely reviewed and used to guide improvement activities, including outcome measures Trends are reported for all measures and most show improving trends over time Results are compared to a larger aggregate dataset for two outcome measures Comparison to a larger aggregate dataset is used to set improvement goals which are met for at least 50% of measures


5

Data: Results for all measures are routinely reviewed and used to guide improvement activities, including outcome measures Trends are reported for all measures and most show sustained improvement over time in areas of importance aligned with organizational goals Results are compared to a larger aggregate dataset for all core national prioritized outcomes measures (such as retention, viral load suppression, etc.) Comparison to a larger aggregate data set is used to set programmatic goals which are met for at least 75% of measures Results for outcomes measures are above the 75th percentile of the comparative dataset

Opportunities/Gaps:

55

What are the major findings from the Organizational Assessment?

What are the key recommendations and suggestions? What specific areas should be improved? What are specific improvement goals for the upcoming year?

56

Organizational Quality Assessment Summary

Domain Score

0 1 2 3 4 5

A. Quality management

A.1. To what extent does senior leadership create an environment that supports a focus on improving the quality of care in the hospital?

A.2. To what extent does the hospital program have an effective quality committee to oversee, guide, assess, and improve the quality of hospital services?

A.3. To what degree does the hospital have a comprehensive quality plan that is actively utilized to oversee quality improvement activities?

B. Workforce engagement in the quality program

B.1. To what extent are clinicians and staff routinely engaged in quality improvement activities and provided training to enhance knowledge, skills and methodology needed to fully implement QI work on an ongoing basis?

C. Measurement, analysis and use of data to improve program performance

C.1. To what extent does the hospital routinely measure performance and use data for improvement?

D. Quality improvement initiatives

D.1. To what extent does the hospital identify and conduct quality improvement initiatives using QI methodology to assure high levels of performance over long periods of time?

E. Patient involvement

E.1. To what extent are patients effectively engaged and involved in the HIV quality management program?

F. Quality program evaluation

F.1. Is a process in place to evaluate the hospital’s QMP and related activities, and processes and systems to ensure attainment of quality goals, objective and outcomes?

G. Achievement of outcomes

G.1. To what extent does the HIV program monitor patient outcomes and utilize data to improve patient care?

57

ANNEX 6. CONSENT FORM An Assessment of the Application of Quality Improvement Programming Knowledge by Health Care Workers Trained under the AIDSFree Project in Zimbabwe

Principal Investigator: Agnes Mahomva (MBChB, MPH, MZCPHP)

You are being invited to participate in an assessment of the application of knowledge by health care workers trained under the AIDSFree Mechanism project in Zimbabwe. But before you decide whether to participate, it is important that you understand why this study is being done and what it will involve. Therefore, I will read out some information about this study, please take time to listen and feel free to ask if you need further clarification

Purpose

The purpose of the study is to determine the extent to which all Healthcare workers trained by EGPAF and MOHCC in quality improvement between May and July of 2016 applied the learned skills and techniques in implementing quality improvement initiatives within their respective health facilities as well as to explore some of the facility-level factors that might have made this easier or more. You were selected as a participant in this study because you were trained in QI training between May and July 2016.

Procedures and Duration

If you decide to participate, you will undergo a one-on-one interview with the interviewer. This is a private interview. During the interview, a series of questions will be asked pertaining to the quality improvement curriculum and the post-training implementation period. Most questions will be guided; that is, a set of responses will be provided for you to choose from. Some of the questions will be open-ended. These questions seek for your views and opinion. The interview will take approximately 20 minutes.

Risks And Discomforts

This research does not present any risk of any sort to you as a respondent. Only the results of the research will be published and cannot be traced back to the participant by your employer or any other interested party. Therefore you are encouraged to provide responses as honestly as possible.

Benefits and/or Compensation

Your participation is well valued, by participating in the research you are providing policymakers with knowledge and information to be used in the planning process for this and other similar trainings in the future. The results of this research will also serve as guidance to the funders of these trainings in informing sustainability of future programming. There are no direct benefits resulting from your participation; however you can take this opportunity to ask questions and share your concerns about the program as well as share your experiences.

58

Confidentiality

If you indicate your willingness to participate in this study by signing this document, we plan to share the findings of this research with MOHCC and other relevant stakeholders; this is for the benefit of patient care. Only the aggregate and de-identified results of this study will be published. Any information that is obtained in connection with this study that can be identified with you will remain confidential and will not be published or shared with third parties. Individual respondents will not be identified by name in any analyses or reports. Responses will be aggregated and reported as summary statistics only. The number printed on the survey is for purposes of questionnaire follow-up only. Only authorised personnel will be handling the study data for analysis purposes only. Under some circumstances, the MRCZ may need to review participants’ records for compliance audits.

Voluntary Participation

Participation in this study is voluntary. If you decide not to participate in this study, your decision will not affect your future relations with EGPAF, its personnel, and associated partners. If you decide to participate, you are free to withdraw your consent and to discontinue participation at any time without penalty.

Offer to Answer Questions

Before you sign this form, please ask any questions on any aspect of this study that is unclear to you. You may take as much time as necessary to think it over.

Authorization

You are making a decision whether or not to participate in this study. Your signature indicates that you have read and understood the information provided above, have had all your questions answered, and have decided to participate.

_______________________________ _____/_____/_________

Name of Research Participant (please print) Date

_______________________________ _________:_________

Signature of Participant or legally authorized representative Time

_______________________________

Relationship to the Participant

_______________________________ _______________________________ _____/_____/_________

Name of Staff Obtaining Consent Signature Date

_______________________________ _______________________________ _____/_____/_________

Name of Witness (if required) Signature Date

59

YOU WILL BE OFFERED A COPY OF THIS CONSENT FORM TO KEEP.

If you have any questions concerning this study or consent form beyond those answered by the investigator, including questions about the research, your rights as a research participant or research-related injuries; or if you feel that you have been treated unfairly and would like to talk to someone other than a member of the research team, please feel free to contact the Medical Research Council of Zimbabwe (MRCZ) on telephone (04)791792 or (04) 791193 and cell phone lines 0784 956 128. The MRCZ Offices are located at the National Institute of Health Research premises at Corner Josiah Tongogara and Mazowe Avenue in Harare.

Audio, Video Recording and Photography (to be used when recording FGDs’ discussions and a few selected settings for study pictures)

Statement of Consent to be photographed, audiotaped or videotaped.

I understand that photographs/audio recordings/video recordings will be taken during the study. (For each statement, please choose YES or NO by inserting your initials in the relevant box)

I agree to having my photograph taken

Yes ☐ No ☐

I agree to being audio recorded

Yes ☐ No ☐

I agree to having video recorded

Yes ☐ No ☐

[delete the options that are not appropriate for this study]

_______________________________ _______________________________ _____/_____/_________

Name of Participant (please print) Signature Date

AIDSFree 1616 Fort Myer Drive 16th Floor

Arlington, VA 22209 Phone: 703-528-7474

Fax: 703-528-7480 Email: [email protected] Web: aidsfree.usaid.gov

An Assessment of the Application of Quality Improvement ... · PDF fileAn Assessment of the...

Documents

Transcript of An Assessment of the Application of Quality Improvement ... · PDF fileAn Assessment of the...