Development of a National Monitoring and Evaluation Framework for WASH activities in

Malawi

Monitoring Sustainable WASH Service Delivery Symposium

April 9-11, 2013

Ashley Meek, Engineers Without Borders Canada

Young Samanyika, Ministry of Health, Malawi

Development of a National M&E Framework Since 2009 there has been a three phase collaboration

to develop a National M&E Framework:

Main Objectives : To address the needs of the various stakeholders To define and harmonize indicators To develop an easy to use information gathering and

analytical system; and To capitalise on the existing capacity of the users

Engineers Without Borders Canada (EWB)

Ministry of Health (MoH), Ministry of Water Development and Irrigation (MoWDI)Multiple Districts

Key Development Partners

Funding and decision making

Health Centre

HSA

Context At the District level,

M&E data collection has not historically been directly funded by the Ministries.

Data collection has predominantly been an individual district initiative where indicators are determined as needed by the district.

Several M&E initiatives and mapping exercises have been initiated but have not proven sustainable over time.

At the National level The MoWDI embarked on a Sector Wide Approach

(SWAp) in 2008 Sector Performance Report was initiated in 2010

Data proved difficult to obtain consistently

M&E Challenges (stimuli) Differences in data collection tools. Non-standard indicator definitions

between departments and organizations. Non-formalized accountability structures. Little or no prioritization on data

collection as compared to providing services.   Inadequate knowledge and skills in using

some computer software for data analysis. Roles and responsibilities not clearly

defined. Systems difficult to maintain and update.

Approach/Methodology

Timeline2009

• EWB began supporting the system initiative through a pilot in partnership with WaterAid Malawi and three District Water Development Offices

• Basic MS Excel system developed

2010

• The systems were replicated with support from EWB to nearly half of the districts in Malawi.

• MoWDI was approved for a grant to develop the SWSME. The project was delayed.

2011

• Malawi National Open Defecation Free (ODF) Task Force (chaired by the MoH) pursues the development of a comprehensive system for sanitation initiatives.

• EWB, the MoH began a WASH M&E pilot program including Sanitation and Hygiene indicators

2012

• MoWDI SWSME project re-initiated• EWB began consulting with the MoWDI to harmonize with the MoH WASH

M&E initiative

2013

• MoH pilot complete• MoWDI pilot including lessons learned from the previous phases.

Phase

I

Phase

II

Phase

III

Phase I In 2009: EWB began supporting the system initiative

through a pilot in partnership with WaterAid Malawi and three District Water Development Offices

Result : simple Microsoft Excel based system with data collection by Health Surveillance Assistants (HSAs). Village level water access was the main unit of analysis. The system used the pivot table function to perform simple data

analysis

By 2010, the original prototype showed promise and other districts as well as the MoWDI showed interest. The systems were replicated with nearly half of the districts

in Malawi.

Phase II reviewed WASH data collection and reporting

tools for various stakeholders and district councils

new reporting tools were developed data collection processes from village level

developed analysis templates and mapping functions

designed three learning workshops to review the

framework, indicators and required calculated indicators.

Phase III Collaboration between the MoWDI, MoH and

EWB under the SWSME with several additional focus areas. Expansion of capacity building for data collection,

analysis and reporting from the District Environmental Health Offices to include District and Regional Water offices.

Strengthening of indicators and creation of analysis templates for learning institutions, Health Centres and other public premises.

Establishment of Data Collection Methodologies for Peri-urban areas.

Explore data verification methods. Provide data for Sector Performance Report.

Data Management

Step 1. HSA completes community data form (2 pages) in their villages of responsibility.

Step 2. Data is entered into a simple MS-excel spreadsheet.

Step 3. Data automatically populates an analysis template built using pivot tables in MS-excel.

Step 4. Data calculated using the templates can be visualized on a district map outline directly within the MS-excel program. Each block represents a Traditional Authority within the District. A map can be generated for each indicator using a drop down menu.

Key findings and recommendations

Key Findings of the Framework

1. The monitoring system should be developed taking existing resource constraints into consideration to encourage systemic data collection.

2. Motivation wanes when there is no formal accountability structure.

3. WASH Data is used by multiple ministries, district departments and external parties.

4. The system must be simple and easy to use

5. Indicators collected need to be prioritized by use for service delivery decision making.

Key Findings of the Framework6. Indicators must be clearly defined7. There is a high probability for error in data.

There needs to be structures in place to verify authenticity of data.

8. Data collection and particularly data entry is a large time requirement that needs to be understood and planned for by management.

9. Resources constraints are a barrier to data collection activities.

10. Formalized communication structures are needed between the MoWDI and MoH regarding M&E

Conclusions The Malawi National M&E framework development has

enabled a system that is adaptable, easy to use, and relevant to the sector players who use it,

Integration of the various initiatives resulted in a Phased approach where the lessons learnt from each step can be built upon instead of being continuously repeated from scratch.

By building the Framework on existing capacity (including financial, human resource, and time constraints) of the users it has be easily transferable and utilized to make decisions and report on WASH service delivery.

Many steps remain to ensure the system moves from pilot phases to a National M&E Framework

Contact InformationAshley MeekEngineers Without Borders CanadaSanitation Coordinator ashleymeek@ewb.ca+265 994307077 Young SamanyikaMinistry of Health, Malawi Principal Environmental Health Officer (Ministry’s National Desk Officer for WASH)y_samanyika@yahoo.com+265 999557170