Monitoring and Evaluation of Community Systems Strengthening

8/2/2019 Monitoring and Evaluation of Community Systems Strengthening

http://slidepdf.com/reader/full/monitoring-and-evaluation-of-community-systems-strengthening 1/20

Monitoring and evaluation ofcommunity systems strengthening

Ms. Nathalie Zorzi

Manager, Monitoring and Evaluation Support TeamThe Global Fund18/19 March 2010



CSS indicator development process

• August 2008: The Global Fund commissioned a review exercise in Pretoria to defineindicators for community level service delivery and systems strengthening;

• January 2009: UNAIDS developed a guidance tool for including Community SystemsStrengthening in Global Fund proposals which includes a number of CSS indicators;

• November-December 2009: A multi-partner Technical Working Groups oversaw theimplementation of 9 field exercises which aimed at capturing best practices in

community level service delivery and systems strengthening including the set of CSSindicators used at country level;

• February 2010: a harmonization workshop was organized in Geneva where M&Eexperts from various organizations reviewed existing CSS indicators and developed anupdated list which was aligned with the proposed building blocks and SDAs of the newCSS framework;

• March 2010: An internal review and further definition of the indicators was done by theGlobal Fund. Comments from the Technical Working Group and other stakeholderswere integrated into this process.



Challenges - Review of CSS case studies

• Challenges at community level and CSOs- Community level workers have limited language

skills, multiple reporting, and high turnover.

- Electricity & internet have limited availability

- Ability to analyze the collected data for decision makingis limited – resulting in a perception of “purposelessdata collection”

- When you can’t make sense of your own data how can

you explain it to others in the community for theirmeaningful involvement?



Challenges

- Limited capacity at sub-national & national level for written feedback &supervision to show purpose in data

- Poor planning for trainings for M&E at all levels

- Poor correlation of program performance, logistics and finance (morepurposelessness)

- Thus, loss of opportunity of learning at community level

- Poor data analysis and strategy at national level for systematicinvolvement of CSOs

- CSOs are registered with departments other than health causingcounting difficulties



Approach

• Include both M&E for service delivery and CSS• Community level M&E means community level of bothCSOs and public health system

• Large CSOs follow M&E for both community level andhealth facility level according to the level of application

• Community level M&E to be integrated with national levelat the planning stage itself – Design jointly by various stakeholders – Harmonized list of indicators – Common reporting formats and data collection tools



Approach

• MoUs between disease program and CSOs – Avoids double counting – Gives baseline for calculation of % for indicators – Better understanding of roles & responsibilities – Facilitated by provision of guidelines from the national level for

involvement of CSOs

• Build capacity of community level in sharing information with thecommunity• Include meetings at community level and program review at various

levels• Include operational research at community level• Include CSOs in evaluations and joint reviews and - learning

opportunities for health facilities and CSOs



Approach

• Assuring data quality – Publication of M&E guidelines giving standardized data collection

tools and reporting formats

– Simple forms with minimal information at peripheral level

– Training & re-trainings of staff on M&E

– Guidelines on frequency and content of written feedback andsupervision

– Ensuring dedicated trained M&E staff at all levels for supervision(and their travel); ensure data entry staff

– Standardization of data base



Current status of CSS indicators (March 2010)

• Currently we have a list of 21 indicators with a description of the rationale,definition, numerator, denominator, proposed method of measurement, datasources and data collection frequency defined

• Page 21 of the CSS indicator document contains a number of additionalindicators for consideration

• The list of CSS indicators should be updated and finalized to be ready forGlobal Fund Round 10 (May 2010)



Building Blocks, SDAs and indicators (1)

BB1: Monitoring

& Evaluation

and Planning

SDA 1:

Monitoring

& evaluation

# and % of organizations working at the community level with a staff

member responsible for monitoring and evaluation (CSS-ME1.1)

# and % of organizations working at the community level using

standard data collection tools and reporting formats that enable to

report to the national reporting system (CSS-ME1.2)

# and % of organizations working at the community level submitting

timely, complete and accurate program reports according tonationally recommended guidelines (CSS-ME1.3)

# and % of organizations working at the community level conducting

reviews of their program performance in the past 3/6 months (CSS-

ME1.4)

SDA 2: Strategic

planning

# and % of organizations working at the community level with a costed

annual work plan in place which includes monitoring and

evaluation activities (CSS-SP2.1)

# and % of organizations working at the community level with at least

one staff member who received training or re-training in planning

and M&E according to nationally recommended guidelines in the

past 2 years (CSS-SP2.2)




BB2: Resources

and capacity

building

SDA 3: Community

actors: skills

building for

service delivery

# and % of community health workers currently employed who

received training or re-training in HIV, TB or malaria service

delivery according to national guidelines in the past 2 years (CSS-

SB3.1)

# and % of currently employed staff members of organizations

working at the community level that have been recruited more

than 1 year ago (CSS-SB3.2)

# and % of organizations working at the community level that receivedsupervision in accordance with national guidelines in last 3/6

months (CSS-SB3.3)

SDA 4: Financial

resources

# and % of organizations working at the community level with a

budget coming from more than one source (CSS-FR4.1)

# and % of organizations working at the community level that submittimely, complete and accurate financial reports according to

nationally recommended standards and guidelines (CSS-FR4.2)

# and % of organizations working at the community level that are

planning to implement resource mobilization activities (CSS-

FR4.3)




BB2: Resources

and capacity

building

SDA 5: Material

resources -infrastructure;

information;

essential

medical and

other products

& technologies

# and % of organizations working at the community level with the

minimum infrastructure in place to deliver quality HIV, TB and

malaria services (CSS-MR5.1)

# and % of organizations working at the community level that have not

experienced stock-outs of essential commodities in the last 3/6

months (CSS-MR5.2)

% of organizations working at the community level that keep adequate

logistics data for inventory management (CSS-MR5.3)

# and % of organizations working at the community level with staff

trained or re-trained in stock management in the past 2 years

(CSS-MR5.4)

# and % of organizations working at the community level that maintain

acceptable storage conditions and handling procedures (CSS-

MR5.5)




BB3:

Organisational

strengthening

SDA 6:Management,

accountability

and leadership

# and % of organizations working at the community level with staff

who have received training or re-training in the past 2 years on

management, leadership and accountability (CSS-MA6.1)

BB4: Community

networks,linkages &

partnerships

SDA 7: Building

community

linkages andcollaboration

for service

delivery


implementing activities in partnership with other organizations

working at the community level or with the national healthsystem (CSS-CL7.1)

BB5: Enabling

environment

SDA 8:

Communicatio

n and advocacy

for enabling

environments

and community

engagement


planning to implement communication or advocacy activities to

address barriers to equitable access for target populations

(CSS-CA8.1)

# of legal arrangements amended or put in place to build an enabling

environment for community engagement (CSS-CA8.2)



Issues for discussion

• Are the indicators useful for actors involved with CSS that aim at strengtheninga limited number of organizations at the community level?

• Are there any considerations regarding the feasibility of data collection for theproposed indicators?

• If the list of building blocks and SDAs will be amended will this have an impacton the proposed indicators and how do we address this?

• How do we ensure that all actors involved with CSS have access to theindicators?

• Are there any suggestions on the next steps in the CSS indicator developmentprocess?



Asia feedback1. Make the definitions more concise

1. Change from Community Systems Strengthening toCommunity SECTOR Strengthening (treat CSS as a strategy& not as a system)

2. Community Actors be changed to mean Key AffectedPopulations and grass-root community members.

2. The framework should focus on GF programming3. A new set of 7 building blocks with 14 SDAs have been proposed

4. The framework should be included in the ‘expanded’ version ofthe M&E toolkit

5. Technical support & assistance should be more local & peer-

based



Proposed BB & SDAs

BB1:Sustainable funding

SDA1: Sustainable financial resources

BB2:

Community sectorworkforce

SDA 2: Build community workforce andstrengthen overall capacity of new and

existing staff and volunteers

SDA 3: Build technical skills for servicedelivery (of CBO staff and volunteers)



Proposed BB & SDAsBB3:

Organizational

& management

Strengthening

SDA 4:

Organizational/management support andtraining for small NGOs/CBOs

SDA 5: Transparent financialmanagement structures

BB 4:

Planning and M&E

SDA 6: Needs assessment, gap

analysis, mapping

SDA 7: Evidence-based strategic planningamong community groups, linked to M&Esystems. (including OR)



Proposed BB & SDAsBB 5:

Communication,

Information &

Networking

SDA 8: Partnership building & collaboration,

including networks of KAP

SDA 9: Information and

knowledge sharing (including

mechanism for information

sharing, material development

and resources)

BB 6:

Accountability and

transparency

SDA 10: Models & tools for community accountability & transparency to itsconstituency



Proposed BB & SDAsBB 7:

Social

Mobilization &

Advocacy

SDA 11: Monitoring and

documentation (of both

community and government

interventions)

SDA 12: Advocacy, Campaigning &Community mobilization

SDA 13: Enabling environment

(policy, legal, governance)

SDA 14: Reducing stigma & discriminationin all settings



Thank you



Covered in this presentation

1. Indicators for community systems strengthening

2. Challenges of community level monitoring and evaluation

3. Approaches to overcome these challenges

4. Feedback on the CSS framework from Asia

Monitoring and Evaluation of Community Systems Strengthening

Documents

Transcript of Monitoring and Evaluation of Community Systems Strengthening