Post on 25-Feb-2022
[1]
Handout 1 Sustainability Planning Workshop Pre/Post-test [To be handed out on the first day of the workshop.]
PRE-/POST-WORKSHOP QUESTIONNAIRE SUSTAINABILITY PLANNING WORKSHOP
Please circle the ONE best answer. 1) Which of the following topics are NOT included in the six components of the
Sustainability Framework? a. Community capacity b. Program outcomes c. Financial capacity d. Health service delivery e. Organizational capacity 2) What are the five steps of the sustainability program design process? a. System situational analysis, system assessment, capacity building, program planning,
and monitoring & evaluation. b. Development of results framework, sustainability scenario planning, system assessment,
program planning, and monitoring & evaluation. c. System situational analysis, sustainability scenario planning, system assessment,
program planning, and monitoring & evaluation. d. System situational analysis, sustainability scenario planning, program planning, process
evaluation, monitoring & reporting 3) Which of the following statements about the local system is FALSE? a. The local system defines the level at which evaluation can take place in a meaningful
way. b. A local system is defined at the beginning of the project and does not change. c. The local system refers to the local stakeholders and communities brought together to
map out their vision and goals. d. A local system has boundaries; some people and groups are included and some groups
might have to be excluded. 4) “Children will not die of preventable causes. They will find quality care in well-managed
health centers and will be cared for by well-informed families.” This is a good example of a:
a. Component goal b. Sustainability scenario c. Program outcome indicator d. Strategic objective e. Vision statement 5) Which of the following is NOT an area that the sustainability scenario should begin to
broadly identify? a. Key strategies for attaining the vision. b. Specific activities that address each sustainability component. c. Broad roles/responsibilities of stakeholders.
[2]
d. Capabilities that are needed in order to fulfill the roles. e. Supportive environmental attributes that will help achieve the vision. 6) Which of the following are important sub-components of the enabling environment? a. Governance and civil stability b. Human development c. Health services d. a & b only e. All of the above 7) Which of the following statements about sustainability monitoring and evaluation is
FALSE? a. Many of the indicators selected for the local system assessment will also be included in
the M&E plan. b. The M&E plan should measure elements that are outside of the scope of the project. c. The local system assessment plan should focus on evaluative indicators; monitoring
indicators should not be included. d. Local system assessment is typically conducted two to three times over the course of a
project. 8) Which of the following is NOT an important criterion for a good indicator? a. Multidimensional b. Valid c. Reliable d. Precise e. Timely 9) A(n) __________ is calculated from a(n) _________ in order to measure the impact of
each __________. a. Scale, score, component b. Score, scale, indicator c. Scale, score, indicator d. Score, index, component 10) Name two important adult learning principles when facilitating the Sustainability Planning
Workshop.
[3]
Handout 2 Definition of Sustainability
An emerging property within a local system which
allows interdependent actors to maintain and
improve the health status of the system’s
(vulnerable) population through negotiated and
coordinated social interactions, allowing the
expression of their respective and collective
capabilities.
[4]
Handout 3 (2 pages) Introduction to the Sustainability Framework Find the triangle labeled “The Sustainability Framework.” Place it in the center of your table. Find the triangle labeled “Health Outcomes”. Place it on top of the Sustainability Framework triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “health outcomes” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.
Component 1, Health Outcomes, does not quite measure the health status of the
population—as this would require measuring indicators of morbidity and mortality. This
is not feasible for most projects. To be the most feasible while still giving the most
useful information, this component tries to come as close as possible to health status
measures. So, the usual data used for Component 1 is a summary of the population
health outcomes (i.e., indicators of key household behaviors and service coverage) that
are known to be associated with a high impact on health. These are the very measures
that many projects use to track their progress.
Find the triangle labeled “Health Service Delivery.” Place it along the right side of the “Health Outcomes” triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “health service delivery” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.
Component 2, Health Service Delivery, addresses how well the local health providers—
both facility and community-based—deliver services and products to the beneficiary
population (i.e., quality of care, use of health management and information systems, and
proportion of anti-malarial stock-outs). This service delivery contributes directly or
indirectly to the health outcomes measured in Component 1.
Find the triangle labeled “Ministry of Health Organizational Capacity & Viability.” Place it on the left side of the Sustainability Framework triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “MOH organizational capacity and viability” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.
Component 3, MOH Organizational Capacity and Viability, refers to the capacity and
viability of the organization that will supply the institutional support for health service
delivery. Capacity refers to the range of functions that are necessary to the life of an
organization, its administration, and its ability to perform its mission. Viability includes
financial viability, but also encompasses the more general idea of securing access to the
inputs necessary to sustain the level of capacity and performance attained during the
project. Some examples of MOH organizational capacity and viability include training
for maternal and child health interventions, supervision for community workers, and
financial resources for maternal and child health.
[5]
Find the triangle labeled “Local NGO Capacity & Viability”. Place it along the right side of the “Ministry of Health Organizational Capacity & Viability” triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “local NGO capacity and viability” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.
Component 4, Local NGO Capacity and Viability, refers to the capacity and viability of
the organization that will supply the institutional support for the activities in the
community necessary to demand services and for household behaviors related to
relevant health outcomes. Capacity and viability are defined the same as in Component
3. Some examples of local NGO capacity and viability include governance and legal
structure, financial management, technical capacity for maternal and child health
programming, and networking with partners.
Find the triangle labeled “Community Capacity.” Place it on the right side of the Sustainability Framework triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “community capacity” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.
Component 5, Community Capacity, refers to a relatively wide range of actors—from
individuals and households to social networks and the community as a whole—through
which community members will assume direct or supportive roles in achieving and
sustaining positive health outcomes. Some examples of community capacity include
organization, participation, and financial management.
Find the triangle labeled “Enabling Environment.” Place it along the right side of the “Community Capacity” triangle. Select one volunteer to read the grey box below. In your small group, discuss the meaning of the definition and write a new definition of “enabling environment” on the back of the triangle in your own words as it relates to HealthPartners’ Collaborations Project.
Component 6, Enabling Environment, refers to the essential social-ecological
environment variables that can either support or weaken gains in health. Some
examples of the enabling environment include health policy and government commitment
to expanding insurance coverage, human development, gender empowerment, and natural
environmental factors. Clearly, more can be achieved in a more favorable environment
than a less favorable one. If we are going to be accountable for progress toward
sustainability, we should be clear about the context in which the local system operates.
[6]
Handout 4 Defining the Local System
Local system: A network of people and institutions whose coordinated
actions will bring about sustainable positive health outcomes in a
population.
“Local system” refers to the local stakeholders and communities brought
together to map out their vision and goals for sustained health improvement in
the community. This local system also defines the level at which evaluation can
take place in a meaningful way. Examples of stakeholders in the Local System
include: villages, women’s associations, local authorities, rural development
associations, health district and health posts, local socially active NGOs, and
private sector partners.
An important question for project planners and stakeholders to ask is, “How
broad is the ‘local system’?” Consider answering this question in terms of three
aspects: (1) It is the level of bodies/stakeholders that can be feasibly brought
together; (2) it is the level at which assessment can be conducted (villages
surveyed, facilities assessed, institutions willing to examine their cooperation
and functioning); and (3) it is the level at which decisions can be made in
response to the sustainability assessment (for example, the national government
is usually not involved though its decisions might be very important for
components of the SF, in particular component 6).
A local system has boundaries: some people and groups are included; groups that
are too remote might have to be excluded; and some groups exclude themselves.
Finally, a local system can evolve: groups once excluded can be included as they
see the benefit of the project’s efforts. Ultimately, a stronger and more
cohesive local system can help sustain better and better health outcomes.
Source: Yourkavitch, J., et al. Lessons learned from applying the CSSA to seven Mother and Child Health Projects.
[7]
Handout 5 Stakeholder Involvement Analysis Matrix
P
OW
ER
/IN
FL
UE
NC
E O
VE
R V
ISIO
N/
MIS
SIO
N
L
OW
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
----
--H
IGH
INFORM & ANSWER CONCERNS
CORE OF THE SYSTEM CHANGE AGENTS
MINIMAL INVOLVEMENT
KEEP INFORMED & FIND OPPORTUNITIES FOR COOPERATION WITH LOW TRANSACTION
COST
INTEREST IN VISION/MISSION
LOW------------------------------------------------------------------------------------------HIGH
[8]
Handout 6 Examples of Vision Statements
Children will not die of preventable causes. They will find
quality care in well-managed health centers and will be
cared for by well-informed families.
All people living with HIV and TB will be able to live healthy
and productive lives. They will access quality health
services and information in a timely manner and within a
supportive environment.
A malaria-free community where people don’t die because of
malaria and incidences of malaria are significantly reduced.
Every community will take a leading role in malaria
prevention, have equal access to effective interventions,
and receive coordinated support from stakeholders led by
the Government.
[9]
Handout 7 Example of a Sustainability Scenario
Vision:
All people living with HIV and TB will be able to live healthy
and productive lives. They will access quality health
services and information in a timely manner and within a
supportive environment.
Sustainability Scenario:
To attain our vision of improved health and well-being of
people living with HIV and TB, we will provide high-quality
services by building the capacity of service providers in
prevention, care and treatment services; strengthen village
health teams’ capacity to provide counseling and referral
services; strengthen supply management systems; improve
community members’ awareness and knowledge of HIV and
TB and the services available to them by mobilizing
community groups, local village committees, and CBOs/FBOs
to provide outreach and education. We will advocate for
supportive policy to allow for more resources to target
services for vulnerable populations that are at a higher risk
for HIV.
[10]
Handout 8 Plan’s Results Framework
Results Framework: Kilifi KID-CARE Project
Intermediate Result 1.1
Improved nutrition
in children
Result 1.2
Increased malaria prevention
and improved treatment
Intermediate Result 1.3
Increased immunization coverage
Intermediate Result 1.4
Improved case management
of pneumonia
Intermediate Result 1.5
Increased awareness of HIV/AIDS
and preventive behaviors
Result 1
Improved household behaviors
and management of childhood illnesses
through IMCI
Intermediate Result 2.1
Dispensaries have health workers
trained and equipped to manage
childhood illness
Intermediate Result 2.2
Villages have CHWs that are adequately
trained and equipped to do prevention
activities and manage basic illnesses
Intermediate Result 2.3
Increased coverage of outreach services
to hard-to-reach areas
Intermediate Result 2.4
Equitable Health Services
to all OVCs
Result 2
Increased access to quality
maternal and child health
services to OVC
Intermediate Result 3.1
Improved capacity of the MOH to offer
consistent and quality child-centered
services
Intermediate Result 3.2
Improved capacity of DHCs and VHCs to
finance and carry out health activities
Intermediate Result 3.3
Improved capacity of local CBOs/NGOs
to carry out quality child survival
activities
Result 3
Improved capacity of local partners and
established systems and structures that allow
for sustained CS activities
Sustained reductions in morbidity and mortality
of children and WRA in Kilifi District, Kenya
[11]
Handout 9 Plan’s Sustainability Framework Dashboard
[12]
Handout 10 (2 pages) Plan’s Local System Assessment Plan (Modified from Plan Kenya’s KIDCARE Project)
Component Indicators Measurement Method/Frequency
Person Responsible
1. Health Outcomes
Percentage of children 0-23 months that are underweight
Knowledge, Practices, and Coverage (KPC) survey/baseline, mid-term and final
Project Monitoring & Evaluation (M&E) Specialist
Child spacing
Skilled attendant at birth
Tetanus Toxoid coverage
Exclusive Breastfeeding
2. Health Services
Health worker (HW) correctly assesses for danger signs
Health Facilities Assessment (HFA)/baseline, mid-term and final
Project Staff
HW gives appropriate treatment for fever
Caretakers know 2 aspects of home care
Caretakers knows 2 signs of getting worse
HW is trained
3. Ministry of Health Capacity
4. Ministry of Health Viability1
District Health Committees (DHCs) have ability to analyze and use information generated by project
Custom capacity assessment tool/baseline, mid-term, and final
Project Capacity Development Specialist
DHCs know how to disseminate knowledge about nine key household/community child survival behaviors
DHCs know essential package of activities and supplies of the health facility
DHCs have a clear vision and mission and understanding of those
DHCs have the ability to motivate, mobilize and organize community
5. Community Capacity
Community has the ability to collect baseline information on any project before embarking on any work
Custom community capacity assessment tool/baseline, mid-term, and final
Project staff
Care group members knows how to disseminate knowledge about nine key household/community child survival behaviors
Village Health Committee (VHC) has the ability to motivate, mobilize and organize the community for health action
1 It should be noted that the previous version of the Sustainability Framework separated out Organizational
Capacity and Viability. We now combine these two aspects, but separate out the type of organization to which it is applied.
[13]
VHC has a constitution that governs its operations
Community members were involved in the formation/creation of the VHC
6. Enabling Environment
Female literacy rate National statistics/baseline, mid-term, and final
Central government
Number of demonstrations, riots, clashes or strikes in the district
Local statistics/baseline, mid-term, and final
local police or district security committee
% of women with cash income (rate) KPC survey/ baseline, mid-term, and final
Project M&E Specialist
Water and sanitation - Latrine coverage
KPC survey/ baseline, mid-term, and final
Project M&E Specialist
State of secondary roads Local statistics/ baseline, mid-term, and final
Local municipal authority or department of public works or transport
Project stakeholders meet after each measurement point to discuss findings and determine course correction, if needed.
[14]
Handout 11 (6 pages) Data Availability for the Sustainability Framework
Component 1: Program Outcomes
Data already available Where other useful data
may be available
New data collection
instruments necessary
[15]
Component 2: Health Service Delivery
Data already available Where other useful data
may be available
New data collection
instruments necessary
[16]
Component 3: Organizational Capacity for Service Delivery
Data already available Where other useful data
may be available
New data collection
instruments necessary
[17]
Component 4: Organizational Capacity for Community Mobilization
Data already available Where other useful data
may be available
New data collection
instruments necessary
[18]
Component 5: Community Capacity
Data already available Where other useful data
may be available
New data collection
instruments necessary
[19]
Component 6: Enabling Environment
Data already available Where other useful data
may be available
New data collection
instruments necessary
[20]
Handout 12 Sample Vision and Component Goals
CRWRC’s Malaria Communities Program in Malawi
Our vision is a malaria-free community where people don’t die because of malaria and
incidences of malaria are significantly reduced by 2020. Every community will take a
leading role in malaria prevention; have equal access to effective interventions (e.g., LA,
SP, ITNs and environmental control); and receive coordinated support from
stakeholders led by the Government.
Component 1 Goal: Significantly reduce malaria related morbidity and mortality
amongst communities by 2020.
Component 2 Goal: Ensure equal access of preventive measures and early treatment of
malaria through uninterrupted provision of necessary services and supplies.
Component 3 Goal: District level malaria staff and local health facilities will be capable
to provide adequate supplies, financial resources and well trained and skilled personnel
to assist in malaria prevention and treatment.
Component 4 Goal: Ensure community empowerment and stakeholder networking
through management of God-fearing, capable and motivated staff who work together as
a team.
Component 5: Ensure that the community will take a leading role to mobilize, manage
and utilize resources to prevent and treat malaria.
Component 6 Goal: Have a conducive environment where civil society is involved, there
is good governance, and proper policies in place to enhance provision of effective
malaria interventions.
[21]
Handout 13 Local System Assessment Plan
Component Goal Indicators Data Sources/Frequency of
Measurement
[22]
Handout 14 (3 pages) HealthPartners Collaborations Project Monitoring & Evaluation Plan (Revised for Workshop)
Performance Indicator Title Indicator definition and unit of measurement Method of Data
collection Disaggre-
gation
Frequency of Data
Collection
Strategic Objective 30,000 WRA (85,000 total beneficiaries) have access to quality health care through financially sustainable community owned health co-ops.
1 # of people2 covered by USG-supported health financing arrangements
Component 1: Program Outcomes
Definition: Number of people reached with health cooperative financing program affiliated to CDP during the reporting period. Unit of measure: Number
Scheme performance
tracking tools and METRICS
3 (Q22)
Gender: Male
Quarterly
2 # of cooperative members who are female
Component 1: Program Outcomes
Definition: Total number of women registered as cooperative members during the reporting period. Unit of measure: Number
Scheme performance
tracking tools and
METRICS (Q23)
Gender: Female
Quarterly
Objective 1: Annual stakeholder action plans for 6 co-ops detail public/private partnerships for improved health
3 # of stakeholder workshop reports detailing public/private partnership action for improved health
Component 4: Organizational Capacity (NGO)
Definition: Number of Health coops with stakeholder workshop reports detailing public/private partnership action for improved health. Unit of Measure: number.
Minutes from annual
stakeholders’ workshops
None Semi-
annual
IR 1.1: 42 Cooperative member groups elect providers based on measures of quality and establish MOUs
1.1.1 # of Coop member groups affiliated to CDP
Component 3: Organizational Capacity for Service Delivery
Definition: Total number of coop member groups formed with support from CDP Unit of measure: Number
Scheme performance tracking tools
None Quarterly
1.1.2 # of co-op providers who signed MOUs with member groups
Component 3: Organizational Capacity for Service Delivery
Definition: The total number of providers who signed MOUs with member groups. Unit of Measure: Number
Scheme performance tracking tools
None Annual
2 Number of people of people to be reached excluding Women of Reproductive Age are 55,000, but also women of Non Reproductive Age are included
3 Measurements for Tracking Indicators of Cooperative Success (METRICS) Questionnaire for Co-ops, OCDC and USAID 2009
[23]
Performance Indicator Title Indicator definition and unit of measurement Method of Data
collection Disaggre-
gation
Frequency of Data
Collection
1.1.3 %of providers with improved ratings on their member quality assessment scores
Component 2: Health Services
Definition: Members conduct annual provider quality assessments of their providers. Numerator: Number of positive ratings Denominator: Number of questions. Unit of Measure: %
Provider quality assessments
shared at AGMs
By Provider
Annual
IR 1.2 6 health co-ops sign annual MOUs with five partners each
1.2.1 # of VSLA cooperative members
Component 5: Community Capacity
Definition: Total number of VSLA members registered with Health cooperatives affiliated to CDP during the reporting period. Unit of Measure: Number
Co-op scheme performance
tracking
Gender: Male & Female
Quarterly
1.2.2 # of VHT cooperative members
Component 5: Community Capacity
Definition: Number of VHTs registered with CDP supported health cooperatives during the reporting period. Unit of Measure: Number
Co-op scheme performance
tracking
Gender: Male & Female
Quarterly
1.2.3 # of women’s organizations
Component 5: Community Capacity
Definition: Number of women’s organizations registered with CDP supported health cooperatives during the reporting period. Unit of Measure: Number
Co-op scheme performance
tracking None Quarterly
1.2.4 # of drama group cooperative members
Component 3: Organizational Capacity for Service Delivery
Definition: Number of drama group members registered with CDP supported health cooperatives during the reporting period. Unit of Measure: Number
Co-op scheme performance
tracking
Gender: Male & Female
Quarterly
Objective 2: Six health cooperative boards maintain their cooperative cost balance with annual provider surpluses and reserve fund
2.1 Total annual average provider surplus by co-op
Component 4: Organizational Capacity (NGO)
Definition: Total annual provider surplus Unit of measure: Amount
Co-op scheme performance
tracking By co-op Annual
2.2 Total annual co-op reserve fund balance
Component 4: Organizational Capacity (NGO)
Definition: 10% of the total annual provider’s surplus Unit of measure: Amount
Reserve fund bank account
statement By co-op Annual
[24]
Performance Indicator Title Indicator definition and unit of measurement Method of Data
collection Disaggre-
gation
Frequency of Data
Collection
IR 2.1 18 private health care providers employ data entrants to track monthly co-op performance
2.1.1 # of providers that turn in co-op performance tracking tools.
Component 4: Organizational Capacity (NGO)
Definition: Number of Health facilities affiliated to CDP that turn in tracking tools which they use to inform decision makers during the reporting period. Unit of Measure: Number
Co-op scheme performance
tracking By co-op Quarterly
IR 2.2 Six health cooperatives member elected boards of directors meet routinely to review performance
2.2.1 Average percentage of self-assessment score by the board members
Component 4: Organizational Capacity (NGO)
Definition: Understanding of roles and responsibilities as measured by average percentage score on self-assessment score sheet. Unit of measure: %
Review Board Assessment
forms By co-op Quarterly
2.2.2 # of performance review meetings held by Co-op boards
Component 4: Organizational Capacity (NGO)
Definition: Total number of performance review meetings held by CDP supported co-op boards in the reporting period Unit of measure: Number
Review of co-op board meeting
minutes
By Health co-op board
Semi-annual
2.2.3 # of people attending co-op performance review meetings
Component 4: Organizational Capacity (NGO)
Definition: Total number of people that attended coops performance review meetings Unit of Measure: Number
Review activity reports
Gender: Male & Female
Semi-annual
Objective 3: One CDP partner collaboration plan developed and lessons learned in cooperative health financing shared with MOH and partners
3.1 Number of CLARITY related collaborative activities(workshops, conferences, seminars…etc)
Component 6: Enabling Environment
Definition: Number of CLARITY related collaborative activities(workshops, conferences, seminars…etc) Unit of measure: #
OCDC None Annual
IR 3.1: HealthPartners annual reports document how lessons learned were shared with international and national health partners
3.1.1 Number of METRICS related collaborative activities (workshops, conferences, seminars…etc)
Component 6: Enabling Environment
Definition: Number of METRICS related collaborative activities (workshops, conferences, seminars…etc) Unit of measure: #
OCDC None Annual
3.1.2 Number of IMPACT related collaborations (workshops, conferences, seminars…etc)
Component 6: Enabling Environment
Definition: Number of IMPACT related collaborations (workshops, conferences, seminars…etc) Unit of measure: #
OCDC None Annual
[25]
Handout 15 Health Facility Assessment Indicators
Area # Domain Indicator
Access
- Geographic Access % population with year-round access to MNC services
1 Service availability % HF in which MNC services are available (Child: sick child, immunizations, GMP; MNC: ANC services)
Inputs
2 Staffing % staff in HF who provide clinical services and are working on the day of the survey
3 Infrastructure % essential infrastructure in HF to support MNC services available on the day of the survey
4 Supplies % essential supplies in HF to support MNC services available on the day of the survey
5 Drugs
% first line medications for MNC services available in HF / CHW on the day of survey (HF: ORS, oral antibiotic for dysentery, oral antibiotic for pneumonia, first line anti-malarial, vitamin A / CHW: context-specific)
Processes
6 Information System % HF/CHW that maintain up-to-date and complete records of sick U5 children / ANC services AND show evidence of data use
7 Training % HF/CHW where interviewed HW reports receiving in-service or pre-service education in MNC in last 12 months
8 Supervision
% HF/CHW that received external supervision at least once in the last 3 months (includes at least one: check records or reports, observe work, give feedback)
Performance
9 Utilization # sick child visits per year per U5 child in HF catchment area
10 HW Performance:
Assessment
% HF in which all essential assessment tasks were made by HW for sick child (pass = 80% observed cases)
11 HW Performance:
Treatment
% HF/CHW in which treatment was appropriate to diagnosis for child with fever, ARI, and/or diarrhea (pass = 80% observed cases for HF / 80% most recent cases in register for CHW)
12 HW Performance:
Counseling
% HF in which caretaker correctly describes how to administer all prescribed drugs for malaria, ARI, and/or diarrhea (pass = 80% exit interviews)
[26]
Handout 16 (2 pages) Somota and Jamuna Group Stories
SSoommoottaa WWoommeenn’’ss GGrroouupp SSttoorryy
The Somoto women’s group is a group of twenty women from Lalbagh slum in Dhaka. This group
was formed with the help of Sathi staff in 1999. This group of women began by saving about
ten cents a week. They would meet once a week to collect savings, to talk about activities they
could do, and to make plans. Nioti, the Sathi staff, worked with them to develop leadership and
committees within the group.
In the first year, all of the 20 women were involved in literacy classes. 16 of them were able to
pass the advanced literacy class, putting them at about a 5th grade level of reading and writing.
Right away, they began to take over the tasks of writing in the passbooks, the meeting minutes,
loans etc. They had elections each year to change the leadership. The group realized that they
needed more than one person to do the writing, the meeting leading, the monitoring of income
generation, etc. So, besides the main person, they had group members learn the tasks as
trainees and then each year a new trainee would take over the activity. The group listed each
activity they did. Their goal was to have each of the 5 women trained in each activity. The
activities included things like maintaining passbooks, weighing children, doing banking, leading
meetings, teaching literacy to new members, auditing accounts, etc. Occasionally one of the
women would move away to another area and have to leave the group. However, her
responsibility is taken over by someone else. In 2005, the leadership had changed through
elections.
QUESTIONS:
According to this story, how would you define capacity?
Identify some of the activities/events that promoted capacity development?
What are some of the things groups or communities in the Collaborations Project do to
build capacity for sustainable groups or communities?
[27]
JJaammuunnaa WWoommeenn’’ss GGrroouupp SSttoorryy
Pari is a rural development project located in Jamalpur, Bangladesh. It works in integrated
community development programs in five subdivisions of Jamalpur. The programs include adult
non-formal education (literacy), health promotion, agriculture, savings based income generation,
leadership development and small business development. The foundation of the program is the
development of primary groups in the community, separate for men and women. These groups
consist of 20-25 individuals who are involved in the various activities. Field staff are hired by
Pari to live in the communities and help to form and develop the primary groups.
Jamuna is a Bengali woman who was hired by Pari in 1994 to move with her family to Nandina,
about 20 km from Jamalpur town. She began by relationship building, Participatory Rural
Appraisal and then began with having the women save weekly.
Jamuna was a very enthusiastic staff and 10 women’s groups were formed in the community.
The groups met weekly for their savings meetings. Jamuna never missed a meeting – rain or
shine she was there. She helped the groups to form bank accounts and each week she brought
the savings to the bank. As the women took literacy, they learned to be able to keep the
records. Each month Jamuna took the record books home and checked them all and made the
corrections and sent the books back to the women’s groups in the villages. Once a year she
audited the accounts book of each group. She corrected mistakes and then met with them to
review the status of their accounts.
The group members learned literacy. Jamuna taught the classes which were for six months
after the group was formed. The literacy classes met every day for the six months. Some of
the women learned to write the minutes. Jamuna would check them each week and made
corrections.
In health, Jamuna taught health lessons each week. She would take the women to local clinics
when they or their children were sick. The women would come to her house on weekends as well
if they needed advice on health issues. Jamuna also started a growth monitoring program and
[28]
weighed the children in each village every month. Jamuna had the mothers keep the weight
cards and she explained the weight to them each month.
When there were land and other disputes in the community, Jamuna was often called on to help
mediate the disputes. When there were arguments about which income generation project to
start, or problems with money, Jamuna would solve those problems.
After Jamuna had been working in Nandina for six years with these ten groups, her supervisor
came and shocked her with the news that she was being transferred and needed to leave
Nandina in two weeks. Jamuna and the women were devastated. The supervisor explained that
since Jamuna had worked with these same groups for six years, she could now leave and these
groups would continue.
It was a sad time. Jamuna only had a few days, so she went to each of the groups and explained
to them that they had learned a lot and she trusted that now they could continue all of their
work on their own and continue with their development. The group members were doubtful.
Jamuna had done so many things and they were frightened.
Jamuna left and moved to another area to work. Three years later she went back to Nandina to
visit with the women’s groups.
QUESTIONS:
What did Jamuna find when she went back to Nandina 3 years later? Do you think
the groups were still functioning? Do you think they were strong?
What do you see were some of the mistakes Jamuna probably made when working with
the groups?
What are 5-6 things Jamuna could have done during her working time to help to
ensure that capacity is built for more sustainable groups?
What are the positive steps you take in the Collaborations Project to help ensure that
capacity is developed?
[29]
Handout 17 (4 pages) Organizational Capacity Indicators
Area Domain Indicator
Governance
Legal Recognition The organization is a legally constituted entity, recognized by the host country government
Governing Committee or Board
Organization has a governance body that meets and makes decisions that guide the organization's development
Constitution / Bylaws Organization has a written constitution and/or bylaws accepted and approved by the governance body
Accountability/Integrity Organization has policies/procedures in place to minimize conflicts of interest among leaders and staff through disclosure of conflicts.
Mission and values Organization has a mission and set of values which are clearly understood, agreed and approved by all the members of the organization and these are used
Transparency of Decision-making
There is a systematic process so that decisions are made by senior leaders and the governance body in a way that all in staff are aware of them and understand
Organizational Structure
There is a clear organizational structure in place, with clearly defined roles and effective coordination and communication among departments.
Participation in Decision-Making
(Board/Senior Leadership with all
staff)
Everyone in organization feel that they have been sufficiently consulted and their concerns addressed for important decisions
Human Resources
HR Management Capacity
The organization has an adequately resourced HR function, with professional qualified HRM staff, and clear plans & budgets for HR activities the HR function plays a valued &strategic role.
Staff Roles & Responsibilities
Staff roles and responsibilities are clearly written and well understood
HR Policies & Procedures
The organization has detailed formal HR policies & procedures
HR Personnel Files The organization has up-to-date, accurate & secure employee data.
Staff Welfare &Morale The organization has mechanisms to promote good employee relations and ensure high staff morale.
Staff satisfaction Staff feel satisfied and well treated by the organization
Remuneration System A formal remuneration policy, job classification &salary scale is in place.
Recruitment There are effective systems to support high quality staff recruitment.
Performance Management
The organization has formal systems for performance management.
[30]
Area Domain Indicator
Project Management
Beneficiary Targeting Each project formally defines its beneficiaries. There is a systematic process to ensure that all beneficiaries receive the outputs of the projects.
Project Planning Organization plans for and maps out individual projects that fall under the overall organization work plan
Project Budgeting Organization practices project- level budgeting, monitoring and evaluation.
Monitoring & Evaluation
Resources and a foundation for monitoring and
evaluation systems
There are secure resources (staff, tools, etc.) to carry out M&E activities and a monitoring and evaluation strategy with indicators and targets has been established.
Monitoring data collection and storage
Data are regularly and systematically collected and maintained.
M&E data informs decisions
Monitoring and evaluation data are systematically used to inform program and management decisions.
Data Quality Assurance
Data quality is measured systematically and on a routine basis and quality issues identified are rectified.
Financial Management
Financial accounts/bookkeeping
Organization keeps accounts of money that can be presented on demand
Internal Controls Organization has designed appropriate internal controls and controls are operating effectively
Payroll System Organization is able to correctly calculate payroll as well as remit all appropriate amounts to employees and to the taxing authority
Accounting policies and procedures
Organization has documented all applicable policies and procedures
Bank account Organization has a secure bank account to hold its funds
Record Keeping Organization maintains organized and standardized supporting documentation for every expenditure
Organizational Budgeting and
Financial Planning
Organization prepares, monitors, and updates its corporate budget on a regular basis, with input from all departments
Development and management of donor
agreement budgets
Organization prepares, reviews, and updates donor agreement budgets consistently and accurately.
Financial reporting/Donor
Reporting
Organization provides high quality and timely financial reports as required
[31]
Area Domain Indicator
Technical Capacity
Staff orientation Organization has a systemic process for orienting new staff and preparing for their new responsibilities.
Staff development and training
Organization has a systemic process for orienting new staff and preparing for their new responsibilities.
Access to technical resources
All technical/program staff have and use access to technical resources necessary for their work.
Office Operations
Procurement System Organization has a procurement system that maximizes competition, obtains best value, and observe compliance requirements.
Security and Safety Management
Organization has a security system that effectively identifies and manages all physical and staff security.
Physical infrastructure Buildings and office space: Organization has sufficient office space, meeting space, and equipment for handling its business.
Technological infrastructure
Telephone/fax: Organization has sufficient telephone and fax facilities which allow for efficient and effective communication.
Motor Vehicles Management
Organization has a system that manages, protects and controls motor vehicle usage.
Travel Management Organization has a travel management system that efficiently manages both domestic and international travel.
Records and Information
Management
Organization has a management system that manages records filing, retention and archiving, observing legal and compliance requirements.
Information Technology
Technological infrastructure
Computers, applications, network and internet facilities: Organization has state of the art fully networked and integrated computing resources. Organization maintains servers for key applications either in-house or via a hosted service
Software Systems
Systems are in place which allow the organization to track, report, and transform Contracts & Grants and Program data. Organization is using a formal accounting system.
Communications
Branding/Messaging Organization has an official name, a mission statement, a logo, and clearly defined messages to communicate its mission to the public.
Media/Public Relations
Organization has a media strategy, including a process to identify and communicate with media (including print, broadcast/radio, and online), respond to media requests, and produce press releases/ statements.
Brochures/Collateral Materials
Organization has a well-designed set of collateral materials for marketing purposes, and the capacity to produce reports and other publications when needed.
[32]
Area Domain Indicator
Program Management
Strategic planning Organization has a strategic plan (an all-encompassing document that guides its programming and aspirations)
Annual Organizational Work plan
Development
Organization plans the development of its activities, involving all relevant staff and stakeholders
Participation in Decision-Making
(within Units/Departments)
Everyone in unit/ department feels that they have been sufficiently consulted and their concerns addressed for important unit and department decisions
Quality Improvement System
There is a process to use information-driven approaches to improve organizational learning and performance (at all levels--technical, programmatic, etc).
Annual workplan completion
The organization regularly checks progress against the workplan and revises plan as necessary.
Resource Mobilization
Identification and tracking of funding
opportunities
The organization has a systematic manner of identifying and tracking new funding opportunities in order to make strategic decisions.
Organization has the capacity to develop funding proposals
There is satisfactory capacity to write, budget, partner and manage the proposal development process.
Success rate in winning bids
Organization has been successful in raising donor funds.
Resource diversification
Organization relies on a diversified resource base.
Networking
Networking to coordinate with other
non-governmental implementers
Organization has relations with other implementers to include CBO's and community based actors in order to coordinate service delivery and avoid duplication of services.
Collaborating with relevant government
agencies
Organization has relations with government entities, for coordinated implementation and/or advocacy for policy change
Partnerships for technical assistance
Organization has partnerships for provision of technical assistance from agencies that include: UN agencies, large NGOs, local universities, and others)
Relations with potential donors for
funding
Organization maintains relations with a diverse set of potential donors and keeps them informed of its work, so that it can efficiently take advantage of funding opportunities as they arise
[33]
Handout 18 (2 pages) Community Capacity Indicators Indicators on Community Capacity to Respond to Burden of Disease (Note that many of these indicators can be obtained through standard KPC surveys. The indicators proposed here need to be further specified according to priority disease areas affecting the community.)
Area Indicator
Perception of the Disease
Perceived threat of the disease on the community
Perceived role of the community confronting the disease (Extent to which the disease is accepted as a “fact of life” versus something the community is empowered to combat)
Knowledge about the Disease
Awareness of existence/presence of disease in the community
Knowledge of how disease is transmitted
Knowledge of how disease is prevented
Knowledge of available treatment for disease
Response to disease in the
community
An action plan to combat disease and ensure health of its members has been developed by the community
The community reaches out to support those who are vulnerable to disease
Access to primary health facility community members
Access to outreach services (in the community) by community members
Use of recommended preventive measures by community members
Use of recommended treatment by community members
Awareness of Gender Issues
Community members are aware of specific gender roles that may have an effect on burden of disease
Community members take action in line with gender roles.
Measuring Change in the Community
The community has articulated the changes it would like to see in the community with respect to its health
The community has put in place a method of measuring progress towards these changes
The community is able to measure changes and demonstrate improvements in the health of its members
[34]
Area Indicator
Learning & Sharing
The community is receptive to learning good health practices from outside sources
The community shares and applies these learnings
The community seeks out people with relevant experience to help them
The community shares its learnings and successes with other communities and also invites other to share their experiences
Mobilizing resources
The community is aware of the resources necessary to achieve their health improvement goals
The community acknowledges their role in mobilizing these resources
The community mobilizes internally to gather a portion of the needed resources
The community harnesses existing capacity to identify potential external resources and mobilize them.
[35]
Handout 19 List of Gender Questions
Integrating gender into a sustainability plan requires a gender analysis of key
stakeholders, including the implementing organization. It is important that
HealthPartners themselves works toward gender equality within the Uganda office, in
order to effectively integrate gender considerations into programming and capacity
building of partners.
The following are key issues to be identified in a gender analysis:
Practical needs of women, men, girls, and boys. (I.e. access to health commodities.)
Strategic needs of women, men, girls, and boys. (I.e. collective organization.)
Decision-makers
Differing roles and responsibilities of women and men, both within the household
and outside of the household/in the wider community
Beliefs, customs, politics, and tradition that could influence gender equality or
inequality
The following are some questions to consider when identifying the points above, and can
be integrated into existing capacity assessment tools:
Are women accepted as decision-makers?
Are women’s organizations present in the community? What activities are they
involved in; what issues do they address?
Who does what, when, where, and for how long in the community?
o Are there differing roles and responsibilities among women and men
within the household and outside? If so, what are they?
o Do beliefs, customs, politics or tradition that women and men hold
influence roles and responsibilities?
Who appears to be accessing services?
Who has access to productive resources like land, markets, equipment, capital,
credit, education, and training?
o How do beliefs/customs/politics/tradition influence access and control
of resources?
Who appears to have decision-making power (in households and in the
community)?
What practical needs of women, girls, men and boys are being met? Which are
not?
What are the opportunities for women, girls, men and boys? This could focus on opportunities for accessing health services, or having good health outcomes.
What are constraints for women, girls, men and boys? This could focus on constraints to accessing health services, etc.
[36]
Handout 20 Modified Gender Analysis Matrix
Below are some categories to consider when thinking through possible issues that influence behaviors and access in your program area. These categories are to assist you with putting on a “gender lens,” but are just a guide; there may not be a significant issue in every category, and there may be multiple issues under a single category.
What are the key
gender issues and
facilitators in each
domain below, in the
program area?
What other
potential
information is
missing, but
needed?
What are gender-
based constraints
and/or opportunities
for programming?
Potential strategies
to address issues or
capitalize on
facilitators
Practices, roles, and
participation:
Women cannot leave livelihood or other children for care seeking
Knowledge, beliefs,
perceptions:
Poor treatment by health facility staff
Access:
Distance to health facility
Assets (for transportation, fees at facility)
Low literacy
Legal rights and status:
Free ANC services in all public facilities
National Legislation Against Domestic Violence
Power and decision-
making:
Unequal representation (gender, ethnicity, etc) in community structures
[37]
Handout 21 (2 pages) Sub-Components of the Enabling Environment
Component 6—Enabling environment
There are essential social-ecological environment variables that can either support or weaken
gains in health. Clearly, more can be achieved in a more favorable environment than a less
favorable one. If we are going to be accountable for progress toward sustainability, we should
be clear about the context in which the local system operates. The stakeholders who need to be
part of the functioning of the local system, the strategies for achieving effective service
delivery and positive health outcomes, the social arrangements, the distribution of roles among
partners, and consequently what capacity and viability needs to be built within partner
institutions are only meaningful within a given environment.
Some environmental factors may be within the ability of a project or local system stakeholders
to influence, whereas others will not be. For instance, a project might identify partners to
advocate for policy change, but on the other hand, vulnerability to drought, food insecurity, and
other disasters will be difficult to mitigate. As a more specific example, if a project promotes
community case management of malaria, a national policy that prohibits use of anti-malarials by
anyone other than a licensed doctor is an “environmental factor” that will constrain the
sustainability of any gain achieved locally. It would be still worse if there was not even a policy
recognizing the legitimacy of community health agents in general.
Given the vastness of the social-ecological environment, you can feel overwhelmed by the task
of addressing it at all. But the task does not have to be daunting. We propose to guide you
through an “environmental scan” that includes six subcomponents. Before presenting them, you
must remember that the detailed content of the subcomponents for your project will ultimately
depend on the local context. But rather than start the discussion of the environment with
partners on a blank page, we suggest that you start with the description of the content of the
six subcomponents presented below.
1. Health Policy and Government Commitment to Health
The specific policy issues to be addressed will depend on the technical focus of the project.
But clearly, the level of commitment and resources devoted to health will be a major factor
in what is possible to implement and sustain.
2. Governance and Civil Stability
There are various measures to ascertain whether government institutions function and are
trusted. This is critical for support of government-sponsored health services and for the
climate in which civil society operates. Specifically, in terms of civil stability, in areas where
there is disruption because of war or insurrection, there is likely to be disruption of
services, strains in social networks, and even physical displacement of people. Disruption of
this sort makes gains in health tenuous at best. When the situation is at its worst, perhaps
immediately post-conflict, project managers should be cautious in terms what they promise
with regards to sustainability.
[38]
3. Human Development
Large swings in the economic and development landscape can make for shifting priorities
among organizations and individuals that may threaten health gains. For instance, in a
situation of drought or even famine, subsistence will predominate over health in terms of
national, local, and household priorities. The United Nations Development Programme (UNDP)
computes a summary measure of human development—the Human Development Index
(HDI)—for all countries on a periodic basis. UNDP country offices sometimes compute this
score for sub national regions as well. Progress on the HDI supports greater expectations
for what can be sustained.
4. Women’s Empowerment
The role of women is critical in population health gains, as women are the main caregivers to
themselves and their family members. Values related to women, in terms of their decision
making authority and power within the household will either endanger or solidify their ability
to act in order to effect positive change for health. This correlates with their level of
education and literacy.
5. Natural Environmental Factors
Many areas are prone to natural disasters that have the ability to quickly wipe out gains in
health and development. Some geographic regions are more prone than others to disruptive
natural phenomena. The profile of vulnerabilities will vary from location to location. Some
areas are prone to drought affecting food security and nutrition; others to quicker onset
disasters causing massive service disruption and/or displacement of populations (e.g.,
earthquakes, tsunamis, hurricanes, flooding, etc.). Unfortunately, the poor are
disproportionately affected by natural disasters.
The ability of a project to influence these subcomponents directly or indirectly is greater for
the first subcomponents listed than for the last. Some will consider the ecological environment
part of ‘the world as it is’ and not deserving to be part of a project’s analytical model. But in
some contexts, projects will find reporting on vulnerability and national preparedness important
to draw attention to threats to the sustainability of local efforts. This can be a tool for
advocacy. There can also be ecological variables with more direct effects, such as whether
mosquito breeding sites are controlled or not.
[39]
Handout 22 Enabling Environment Indicators Subcomponent Index Data Source
a. Health Policy and Government Commitment to Health
Commitment to Health Center for Global Development http://www.cgdev.org/content/publications/detail/10016
b. Governance and Civil Stability
Voice and Accountability World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp
Political Stability and Absence of Violence
World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp
Government Effectiveness World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp
Regulatory Quality World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp
Rule of Law World Governance Indicators http://info.worldbank.org/governance/wgi/index.asp
Corruption Perception Index Transparency International www.transparency.org/policy_research/surveys_indices/cpi
c. Strength of Civil Society
Civil Society Strength Indicator
Freedom House www.freedomhouse.org
Indicator of Civil Liberties and Political Rights
Freedom House www.freedomhouse.org
d. Human Development Human Development Index UNDP http://hdr.undp.org/en/
Human Poverty Index UNDP http://hdrstats.undp.org/indicators/18.html
Progress toward Millennium Development Goals (MDGs)
UNDP www.mdgmonitor.org
Gini Coefficient (Measure of Income Inequality)
World Development Indicators www.worldbank.org/data
Equitable Access to Education and Health
World Development Indicators www.worldbank.org/data
e. Gender Empowerment
Gender-Related Development Index
UNDP http://hdr.undp.org/en/statistics/indices/gdi_gem/
Gender Empowerment Measure
UNDP http://hdr.undp.org/en/statistics/indices/gdi_gem/
Share of women in political office
UNDP
f. Ecological Environment and Natural Factors
Environmental Risk Index United Nations Environment Programme (UNEP)—Environmental Vulnerability Index (EVI), especially the questions related to disasters): http://www.vulnerabilityindex.net/EVI_2005.htm
General/Cross-Cutting Quality of Life Index The Economist‘s Intelligence Unit www.economist.com/media/pdf/quality_of_life.pdf
[40]
Handout 23 Detailed Sustainability Assessment Plan
Component
Indicators (Monitoring& Evaluation)
Definition of Indicator Source of Data
Frequency of Collection
Partner/ Institution
Responsible Baseline
Value
Program Outcomes
Health Services
Org. Capacity for Service Delivery
Org. Capacity for Comm. Mobilization
Community Capacity
Enabling Environment
[41]
Handout 24 (2 pages) Creating Scores from Scales A 'Scale' is a type of measuring device or weighing machine. As we begin to think about Scales, give your answers to the following: For each of the items below, check if it is better to rank high on the Scale, or low:
Better high on scale Better low on scale The number of cockroaches in my house My results on the pre-test Our team score in football The number of children in my family My body weight in kilograms
What did you choose? When is it better to rank high? When is it better to rank low? When is there debate about which is better? Now repeat the exercise using these sustainability-related issues.
Better high on scale Better low on scale Proportion of births attended by skilled health professional Cases of malaria Average annual provider surplus Number of days without rain in the last year Number of co-ops functioning
What did you choose? When is it better to rank high? When is it better to rank low? When is there debate about which is better? Sometimes the progression from poor to good or from undesirable to desirable does not occur at an even rate. Consider the following: If we are ranking the percent of community health volunteers who have received a supervisory visit during the previous quarter, we can use an evenly graduated Scale. It would look something like this:
Measured Value
Strong 81 to 100%
Promising 61 to 80%
Intermediate 41 to 60%
Emerging 21 to 40%
Poor 0 to 20%
However, if we try to apply the same Scale for ranking diarrhea treatment for children, it would not be appropriate. If a community has 61% of its community health volunteers receiving supervisory visits each quarter, that is promising! But if only 61% of the same community's children are receiving any treatment for diarrhea, that is not very promising. It could perhaps be better described as intermediate. A more useful Scale might be something like this:
[42]
Measured Value
Strong 90 to 100%
Promising 75 to 89%
Intermediate 55 to 74%
Emerging 35 to 54%
Poor 0 to 34%
Each of these Scales can be plotted on a graph, much like the common growth chart. Plot the following two charts using the lowest number in the Scales above and draw a line to connect the dots. Strong Promising Intermediate Emerging Poor 0 10 20 30 40 50 60 70 80 90 100
Strong Promising Intermediate Emerging Poor 0 10 20 30 40 50 60 70 80 90 100
If you plotted the above graphs correctly, the line drawn will look different on each. The first graph should have a line that goes like this:
This is called a 'linear scale'. The second graph should have a line that goes like this: This is called a 'non-linear scale.'
Percent of community health volunteers receiving a supervisory
visit in the past quarter
Percent of community children who get treatment when they have
diarrhea
[43]
Handout 25 An Example Sustainability Dashboard
Questions for Small Groups: Baseline How would you describe this Project at Baseline? Which Components are strongest? Which Components need more attention? If all 6 Components are critically important to the Project’s sustainability, on which Component(s) should we focus more attention in this example according to the Baseline measurements? Final How did the Project improve at Final? Which Components experienced growth? Which Components did not experience as much growth? List 3 reasons why you think that some Components improved more significantly than other Components. 1) 2) 3) What do you think is most valuable about this type of graphical summary?
[44]
Handout 26 An Example Pro-Sustainability Results Framework
SO: Increased uptake of HIV/AIDS and TB prevention, care and
treatment services
IR1: Improved coverage of
comprehensive HIV and TB services
IR 1.1: Expanded HIV and TB counseling & testing
service outlets
IR 1.2: Expanded HIV/AIDS care &
treatment services
IR 1.3: Increased coverage of HIV prevention services
(PMTCT and MC)
IR2: Improved quality of HIV and TB
health services
IR 2.1: Improved health worker performance for key HIV and TB care and
treatment services
IR 2.2: Improved supervisory system for facility and community-
based health workers
IR 2.3: Improved laboratory testing
services
IR3: Improved capacity & viability of Provincial Health
Teams
IR 3.1: Improved leadership and
management of HIV/AIDS and TB programs
IR 3.2:Improved technical capacity of district health
teams
IR 3.3: Improved capacity to lead and support
quality improvement initiatives
IR4: Improved capacity & viability of local NGOs and
CBOs
IR 4.1: Improved capacity to provide community-
based care and support to PLHIV
IR 4.2: Establishment of functional referral system
IR 4.3: Establishment of strong partnership among
NGOs/ CBOs
IR5: Improved community
awareness & knowledge of HIV &
TB
IR 5.1: Increased knowledge of HIV prevention among
community members
IR 5.2: Improved attitudes towards PLHIV
IR 5.3: Improved awareness of HIV/AIDS
and TB services available
IR6: Improved policy environment for
MARP
IR6.1 Provincial and national policymakers
informed of best practices for targeting services for
MARPs
IR6.2: National MOH decision makers aware of
specialized needs of MARPs
Vision: All people living with HIV and TB will be able to live healthy and productive lives. They will access quality health services and information, in a timely manner and within a supportive
environment.
[45]
Handout 27 Results Framework Template
Strategic Objective
IR1
IR1.1
IR1.2
IR2
IR2.1
IR2.2
Local System Vision
[46]
Handout 28 Program Planning Matrix
Result Area Strategy Activities Time Frame Stakeholder/ Partner Responsible
IR:
Sub-IR 1:
Sub-IR 2:
Sub-IR 3:
[47]
Handout 29 (5 pages) Sustainability Framework Checklist
Instructions for Participants
PURPOSE OF CHECKLIST:
This checklist can be used to review program plans and processes in order to integrate
sustainability into the design, implementation and evaluation of programs. It can be used during
the early conception of a project, the review of a project proposal, during project planning and
during a review of a project.
INSTRUCTIONS FOR THE PLANNING EXERCISE:
Use the checklist to think through what items you think are important for the implementation
of sustainability into the design of your project and to help you determine what actions to
include in your action plan.
The checklist can also help you to think about things that the project may have already done,
whether it be implicitly or explicitly. For any items that you identify that the project has
already done, think about and consider if there is anything else that you may need to do to build
upon the work that you’ve already done.
I. System Situational Analysis
# √ Question Comments
I.1 You can list all the essential members of
the local system [stakeholders] that will
have to play a role for positive health outcomes to be sustained 5-10 years
after the project ends.
I.2 You have a plan for approaching or you have already involved the identified
stakeholders in your project planning.
I.3 You can list other local system stakeholders (“key facilitators” and “key
allies”) and you have a plan to form
alliances with them.
I.4 You have identified the key outside influencers outside the local system who
should provide support to health related activities (e.g., national MOH, UN, NGOs,
Government); and you can describe how they influence those in the local system.
I.5 You can list and you have a reasonable
understanding of the priorities and plans
of the key local system actors and the key outside influencing organizations.
[48]
II. Sustainability Scenario Planning
# √ Question Comments
II.1 Local stakeholders have developed a
coherent vision for achieving and maintaining positive health outcomes in
the appropriate beneficiary group(s). This vision is a consensus of local stakeholders
and is meaningful, coherent, and (eventually) achievable.
II.2 You have identified potential conflicts
between the local system and the
agendas of other key local or outside stakeholders. You have a plan to work
jointly with implementing partners to resolve these conflicts.
II.3 The sustainability scenario offers the
project opportunities to make a significant and measurable contribution toward the
realization of the vision.
II.4 You have begun identifying implementing
stakeholders of the long-term strategies in the sustainability scenario for any
components/elements that are outside the mandate of the project.
[49]
III. System Assessment
# √ Question Comments
III.1 Project partners and other stakeholders
have defined the components and sub-components of the Sustainability
Framework for their local system context to guide what will be measured in the
local system assessments.
III.2 Project partners and other stakeholders have identified indicators to measure the
components of the Sustainability
Framework that apply to their local system context (for the baseline
assessments).
III.3 Project partners and other stakeholders have discussed and negotiated how the
local system assessments will be carried out, including:
a) Identifying existing data sources
b) Identifying and/or developing data collection tools
c) Determining roles and responsibilities for carrying out
local assessments.
III.4 Project partners collect, analyze and interpret data from local system
assessments carried out.
III.5 The project has synthesized the data to
make it meaningful for project partners and other stakeholders.
III.6 Results from the local system assessment
are shared with project partners and other stakeholders to inform the selection
of strategies/activities that will move the local system closer towards its vision.
[50]
IV. Program Planning
# √ Question Comments
IV.1 Project partners develop a Results
Framework or Log Frame, that is integrated into the Local System’s
Sustainability Framework (i.e., the project does not necessarily work on all
components of the SF equally, but has
taken account of all of them and is consistent with them).
IV.2 Project partners identify strategies,
activities and transition outcomes that will lead the local system closer to achieving
the vision.
IV. 3 The identified strategies and activities are not the sole responsibility of the project,
but other stakeholders have been
identified to work on the components or elements that are outside the scope of
the project.
IV.4 Project partners share the Results Framework/Log Frame and detailed work
plan with all relevant stakeholders in the local system for feedback and input.
IV.5 Project partners and other stakeholders
identify and negotiate the roles and
responsibilities of all stakeholders for carrying out the identified
strategies/activities.
IV.6 Project partners review the Results Framework or Log Frame and the detailed
work plan (consists of strategies, activities and transition outcomes), to
ensure that it is coherent and that it
captures all the elements necessary for sustainability.
[51]
V. Program Monitoring and Evaluation
# √ Question Comments
V.1 The Results Framework or Log Frame is
used to inform the M&E plan for the program.
V.2 The project M&E plan includes a
monitoring plan that involves active stakeholder review of progress toward
sustainability in all components of the SF,
and not just the components (and sub-components) that are within the mandate
of the project.
V.3 Project evaluations have maintained a focus on the local system in addition to
the strict requirements of accountability toward the project donor.
V.4 Monitoring, evaluation, and sustainability
assessment know-how are being
developed among local stakeholders themselves. Local stakeholders
demonstrate the freedom and initiative to call for review steps.
V.5 The nature and involvement of
stakeholders is reviewed over time; adjustments are made to project
implementation to increase cohesion and
ownership within the local system.
V.6 The project has shown flexibility to
respond to changes, shifts in the
environment, and lessons learned.
[52]
Handout 30 Sustainability Planning Action Plan Template
Sustainability Program Design
Process Step Activities Point Person(s) Time Frame
System Situational Analysis
Sustainability Scenario Planning
System Assessment
Program Planning
Program Monitoring & Evaluation
[53]
Handout 31 Creating a Sustainability Dashboard in Excel Task #1: Enter the Indicators into the Index Computation spreadsheet a) In the software provided, open the Index Computation spreadsheet b) In each of the six worksheets labeled at the bottom of the page Component 1-6, type in the
Indicators you selected under the column labeled “List Indicators.” c) To the right of the column labeled “List Indicators,” there is a column labeled “Measured Value.”
Enter the corresponding data from each Indicator in this column. For example, the Component 3 worksheet would look like this:
d) Enter your data for every Indicator, in all six Components, into the “Measured Values” column. e) Why is it important to transform these Measured Values into Scores? Discuss the following:
what is the difference between a Measured Value in percentage (%) and a Score in points? Once this is clear, proceed to the next step.
Task #2: Transform Indicator values into computable Scores a) Return to the first Component in the Index Computation spreadsheet. b) Select the first Indicator. Does the Indicator follow a liner scale or a non-linear scale? c) Open the Transform spreadsheet. d) For the selected Indicator, choose the appropriate Status Scale as seen in the yellow Tables
below:
Status Scale 1: Non-linear, positive slope Indicator status: Poor Emerging Interm. Promising Strong
max .349 0.549 0.749 0.899 1
min 0 0.35 0.55 0.75 0.90
[54]
Status Scale 2: Non-liner, negative slope Indicator status: Poor Emerging Interm. Promising Strong
max 1 0.45 0.3 0.15 0.05
min 0.4501 0.3001 0.1501 0.0501 0
Status Scale 3: Linear, positive slope
Indicator status: Poor Emerging Interm. Promising Strong
max .199 .399 .599 .799 1
min 0 .20 .40 .60 .80
e) Once you choose the appropriate Status Scale, enter the “min” and “max” numbers into the
corresponding boxes on the Transform spreadsheet. f) Examine the Indicator Status figure at the bottom of the Transform spreadsheet and briefly
discuss whether it represents a reasonable picture of progress on this indicator.
g) Identify the other Indicators in Component 1 which progress on the same Status Scale. [Note:
Indicators from the same Component might not all fit the same pattern, although they often do.] h) Enter the listed Measured Values of all Indicators in Component 1 sharing the same Status Scale
(and only those sharing this Status Scale!) under column D of the Transform spreadsheet. i) Which Indicators do not fit this Status Scale? j) Read and copy the resulting Scores from Column L of the Transform spreadsheet onto a piece of
paper.
[55]
k) Type the Scores from Column L (Transform spreadsheet) into the “Score” column of the Index Computation spreadsheet.
l) Select an Indicator that has a different Status Scale than the first Indicator you chose in the
Component you are currently working on. m) Repeat steps d through k for all of the Indicators that share this same Status Scale for this
Component. n) Once you are finished creating Scores for Component 1, repeat steps d through m for
Components 2 and 5.
[56]
o) The Transform spreadsheet we just used is for Indicators measured on a QUANTITATIVE scale. A different approach is needed for Indicators measured on an ORDINAL scale. For example, Components 3 and 4 (the OCIs) are measured on a 1 – 5 ordinal scale.
p) Select the appropriate Status Scale for the Organizational Capacity Indicators:
Status Scale 1: Non-linear, positive slope
Indicator status: Poor Emerging Interm. Promising Strong
max 2.4 3.2 4 4.6 5
min 1 2.41 3.21 4.01 4.61
Status Scale 2: Linear, positive slope
Indicator status: Poor Emerging Interm. Promising Strong
max 1.79 2.59 3.39 4.19 5
min 1 1.80 2.6 3.4 4.2
q) In order to change the status Scale for an ordinal Indicator, you must also change the x-axis (or
Measured Value) of the Indicator Status figure at the bottom of the Transform spreadsheet. r) In the Transform spreadsheet, double-click on any number on the x-axis of the Indicator Status
figure. A Format Axis window will appear. Click on the Scale tab. Type in your Minimum measure (in our example we will use 1), your Maximum measure (in our example we will use 5), your Major unit (in our example we will use 0.5), and your Minor unit (in our example we will use 0.1). Be certain that none of the boxes under Auto are checked. Click OK.
s) Examine the NEW Indicator Status figure at the bottom of the Transform spreadsheet and briefly discuss whether it represents a reasonable picture of progress on this indicator. If it does not represent your Indicator, please change the Status Scale (in yellow) and the x-axis of the Indicator Status figure. (See Transform_OCI.xls for an example of steps o through q.)
t) For examples of other Scales used to transform Indicators into Scores, please open the Examples_CSSAindexes.xls spreadsheet.
Task #3: Compile Scores into Index values a) For this exercise we have pre-selected the simplest way to compile the Scores into an Index (the
average). b) The Index Computation spreadsheet is pre-set to provide an average Score, which is the Index.
For example, you can calculate the average of column C, rows 4 through 15, by typing in the following text into an empty cell: =AVERAGE(C4:C15)
[57]
c) Follow this step for all 6 Components. Task #4: Create a dashboard a) Go to the worksheet labeled “Map” (always in Index Computation spreadsheet). The Indices
have been automatically inserted in the table. A hexagonal radar diagram (dashboard) has automatically been produced to represent all 6 Indices.
b) Discuss your dashboard with your group. In which Components is your Project strong? In which
Components does your Project need to grow? List three strategies for improvement (from three different Components). Each group will present their dashboard to the other groups.