Post on 18-Jun-2020
WORKSHOP HANDOUT
Designing capacity-building interventions: what the evidence tells us
Prepared by the Health Promotion and Special Projects team, Health Promotion and Chronic Disease
Prevention Department at Public Health Ontario. Contact Benjamin.rempel@oahpp.ca for details.
Introduction Public Health Ontario recently conducted two systematic reviews to determine the effectiveness of
capacity building interventions relevant to health promotion and public health. The purpose of this
workshop is to translate the knowledge gained from these reviews and apply it to the interventions used
in participants’ own practice.
What is capacity building? The World Health Organization defines capacity building as “the development of knowledge, skills,
commitment, structures, systems, and leadership to enable effective health promotion, with actions to
improve health.”1(p.341-342) The definition includes action at three levels:
1. The advancement of knowledge and skills among practitioners
2. The expansion of support and infrastructure for health promotion in organizations
3. The development of cohesiveness and partnership for health in communities1
PHO conducted two systematic reviews to examine the evidence around the role of capacity building in
public health. Two articles summarizing their results were published in BMC Public Health: Theories,
models and frameworks used in capacity building interventions relevant to public health;2 and
Effectiveness of capacity building interventions relevant to public health practice.3
Theories, models and frameworks This systematic review2 was conducted to identify conceptual foundations for capacity building by
examining the theories, models and frameworks used to support capacity building. Nineteen papers
were included in the review: eight were rated as strong and 11 papers were rated moderate, primarily
as the theory, model or framework was implicitly rather than explicitly stated.
Using theories, models or frameworks as a foundation for developing capacity building interventions can
provide a road map for studying programs, developing appropriate interventions, and evaluating their
effectiveness.2 A good theory provides a clear explanation of how and why specific relationships lead to
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 2
Adapted from McLeroy K, Steckler A, Bibeau D. The social ecology of health
promotion interventions. Health Education Quarterly, 1988 15(4):351-377.6
Societal
Community
Organizational
Interpersonal
Individual
specific events2 and how individuals, groups and organizations behave and change.4 Using this
knowledge can help us to design programs and to:
build understanding of our stakeholders
explore the factors that can promote or inhibit change
help us to select and develop health promotion strategies2
In other words, theories can help us to understand who we should be targeting, why certain
individuals/groups engage in certain behaviours, and how we can achieve change through our
programming.4 Using theories, models and frameworks to design our capacity building interventions can
help us to:
describe/guide the process of translating research into practice
understand and/or explain what influences implementation outcomes
evaluate implementation5
Four theories, 17 models, and seven frameworks were identified in the studies included in the
systematic review. Five were identified for consideration as a theoretical foundation for designing
and implement capacity building approaches relevant to public health. Each is described below:
DIFFUSION OF INNOVATIONS
Diffusion of Innovations is a behaviour change theory which addresses how ideas, products, and social
practices that are perceived as new spread throughout a society, or from one society to another.4 A
number of factors determine how quickly, and to what extent, an innovation will be adopted and
diffused:
The relative advantage of an innovation shows its superiority over whatever it replaces.
Compatibility is an appropriate fit with the intended audience.
Complexity has to do with how easy it is to implement the innovation or adopt the behaviour
Trialability refers to whether it can be tried on an experimental basis.
Observability reflects whether the innovation will produce tangible results.4
ECOLOGICAL FRAMEWORK
The Ecological Framework focuses on the complex
interactions between people, groups and environments.6
This framework recognizes that each of us are influenced
by the environments in which we live, and that each of the
factors in the framework shape the choices that we make.6
At the most basic level, an ecological perspective points to
two approaches addressing health problems: change people’s
behaviour, or change the environment. There is a third and
more powerful option, which is to integrate these two
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 3
Adapted from: Wandersman A. et al. Bridging the gap between prevention research and
practice: The interactive systems framework for dissemination and implementation.
American Journal of Community Psychology, 2008;41(3-4); 171-81.11
approaches and treat them both as essential.4 This option recognizes reciprocal causation: that
individual behaviour both influences, and is influenced by the environment.4
TRANSFORMATIONAL LEARNING
Transformational Learning Theory is a behaviour change theory2 that uses “disorienting dilemmas” to promote growth and change. Individuals are challenged to assess how a learning experience interacts with their existing value system and worldview. Successful learning experiences result in individuals who are in some way (unique to each individual) transformed by an experience.7
INTERACTIVE SYSTEMS FRAMEWORK FOR DISSEMINATION AND IMPLEMENTATION
This framework seeks to address the “how to”
gap between science and practice.8 It was
developed in the violence prevention sector,
where despite the evidence of what works and
what does not, there has not been wide spread
of effective strategies.8 The framework shows
how different systems can work together for
successful dissemination and implementation
of prevention innovations. Each system
involves many different people in different
roles, different stakeholders and organizations,
with all three working together to:
Distill the scientific literature into
findings that are disseminated to the
field
Support the work of capacity building,
training and technical assistance
Deliver and evaluate prevention
strategies.8
BLOOM’S TAXONOMY OF LEARNING
This is one of the most recognized
learning theories in the field of
education.9 Bloom’s Taxonomy can be
used to create learning outcomes that
target not only subject matter but also the
depth of learning that is desired.10 The
taxonomy contains a 6-part hierarchy:
remember, understand, apply, analyze,
evaluate and create.
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 4
Intervention Type
Internet-based instruction
Training and workshops
Technical assistance
Education using self-directed learning
Communities of practice
Multi-strategy interventions
Outcomes
Knowledge
Skills
Self-efficacy
Change in practice and policy
Behaviour change
Application
Organizational support
Perceptions of system-level capacity
Effectiveness of capacity building interventions This systematic review assessed the effectiveness of capacity-building interventions relevant to public
health practice. Thirty-eight papers were assessed as relevant, and 24 of those were excluded due to the
low quality of the study. The remaining 14 were included.
The impact of 6 capacity-building interventions on 8
outcomes were assessed:
Internet-based instruction is defined as any computer assisted instruction using the internet as the
mode of delivery. Internet-based instruction increased knowledge and skills, but only in comparison to
no other intervention. Including discussion groups in the intervention improved knowledge, while
including practice sessions improved skills. Level of interactivity and repetition made no difference for
knowledge or skill enhancement. Recommendation: use when no other interventions are feasible or
accessible (for example due to limited resources or geography/reach).
Training and workshops are defined as in-person training and workshop sessions. These interventions
were grouped together as it was difficult to clarify the difference between training and workshops.
Training and workshops resulted in enhancements across all outcomes with the exception of
understanding, though this is likely because of differences of groups at baseline in the study. Not all
studies examined impacts on behaviour, improvement in practice/policy, and application of knowledge.
Recommendation: closely read the studies that showed effects on behaviour change, improvements in
practice and policy, and application or knowledge in order to identify the key elements of the
interventions that results in these outcomes.
Technical assistance includes personalized support, including face-to-face meetings. The included
interventions reported changes in knowledge, skills, leadership and self-efficacy. The results for changes in
practice and organizational support were mixed. Recommendation: use technical assistance to
complement training and workshops. Evaluate baseline conditions and consider a feasible level of change.
Self-directed learning is any self-directed methodology that includes: facilitators (versus teachers);
learners involved in in selecting learning resources; and self-assessment. This intervention showed some
impact on knowledge when learners are more involved in learning resource selection and for those who
are more advance in learning. Interactivity of learning modules did not make a significant difference in
outcomes. There is a moderate increase in knowledge compared to traditional teaching methods.
Recommendation: build on self-directed learning as a phased approach or series of events.
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 5
Communities of practice: A single paper reported outcomes for a community of practice which was
focussed on capacity building. The intervention focussed on relationships between researchers and
practitioners (occupational therapists), active learning strategies (reading, face to face, and online
discussion), shared learning and problem solving, with sharing of resources and new strategies. The
study reported knowledge gains as measured pre/post-test. Self-reported changes in practice and policy
were also reported.
Multi-strategy interventions: For the purpose of this review, multi-strategy interventions are those
which explicitly described the intervention as consisting of multiple strategies. The two papers included
in this review defined their interventions as having a training component as well as written materials.
Other components included technical assistance, communities of practice, mentoring/coaching,
internship, appreciative inquiry, meetings, and use of technology. Knowledge and skills were enhanced
by multi-component interventions. One study (rated weak in quality but a useful example) evaluated a
system-level, two year, multi-component intervention aimed at generating system-level change in
evidence-informed trauma care. Post/post evaluation found significant improvement in perceived
readiness and capacity between years one and three.
Implications for practice Overall, the two systematic reviews support the effectiveness of capacity building interventions to
increase knowledge, skills, self-efficacy, changes in practice/policy, application and perceptions of
system-level capacity. However this evidence exists mainly at the individual level. There are a number of
implications from the literature to consider when designing capacity building interventions:
Consider an underlying theory/model/framework when designing your capacity building
intervention. Your organization may currently subscribe to one (or more than one). If not,
consider exploring those that are most relevant to your work. Five are recommended for public
health capacity building as a part of this review. Additionally, explicitly reference the theory,
model or framework and explain how it is being used.
Select the capacity building intervention based on the outcome you are seeking. Incorporate
this into your program planning and evaluation processes by clearly identifying what you are
trying to accomplish as a result of your intervention and then select the intervention type
accordingly.
Optimize delivery of interventions. Consider how you could optimize the strategies you already
have in place, and think about what has been shown to work when planning new strategies. The
literature did find several strategies that can optimize the effectiveness of specific interventions.
For example, with internet based instruction, include discussion groups if your intention is to
improve knowledge, and include practice sessions if your intention is to improve skills. For self-
directed learning, involve participants in selecting learning resources. Also consider using
multiple strategies and interventions.
Evaluate, evaluate, evaluate. There is a need to strengthen the evaluation of capacity-building
interventions in the published literature. More consistent and better evaluations are needed.
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 6
Move beyond pre/post-test approaches and consider methods such as multi-time point data
collection and longer-term data collection. Consider data collection methods beyond self-report.
In addition, the evaluation strategy should:
o Reflect the theory/framework underpinning the intervention;
o Include objective measures of effect;
o Assess baseline conditions; and
o Evaluate features most critical to the success of the intervention.
System level action: More than 10 years after the WHO definition called for capacity building at multiple
levels,1 with one exception, the studies reviewed aimed only at the individual level. Consider, when
planning your capacity building interventions, how you can address the systems in which individuals
operate- the Ecological Framework and the Interactive Systems Framework both provide a structure for
doing so.
Resources Bloom’s Taxonomy Action Verbs. This handy list of verbs, divided into the 6 hierarchies of Bloom’s
Taxonomy, is very helpful for writing objectives for capacity building interventions. Available from:
apu.edu/live_data/files/333/blooms_taxonomy_action_verbs.pdf
The Health Promotion Foundations Course, developed by PHO, contains a useful overview of why and
how health promotion theories, models and frameworks can be used. Available from:
https://www.publichealthontario.ca/en/LearningAndDevelopment/OnlineLearning/HealthPromotion/Pa
ges/HP-Foundations.aspx
We heart health literacy has a useful blog that summarizes many topics relevant to health promotion in
simple language. Their posts include: Diffusion of Innovations Theory
https://medium.com/@whhl/useful-theory-diffusion-of-innovations-a2bad2bac4c8 and Self-Efficacy
https://medium.com/wehearthealthliteracy/i-think-i-can-i-think-i-can-a-tribute-to-self-efficacy-
9dec19da90ff.
These knowledge to action tools are a useful way to determin if and how to apply evidence in your
work:
1. Buffet C, Ciliska D, Thomas H. Can I use this evidence in my program decision? Assessing
applicability and transferability of evidence [Internet]. Hamilton, ON: National Collaborating
Centre for Methods and Tools; 2007 [cited 2019 Mar 4]. Available from:
https://www.nccmt.ca/uploads/media/media/0001/01/110008a2754f35048bb7e8ff446117133
b81ab13.pdf
2. Buffet C, Ciliska D, Thomas H. Will it work here? Tool for assessing applicability and
transferability of evidence (version A: when considering starting a new program) [Internet].
Hamilton, ON: National Collaborating Centre for Methods and Tools; 2011 [cited 2019 Mar 4].
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 7
Available
from: https://www.nccmt.ca/uploads/media/media/0001/01/aaa6890e618d12b9983e156ad89
a582af89c34e6.pdf
3. Moberg J, Oxman AD, Rosenbaum S, Schünemann HJ, Guyatt G, Flottorp S, et al. The GRADE
Evidence to Decision (EtD) framework for health system and public health decisions. Health Res
Policy Syst. 2018;16(1):45. Available from: https://health-policy-
systems.biomedcentral.com/articles/10.1186/s12961-018-0320-2
References
1. Smith BJ, Tang KC, Nutbeam D. WHO Health promotion glossary: new terms. Health Promotion International. 2006;21(4):340-5.
2. Bergeron K, Abdi S, DeCorby K, Mensah G, Rempel B, Manson H. Theories, models and frameworks used in capacity buildling interventions relevant to public health: a systematic review. BMC Public Health. 2017;17(914)
3. DeCorby-Watson K, Mensah G, Bergeron K, Abdi S, Rempel B, Manson H. Effectiveness of capacity buildling interventions relevant to public health practice: a systematic review. BMC Public Health. 2018;18(684)
4. Theory at a glance: a guide for health promotion practiceNational Cancer Institute; 2005.
5. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53,015-0242-0.
6. McLeroy K.R, Steckler A, Bibeau D. The social ecology of health promotion interventions. Health Educ Q. 1988;15(4):351.
7. Risly K CH. Professional coaching: an innovative and promising leadership development and career enhancement approach for public health professionals. Health Promot Pract. 2011;12(4):497.
8. National Center for Injury Prevention and Control. Understanding the Interactive Systems Framework for Dissemination and ImplementationCentres for Disease Control and Prevention. Available from: https://www.cdc.gov/violenceprevention/pdf/asap_isf-a.pdf
9. Bloom's taxonomy [Internet].: Centre for Teaching Excellence, University of Waterloo; cited 2018 June 12]. Available from: https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/planning-courses-and-assignments/course-design/blooms-taxonomy
10. Bloom's Taxonomy [Internet].: Vanderbilt University Center for Teaching; cited 2018 June 12]. Available from: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/
11. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the Gap Between Prevention Research and Practice: The Interactive Systems Framework for Dissemination and Implementation. Am J Community Psychol. 2008;41(3-4):171-81.
Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 8