Scaling up to a population health
intervention:
A readiness assessment framework
Duyen Thi Kim Nguyen, MSc (Psychology)Doctoral student (Population & Public Health)
Department of Community Health Sciences [email protected]
PHIRNET Student Webinar
Purpose of this webinar
To share my study purpose, research questions, background, and methods with the PHIRNET community
To obtain constructive feedback from trainees and experts in the field of population health, intervention research, evaluation, and scale up
To provoke discussions regarding knowledge gaps in scaling up and how to advance PHIR methods
Purpose of my dissertation
To develop a readiness assessment framework to assist stakeholders to determine if a health intervention is ready to scale up to a population-level intervention.
ready
stakeholder
health
intervention
scale up
population-level
Key terms:
Research Questions
1) How are health interventions initially scaled up successfully to population-level interventions?
2) How are health interventions sustained successfully as scaled up population-level interventions?
3) What are the key elements of successfully scaled up population-level interventions ?
Phase 1: Scoping Review
4) What can we learn by piloting the readiness assessment framework on an existing community-level intervention?
Research Questions cont’d
Phase 2: Case study evaluation (CSE) Case = Stardale
Phase 3: Test and refine the framework
Background
“Canada is a country of perpetual pilot projects” (Bégin et al., 2009, p. 1185)
Not necessary for all interventions to be scaled up (e.g., programs treating individuals)
Interventions using upstream approaches should be considered
PHI have potential for great health impact Root causes Reducing inequities and incidence
Types and paths of scaling up (Uvin, 1995)
Scaling up
Mangham & Hanson (2010) literature search in June 2008 PubMed: ‘scaling up’ & ‘scaling-up’ 1989-2001 = 2 citations re. scaling up & health interventions 2001-2008 = 51 citations July 2008-Feb 2012 = 132 citations
Scaling up Canada and globally
Conference to Advance the State of the Science and Practice on Scale-up and Spread of Effective Health Programs (2010)
Phase 3: Test and refine the framework
-Assess its utility & practicality (Stardale)-Feedback & assess validity (Experts)
Phase 3: Test and refine the framework
-Assess its utility & practicality (Stardale)-Feedback & assess validity (Experts)
Phase 2: CSE
-Provide evaluation report -Greater understanding of
Stardale’s processes, outcomes, and more
Phase 2: CSE
-Provide evaluation report -Greater understanding of
Stardale’s processes, outcomes, and more
Phase 1: Scoping Review
-Develop a Readiness Assessment Framework
-Expert feedback and assess validity
Phase 1: Scoping Review
-Develop a Readiness Assessment Framework
-Expert feedback and assess validity
Readiness assessment framework to assist stakeholders to determine if a health intervention is ready to scale up to a population-level intervention
Scoping Review: 101
Systematic review Focused question
Hypothesis-testing
Systematic & explicit method
I/E Criteria defined a priori
Study filters
Formal quality assessment
Synthesize the data to provide evidence in support/rejection of the research hypothesis
Scoping review Broad question
Assessing scope & breadth
Less constrained & iterative
I/E Criteria defined post hoc
Typically no study filters
Quality assessment (optional)
Examine extent, range, & mapping; Value of a sys review; Summarize & disseminate; Gaps
Phase 1: Conceptual Framework
Scoping: Arksey & O’Malley (2005); Levac et al, (2010)
Readiness Assessment Framework: Population health approach
Reducing health inequities and improving health among population groups (aggregate)
Focus on broad range of underlying conditions (PHAC, 2001)
Complex Adaptive Systems “a collection of individual agents”; “not always
predictable”; interconnected (Plesk & Greenhalgh, 2001, p. 625)
Individ properties ≠ Collect. properties (Matheson et al., 2009)
1) Database – PubMed, Medline, EMBASE, Global Health, Canadian Health Research Collection, CINAHL, PsycINFO, Social Services Abstract, Social Work Abstract, Cochrane Database of Systematic Reviews, ERIC, EconLit, & WHO Library and Information Networks for Knowledge Database (WHOLIS)
2) Grey literature - Scopus, select conferences regarding scale up (e.g., Conference to advance the state of the science and practice on scale-up and spread of effective health programs) and scale up studies and evaluations of organizations, companies, and government initiatives (e.g., ExpandNet)
3) Authors will be contacted for clarification, data, or further studies
4) Experts and research teams identified during the review process will be contacted for information regarding their knowledge of ongoing or unpublished studies or other consultation
5) Bibliography list of identified relevant publications; Select journals
Quality Assessment Tools
Downs & Black (1998) Checklist Randomized & non-randomized studies Scores fro overall study quality, quality of reporting, power, internal
validly (bias & confounding), and external validity
Critical Appraisal Skills Programme (CASP) appraisal checklist (Public Health Resource Unit, England, 2006)
10 questions regarding rigour, credibility, and relevance
Quality in Qualitative Evaluation (Spencer et al, 2003)
Focus on qualitative evaluation research
Search on the Cochrane Qualitative Research Methods Group
Categorization for organization & analysis
Research Questions:
Underlying condition(s)Underlying condition(s)
Population characteristics
Population characteristics
NVivo 9
Health outcome(s)
Health outcome(s)
ContextContext
ComplexityComplexity
Characteristics of Stardale
Characteristics of Stardale
Theory behind the
intervention
Theory behind the
intervention
Sustainability processes
Initial processes
Key elements
Key elements
Expert feedback
Experts (academic, practice) Population health Intervention research Scaling up
Surveys and/or semi-structured interviews Face validity Content validity (main domains present) Feedback re. strengths, weaknesses, gaps
Phase 2: Case Study Evaluation
Stardale: Honouring the girls stories
Non-profit organization and registered charity
Honouring the Girls Stories (est. 2005) in Calgary, AB
Provides life skills, literacy education, and advocacy to urban Aboriginal female youths (10-17 years)
Organized activities afterschool (e.g., arts, tutoring, sporting events, dance, acting, modeling)
2/wk from Sept-June; 6:00p-8:00p; with supports all year-round
Participatory Approach
Doing research “with”, not “on” the participant (Minkler & Wallerstein, 2003)
Participatory = research design & implementation (Merriam, 2009), developing sense of ownership, create meaning, and increase likelihood of using the results (Patton, 2002)
Principles: Flexible design, building theory based on participants perspectives, simplicity, agreed topics, analysis reflects the detail and diversity, & share results with participants for informed conclusion (Rothe, 2000)
Aboriginal Research/Evaluations
Tri-council guidelines re. Research involving the First Nations, Inuit and Métis peoples of Canada (Gov of Canada, 2011)
Aboriginal advisory committee
Key components (Johnston, 2008): Unique way of knowing Explaining in relevance to Aboriginal culture In-person, verbal communication Acknowledge Aboriginal core values Time
Case Study Approach
Case study method ideal when: addressing “why” and “how” questions; lack of control over events; contemporary phenomenon (Keen & Packwood, 1995; Yin, 2009)
CSE Framework (Yin, 1992; 2009): Develop understanding of the program & combine with
literature Define units of analysis; Create timeline Define measures, protocols, & procedures Data collection, analysis, and synthesis; Compose a report
Reflection: Myself and Stardale stakeholders (i.e., staff)
Phase 3: Test and refine the framework
Research Question:4)What can we learn by piloting the readiness assessment framework on an existing community-level intervention?
Pilot by Stardale staff and stakeholders
Semi-structure interview for framework feedback Practicality Function Comprehension Feedback re. strengths, weaknesses, gaps
Establishing Validity
Experts (different from study 1) Population health Intervention research Scaling up
Semi-structured interviews Face validity Content validity (main domains present) Feedback re. strengths, weaknesses, gaps
Phase 2: Case Study Evaluation
-Greater understanding of a community-level
intervention, Stardale
Phase 2: Case Study Evaluation
-Greater understanding of a community-level
intervention, Stardale
Phase 1: Scoping Review
-Develop a Readiness Assessment Framework
-Expert feedback and assess validity
Phase 1: Scoping Review
-Develop a Readiness Assessment Framework
-Expert feedback and assess validity
Readiness assessment framework to assist stakeholders to determine if a health intervention is ready to scale up to a population-level intervention
Phase 3: Test and refine the framework
-Assess its utility and practicality (Stardale)-Feedback & assess validity (Experts)
Phase 3: Test and refine the framework
-Assess its utility and practicality (Stardale)-Feedback & assess validity (Experts)
Select References
Arksey, H. & O’Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1):19-32.Edwards, N. (2010). Scaling-up health innovations and interventions in public health: a brief review of the current state-of-the-science. Draft Paper Commissioned for the Conference to Advance the State of the Science and Practice on Scale-up and Spread of Effective Health Programs. Available from: http://ihiscaleupconference10.blogspot.com/2010/06/june-25-introductory-call.htmlJohnston, A. L. K. (2008). Aboriginal ways of knowing: Aboriginal-led evaluation. The Canadian Journal of Program Evaluation, 23(1), 1-6.Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science; 5:69. Available from: http://www.implementationscience.com/content/5/1/69Mangham, L. J. & Hanson, K. (2010). Scaling up in international health: What are the key issues? Health Policy and Planning, 25(2): 85-96. doi:10.1093/heapol/czp066Matheson, A., Dew, K., Cumming, J. (2009). Complexity, evaluation and the effectiveness of community-based interventions to reduce health inequalities. Health Promotion Journal of Australia, 20(3): 221-226.Uvin, P. (1995). Fighting hunger at the grassroots: Paths to scaling up. World Development, 23(6): 927-939.Yin, R. K. (2009). Case study research. California: Sage.
Literature reviews:Bambra, C. (2011). Real world reviews: A beginner’s guide to undertaking systematic reviews of public health policy interventions. J Epidemiol Community Health, 65, 14-19.Grant, M. J. & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26, 91-108.Jackson, N. & Waters, E. (2005). Criteria for the systematic review of health promotion and public health interventions. Health Promotion International, 20, 367-74.Riaz, M., Sulayman, M., Salleh, N., & Mendes, E. (2010). Experiences conducting systematic reviews from novices’ perspective. Avail from: http://www.cs.auckland.ac.nz/~norsaremah/EASE2010.pdf
Evaluation research:Hawe, P., Degeling, D., & Hall, J. (1990). Evaluation health promotion: A health worker’s guide. Australia: Elsevier.Weiss, C. H. (1998). Evaluation, 2nd ed. New Jersey: Prentice Hall.
Qualitative research:Denzin, N. K. & Lincoln, Y. S. (Eds). (2011). The SAGE handbook of qualitative research. Thousand Oaks: SAGE. Merriam, S.B. (2009). Qualitative research: A guide to design and implementation. San Francisco: Jossey-Bass.Stake, R. E. (2010). Qualitative research: Studying how things work. New York: Guilford Press.
Helpful Resources re. PHIR Methods
Thank You!Supervisors: Drs. Lynn McIntyre & Lindsay McLarenSupervisory committee and members of PHIRC at UofC
Funding for my dissertation has been provided by the CIHR Population Health Intervention Research Centre PhD Scholarship.
Funding for my internship at the WHO has been provided by the CIHR Population Health Intervention Research Network Internship Scholarship.
Acknowledgements
A CIHR Centre for Research Development in Population Health
Top Related