Post on 25-Mar-2018
DESIGNING SUSTAINABLE DESIGNING SUSTAINABLE PROGRAMS FOR PROGRAMS FOR
PUBLIC HEALTH IMPACTPUBLIC HEALTH IMPACT
Paul Estabrooks, Ph.D.Paul Estabrooks, Ph.D.Russell E. Glasgow, Ph.D.Russell E. Glasgow, Ph.D.
22
OVERVIEWOVERVIEW
General Issues in Creating Practical, General Issues in Creating Practical, Sustainable ProgramsSustainable Programs
Producing Public Health ImpactProducing Public Health Impact——The The RERE--AIM ModelAIM Model
Creating Sustainable ProgramsCreating Sustainable Programs
33
BARRIERS TO IMPLEMENTATION BARRIERS TO IMPLEMENTATION AND DISSEMINATIONAND DISSEMINATION
Characteristics of the Intervention: Characteristics of the Intervention:
Context Adaptable?Context Adaptable?
High costHigh costIntensive time demandsIntensive time demands
High level of staff expertise requiredHigh level of staff expertise requiredDifficult to learn or understandDifficult to learn or understandNot packaged or Not packaged or ““manualizedmanualized””
Not developed considering user needsNot developed considering user needsNot designed to be selfNot designed to be self--sustainingsustainingHighly specific to particular settingHighly specific to particular settingNot modularized or customizableNot modularized or customizable
44
BARRIERS TO IMPLEMENTATION AND DISSEMINATIONBARRIERS TO IMPLEMENTATION AND DISSEMINATION(cont.)(cont.)
Characteristics of Potential Adopting Characteristics of Potential Adopting ““SettingsSettings””: :
Context IssuesContext Issues
Competing demands occurCompeting demands occurProgram imposed from outsideProgram imposed from outside
Finance or organizations are unstableFinance or organizations are unstableClients and setting have specific needsClients and setting have specific needs
Resources are limitedResources are limitedTime is limitedTime is limited
Organizational support is limitedOrganizational support is limitedPrevailing practices that work against innovationPrevailing practices that work against innovation
Perverse incentives or regulations that oppose changePerverse incentives or regulations that oppose change
55
BARRIERS TO IMPLEMENTATION AND DISSEMINATIONBARRIERS TO IMPLEMENTATION AND DISSEMINATION(cont.)(cont.)
Characteristics of Research Design Used: Characteristics of Research Design Used:
Context Informative?Context Informative?
Not relevant, diverse, or representative:Not relevant, diverse, or representative:Sample of patientsSample of patientsSample of settingsSample of settingsSample of cliniciansSample of clinicians
Failure to evaluate costFailure to evaluate costFailure to assess implementationFailure to assess implementationFailure to evaluate maintenanceFailure to evaluate maintenanceFailure to evaluate sustainability Failure to evaluate sustainability
Glasgow RE, Marcus AC, Bull SS. Disseminating effective cancer screening interventions. Cancer 2004;101:1239-1250
66
BARRIERS TO IMPLEMENTATION AND DISSEMINATIONBARRIERS TO IMPLEMENTATION AND DISSEMINATION (cont.)(cont.)
Interactions among the Three Other Barrier Types; FitInteractions among the Three Other Barrier Types; Fit
Because of barriers, the program reach or participation is lowBecause of barriers, the program reach or participation is low
Intervention is not flexibleIntervention is not flexible
Intervention is not appropriate for the target populationIntervention is not appropriate for the target population
Staffing pattern does not match intervention requirementsStaffing pattern does not match intervention requirements
Inconsistent organization and intervention philosophiesInconsistent organization and intervention philosophies
Inability to implement intervention adequatelyInability to implement intervention adequately
Glasgow et al. Translation Research in Diabetes…In: Evidence-based Endocrinology; VM Montori (Ed).Humana Press, Totowa, NJ. Pages 241-256, 2005.
77
As Tested
CriticalElements
Program or Policy
Simplified Systems Model for Translational Research
Organization
Clinic
Program Delivery
Staff
Delivery Site(s)
Fit
ResearchDesign
AppropriateFor Question
PartnershipBroader HealthPolicy and
CulturalContext
Estabrooks PA, Glasgow RE. Am J Prev Med 2006;31(4S):S45-S56
99
ULTIMATE IMPACT OF MAGIC DIET PILLULTIMATE IMPACT OF MAGIC DIET PILL
50% of Clinics Use Adoption 50%
50% of Clinicians Prescribe Adoption 25%
50% of Patients Accept Medication Reach 12.5%
50% Follow Regimen Correctly Implementation 6.2%
50% of Those Taking Correctly Benefit Effectiveness 3.2%
50% Continue to Benefit After 6 Months Maintenance 1.6%
Dissemination Step Concept % Impacted
1010
MORAL OF THE STORY?MORAL OF THE STORY?
(Each step of the dissemination(Each step of the disseminationsequence, or each sequence, or each ““RERE--AIMAIM””
dimension is important)dimension is important)
““Focus on the DenominatorFocus on the Denominator””
1111
RERE--AIM FRAMEWORK FOR AIM FRAMEWORK FOR FOCUSING ON KEY ISSUES IN FOCUSING ON KEY ISSUES IN
INTEGRATING RESEARCH INTO INTEGRATING RESEARCH INTO PRACTICEPRACTICE
1212
HISTORY/BACKGROUND HISTORY/BACKGROUND FOR REFOR RE--AIMAIM
Late 1990s: Late 1990s: Increasingly clear that major Increasingly clear that major problems moving research into practice.problems moving research into practice.
Helpful models for understanding (e.g., Helpful models for understanding (e.g., Rogers) and planning (e.g., Green & Rogers) and planning (e.g., Green & KreuterKreuter) ) health care programs but no systematic health care programs but no systematic models for translation.models for translation.
Almost total focus on efficacy. Assumed that Almost total focus on efficacy. Assumed that linear linear ““automaticautomatic”” process of efficacy process of efficacy effectiveness effectiveness dissemination.dissemination.
1313
RERE--AIM TO HELP PLAN, EVALUATE, AIM TO HELP PLAN, EVALUATE, AND REPORT STUDIESAND REPORT STUDIES
RR IncreaseIncrease RReacheach
EE IncreaseIncrease EEffectivenessffectiveness
AA IncreaseIncrease AAdoptiondoption
I I IncreaseIncrease IImplementationmplementation
MM IncreaseIncrease MMaintenanceaintenance
Glasgow, et al. Ann Behav Med 2004;27(1):3-12
1414
PURPOSES OF REPURPOSES OF RE--AIMAIM
To broaden the criteria used to evaluate To broaden the criteria used to evaluate programs to include external validity and programs to include external validity and context.context.
To evaluate setting issues relevant to program To evaluate setting issues relevant to program adoption, implementation, and sustainability.adoption, implementation, and sustainability.
To help close the gap between research studies To help close the gap between research studies and practice by:and practice by:
Informing design of interventionsInforming design of interventions
Providing guides for Providing guides for adopteesadoptees
Suggesting standard reporting criteria to Suggesting standard reporting criteria to increase transparent reportingincrease transparent reporting
1515
DefinitionDefinition: The number, percent of target : The number, percent of target audience, and audience, and representativenessrepresentativeness of those of those who participate.who participate.
ExampleExample: 65% of chronic illness patients : 65% of chronic illness patients invited to group medical visit attended initial invited to group medical visit attended initial session; those declining more likely to be session; those declining more likely to be Latino.Latino.
RREE--AIM ELEMENTS: AIM ELEMENTS: REACHREACH
Key IssuesKey Issues: Does program reach those at : Does program reach those at highest risk? Are different promotional highest risk? Are different promotional approaches or visit options required?approaches or visit options required?
1616
DefinitionDefinition: Change in temporally appropriate : Change in temporally appropriate outcomes, and impact on quality of life and any outcomes, and impact on quality of life and any adverse outcomes.adverse outcomes.
ExampleExample: Mailed reminder/telephone outreach : Mailed reminder/telephone outreach program increased colon cancer and program increased colon cancer and mammography screening rates by 20%; with no mammography screening rates by 20%; with no adverse effects on quality of life or cardiovascular adverse effects on quality of life or cardiovascular screening rate.screening rate.
RREE--AIM ELEMENTS: AIM ELEMENTS: EFFECTIVENESSEFFECTIVENESS
Key IssuesKey Issues: Logic model helps to clarify anticipated : Logic model helps to clarify anticipated effects; quality of life provides common metric effects; quality of life provides common metric across conditions and interventions; anticipate across conditions and interventions; anticipate unintended consequences.unintended consequences.
1717
RERE--AAIM ELEMENTS: IM ELEMENTS: ADOPTIONADOPTION
DefinitionDefinition: Number, percent and : Number, percent and representativenessrepresentativeness of settings and clinicians of settings and clinicians who participate.who participate.
ExampleExample: Six months after CME introduction, 52% : Six months after CME introduction, 52% of primary care physicians have used panel of primary care physicians have used panel management toolmanagement tool——but only 30% of specialty but only 30% of specialty care providers.care providers.
Key IssuesKey Issues: Need to focus on : Need to focus on ““denominatordenominator”” and and barriers among nonbarriers among non--users. Do initial users. Do initial adopteesadopteesinclude peer opinion leaders?include peer opinion leaders?
1818
RERE--AAIIM ELEMENTS: M ELEMENTS: IMPLEMENTATIONIMPLEMENTATION
DefinitionDefinition: Extent to which a program or policy : Extent to which a program or policy is delivered consistently, and the time and is delivered consistently, and the time and costs of the program.costs of the program.
ExampleExample: Patients being asked about their : Patients being asked about their race/ethnicity 65% of the time; takes an race/ethnicity 65% of the time; takes an average 30 seconds of time, but not all average 30 seconds of time, but not all reception staff asking consistently. reception staff asking consistently.
Key IssuesKey Issues: Consistency across staff, program : Consistency across staff, program components, and time. Balance between components, and time. Balance between fidelity and local customization.fidelity and local customization.
1919
RERE--AIAIMM ELEMENTS: ELEMENTS: MAINTENANCEMAINTENANCEDefinitionDefinition: :
Individual/member target: LongIndividual/member target: Long--term effects and term effects and attrition.attrition.
Setting/clinician: Extent of discontinuation, Setting/clinician: Extent of discontinuation, modification, or sustainability of program. modification, or sustainability of program.
ExampleExample: At one: At one--year followyear follow--up, was 58% attrition from up, was 58% attrition from Internet weight loss program; those present Internet weight loss program; those present maintained weight loss. Only 40% of clinicians maintained weight loss. Only 40% of clinicians initially referring continued to do so.initially referring continued to do so.
Key IssuesKey Issues: Does attrition bias results; qualitative : Does attrition bias results; qualitative approaches to understanding program adaptation.approaches to understanding program adaptation.
2020
RECOMMENDED PURPOSE OF RECOMMENDED PURPOSE OF TRANSLATION/EFFECTIVENESS RESEARCHTRANSLATION/EFFECTIVENESS RESEARCH
To determine the characteristics of interventions To determine the characteristics of interventions that can:that can:
ReachReach large numbers of people, especially large numbers of people, especially those who can most benefit.those who can most benefit.
Be widely Be widely adoptedadopted by different settings.by different settings.
Be consistently Be consistently implementedimplemented by staff members by staff members with moderate levels of training and expertise.with moderate levels of training and expertise.
Produce Produce replicablereplicable and and longlong--lastinglasting effects (and effects (and minimal negative impacts) at reasonable cost.minimal negative impacts) at reasonable cost.
2121
RERE--AIM SELFAIM SELF--RATING EXERCISERATING EXERCISE
Choose a specific program youare considering for implementation
2222
RATE (RERATE (RE--AIM) YOUR PROGRAMAIM) YOUR PROGRAM
REACHREACH
1. What percent of your target population has received or participated in your program?
2. How confident are you that your program is successfully attracting all members of your target population regardless of race/ethnicity, gender, age, or socio-economic status?
1 2 3 4 5 6 7 8 9 10
____ %
Not at all Somewhat Completely
2323
RATE (RERATE (RE--AIM) YOUR PROGRAMAIM) YOUR PROGRAM
EFFECTIVENESSEFFECTIVENESS3. To what extent is your program achieving the full key
targeted outcomes that you had hoped for?
4. How confident are you that your planned program is being implemented without producing unintended adverse consequences (e.g., staff doing less of other recommended activities)?
1 2 3 4 5 6 7 8 9 10Not at all Somewhat Completely
1 2 3 4 5 6 7 8 9 10Not at all Somewhat Completely
2424
RATE (RERATE (RE--AIM) YOUR PROGRAMAIM) YOUR PROGRAM
ADOPTIONADOPTION
1 2 3 4 5 6 7 8 9 10
5. What percent of other clinics or organizations like yours do you estimate will be willing to participate in your innovation after you are done testing?
6. How confident are you that your program will be adopted by those settings and staff who provide services for people in your target population who have the highest need?
Not at all Somewhat Completely
____ %
2525
RATE (RERATE (RE--AIM) YOUR PROGRAMAIM) YOUR PROGRAM
IMPLEMENTATIONIMPLEMENTATION
1 2 3 4 5 6 7 8 9 10
7. What percent of the time are all the various program components of your intervention are being consistently delivered as intended?
8. How confident are you that the program can be delivered by a variety of staff representing a variety of positions and levels of the organization?
Not at all Somewhat Completely
____ %
2626
RATE (RERATE (RE--AIM) YOUR PROGRAMAIM) YOUR PROGRAM
MAINTENANCEMAINTENANCE
1 2 3 4 5 6 7 8 9 10Not at all Somewhat Completely
10. How confident are you that your program will be sustained in your setting 2 years after it is introduced?
9. How confident are you that the program will produce lasting benefits for participants (1-2 years or longer)?
Not at all Somewhat Completely
1 2 3 4 5 6 7 8 9 10
2727
RERE--AIM SCORING AND PROFILE SHEETAIM SCORING AND PROFILE SHEETScoreScore
1) Reach = Percent in Item #1 plus (10 x Item 2 Score) =1) Reach = Percent in Item #1 plus (10 x Item 2 Score) = ____/2 =____/2 = ________
2) Effectiveness = (10 x Score on Item #3) plus (10 x Item 4 2) Effectiveness = (10 x Score on Item #3) plus (10 x Item 4 Score) =Score) =
____/2 =____/2 = ________
3) Adoption = Percent in Item #5 plus (10 x Item 6 Score) =3) Adoption = Percent in Item #5 plus (10 x Item 6 Score) = ____/2 =____/2 = ________
4) Implementation = Percent in Item #7 plus (10 x Item 8 4) Implementation = Percent in Item #7 plus (10 x Item 8 Score) =Score) =
____/2 =____/2 = ________
5) Maintenance = (10 x Item 9 Score) plus (10 x Item 10 5) Maintenance = (10 x Item 9 Score) plus (10 x Item 10 Score) =Score) =
____/2 =____/2 = ________
2929
A couple of other programs
•Diabetes prevention class
•Environmental-level physical activity intervention
•Individual-level physical activity intervention
Application of Model to Diabetes Prevention
Tested in
Be a Fat DetectiveThree Ways to Eat Less Fat
Healthy Eating/Move Those MusclesBeing Active: A Way of Life
Tip the Calorie BalanceTake Charge of What’s Around You
Problem SolvingThe Four Keys to Healthy Eating Out
Talk Back to Negative ThoughtsThe Slippery Slope of Lifestyle Change
Jump Start Your Activity PlanMake Social Cues Work for You
You Can Manage Stress Ways to Stay Motivated
DPP Intervention Organization
Clinic
Dietitians
Delivery Sites
Fit
DemonstrationProject
AppropriateFor Question
Partnerships across organizationalstructure defined necessary results for decision to deliver broadly across multiple clinics
Broader Health Policy and Cultural Context
1. National Organizational CampaignTo Promote Healthy Lifestyles
2. Prevention Department Desire to Provide Diabetes Prevention Program
Diverse Samples
Multiple Settings
Freq
uent
Con
tact
Research Staff D
elivery
Spac
e Li
mits
Limited Staff Tim
e
Approval of PIC & MOA
Clinic Staff Engagement
Prevention Department
Primary C
are
Scheduling, Cost, & Co-pay
Elec
tron
ic M
edic
al R
ecor
d
Re-invention of intervention retainingcritical elements but drastically reducing
contact
Critical Elements
Note: PIC=Physician in Charge; MOA=Medical Office Administrator
Participant PathParticipant Path
5 weeks pass
1 week passes
7 weeks pass
1 week passes
1 week passes
RandomizationWeek 0
Mailing 2: WEEK 5
Mailing 3WEEK 13
Mailing 4WEEK 21
6 month follow-up visitTreadmill; Accelerometer;
Surveys; WeightWEEK 26
18 month follow-up visit
Treadmill; Accelerometer; Surveys; Weight18 MONTHS
1 week passes
2 weeks pass
Computer session; Choose rec facility
WEEK 1
RecruitmentDuring Stress
Test VisitTreadmill;
Accelerometer; Surveys; Weight;
Height
DAY 1
Mailing 1: WEEK 2
IVR 2WEEK 12
IVR 3WEEK 20
KEY:Clinic Visit IVR Call MAILING
12 month follow-up visit
Accelerometer; Surveys; Weight
WEEK 52
Begin contacting participants for 6-
mo f/u
7 weeks pass
IVR 1WEEK 4
What to do when you receive a voucher: 1) Establish a one year membership with the study participant
as you normally would for anyone interested in arranging a membership, effective immediately.
2) The study participant will give you his/her study voucher instead of paying for the one year membership.
3) Be sure that the study participant’s name is written on the voucher and fax it to the research study project manager (contact information listed below).
4) Kaiser Permanente will disperse payment for recreation center memberships once a week. The research project manager will phone the facility to make payment by credit card every Friday. Membership rights should be granted to the study participant immediately, despite the fact that you may not received payment on the day that membership begins. This payment method has been approved by the facility recreation manager.
As part of your partnership with the CardiACTION research study, you will receive vouchers for a free 6-month membership from some of our study participants.
Kaiser Permanente, Clinical Research Unit2550 S. Parker Rd., Aurora, CO 80014PHONE: 303-636-3117FAX: 303-636-3159
Name of Recipient:
One year Period Begin Date:
RECREATION FACILITY NAMEFACILITY ADDRESS
CITY, STATE ZIPPHONE
Please contact Jane Smith with questions: 303-636-3117
Instructions for Participating Recreation Facilities CardiACTION Research Study
This certificate is redeemable for a FREE one year membership at
Map #1. From Your Home
£¤470
§̈¦I 225
Arapahoe Rd
Peoria St
Parker Rd
Cherry Creek State Park
Your Home
Horizon Park
Hampden Ave
Cham
bers Rd
Quincy Ave
Tower R
d
Heartwood Athletic Club
Cherry Creek Equestrian Trail
Gun C
lub Rd
Arapahoe Rd
home work rec. center park lake
Nearby Parks: 1. Horizon Park: 3.00miles north from your home, features include fields and walking trails.
Walking/Driving direction : Start out going NORTHEAST on E. Powers Pl → Turn RIGHT onto E. Berry Dr. → Turn LEFT onto S. Danube St. → Turn RIGHT onto E. Smoky Hill Rd. → Turn LEFT onto S. Himalaya Rd. → End at Horizon Park.
2. Cherry Creek State Park: 2.50 miles west from your home, wildlife habitat, multi-user trails, pond and creek, camping, picnic sites (4201 South Parker Road, Aurora, CO 80014).
Walking/Driving direction: Start out going NORTHEAST on E. Powers Pl → Turn RIGHT onto E. Berry Dr. → Turn RIGHT onto S. Danube St. → Turn RIGHT onto E. Orchard St. → Turn RIGHT onto S. Parker Rd. → End at Cherry Creek State Park.
Parks, Trails and Recreation Centers
Map #2. From Your Work
§̈¦I 70
Colfax AveM
oline St
26th Ave Smith Rd
§̈¦I 225
Syracuse St
d
Ironton St.
Montview Blvd.
Colfax Ave
Your Work
Highline Canal Trail
Andrews Dr.
Rocky Mountain ArsenalHavana St.
Tower R
d.
Montbello Central Park
river major road trail
Nearby Parks: 1. Montbello Central Park: 2.00 miles northwest from your work, features include a baseball field, soccer field, lighted softball field and tennis courts.
Walking/Driving direction: Start out going SOUTH on Kittredge St. → Turn Right onto E. 40th Ave. → Turn RIGHT onto Andrews Dr. → End at Montbello Central Park.
2. Rocky Mountain Arsenal: 2.24 miles north from your work, 2400 acres, features include fishing, nature programs and tours, wildlife watching..
Walking/Driving direction: Start out going SOUTH on Kittredge St. → Turn Right onto E. 40th Ave. → Turn RIGHT onto Chambers Rd. → End at Rocky Mountain Arsenal.
Mail Example
Automated telephone messages
Automated telephone messages
• Example message…
• Public courts and recreation fields are great resources for getting in some physical activity. Basketball, tennis, lawn bowling, badminton, and soccer….are all activities that are fun and can be done at many places in Denver!
EnvironmentEnvironment
Env_OutsideResources_Options.wav
1.Local Parks
2.Paths
& Trails
3.Courts
& Fields
Env_OutsideResources_CourtsFields.wav
Env_OutsideResources_PathsTrails.wav
Env_OutsideResources_Parks.wav
Env_OutsideResources_ArrangeNewCall.wav
Call Number
1
Env_WorkResources_Options.wav
1.Fitness Options
2.Existing
3.Exercis
es
Env_WorkSimpleEX.wav
Env_WorkResources_Existing.wav
Env_WorkResources_Afford.wav
Env_WorkResources_ArrangeNewCall.wav
GoTo: Concluding Assessment Flow Chart
2
From: Individual Arm Entry Flow Chart:
Combo_OutsideResources_Options.wav
From: Environmental Arm Entry Flow
Chart:
4. Already know .
Pre_Env_OutsideResources_Options.wav Pre_Env_WorkResources_Options.wav
Env_HomeResources_Options.wav
1.Afford
2.Existing
3.Equipment
Env_HomeResources_Equipment.wav
Env_HomeResources_Existing.wav
Env_HomeResources_Afford.wav
Env_HomeResources_ArrangeNewCall.wav
4. Already know
Pre_Env_HomeResources_Options.wav
3
4. Already Know
Automated telephone callsAutomated telephone calls
• Tailored based upon goal attainment
• Potential to complete new action plan• Change Barriers
• Change Strategies
• Change Goals
• Assess mediators used for mailing tailoring
IndividualIndividual
4848
RE-AIM Questions to Ask and Ways to Enhance Overall Impact
RE-AIMDimension
Questions to Ask of Potential Programs Possible Ways to Enhance Dissemination
REACH(Individual Level)
What percent of the target population comes into contact?
Does program reach those most in need?
Will participants be representative of the targeted population?
Formative evaluation with potential users with those declining
Small scale recruitment studies to test methods
Identify and reduce barriers
Use multiple channels of recruitment
EFFECTIVENESS (Individual Level)
Does program achieve key targeted outcomes?
Does it produce unintended adverse consequences?
How will impact on quality of life (QOL) be assessed?
Incorporate more tailoring to individual
Reinforce via repetition, multiple modalities, social support and systems change
Use stepped care approach
Evaluate adverse outcomes and QOL for program revision and cost-to-benefit analyses
ADOPTION (Setting/Organizational Level)
What percent of target settings and organizations will use?
Will these organizations having underserved or high-risk populations use it?
Does program help the organization address its primary mission?
Conduct formative evaluation with adoptees and settings that decline
Recruit settings that have most contact with target audience
Provide different cost options and customization of intervention
Develop recruitment materials outlining program benefits and required resources
IMPLEMENTATION (Setting/Organizational Level)
How many staff within a setting will try this?
Can different levels of staff implement the program successfully?
Are different components delivered as intended?
Provide delivery staff with training and technical assistance
Provide clear intervention protocols
Consider automating all or part of the program
Monitor and provide staff feedback and recognition for implementation
MAINTENANCE (Individual and Setting Levels)
Does the program produce lasting effects at individual level?
Can organizations sustain the program over time?
Are those persons and settings that show maintenance those most in need?
Reduce level of resources required
Incorporate "natural environmental" and community supports
Conduct follow-up assessments and interviews to characterize success at both levelsIncorporate incentives and policy supports
Sure Sustainability Is Important, Sure Sustainability Is Important, But How Do You Do It? But How Do You Do It?
5252
OutlineOutline
Essential Elements for Sustainability Essential Elements for Sustainability
Developing a sustainability action planDeveloping a sustainability action plan
Active for Life Case StudyActive for Life Case Study
RecommendationsRecommendations
5353
Essential for SustainabilityEssential for Sustainability
Producing Evidence and Materials to Promote the Producing Evidence and Materials to Promote the Program. Program.
Funding Development Committee. Funding Development Committee.
Fostering & Utilizing Program Champions.Fostering & Utilizing Program Champions.
Keeping the Community Involved.Keeping the Community Involved.
Planning for a sustainable program delivery team. Planning for a sustainable program delivery team.
Integration into Existing Operations Integration into Existing Operations
Have markers to let you know when you are Have markers to let you know when you are ““donedone””..
5454
Sustainability Action PlanSustainability Action Plan
Purpose: To provide a contextually flexible process Purpose: To provide a contextually flexible process that will result in a higher likelihood of the programs that will result in a higher likelihood of the programs becoming institutionalized and maintained. becoming institutionalized and maintained.
Step 1: Address the factors that could influence the Step 1: Address the factors that could influence the sustainability of your program.sustainability of your program.
Step 2: Examine the current and future resources Step 2: Examine the current and future resources available for program delivery and identify changes available for program delivery and identify changes that could make the program more sustainable.that could make the program more sustainable.
Step 3: Detail specific strategies to ensure Step 3: Detail specific strategies to ensure sustainability. sustainability.
5555
Step 1 OverviewStep 1 Overview
Address the factors that could influence Address the factors that could influence the sustainability of your program. the sustainability of your program.
Sustainability Needs AssessmentSustainability Needs Assessment
Initial PlanInitial Plan
Enabling StrategiesEnabling Strategies
5656
Step 2 OverviewStep 2 Overview
Identify potential operational and program Identify potential operational and program adaptations that you think will make the adaptations that you think will make the program more sustainable while at the same program more sustainable while at the same time keeping the programtime keeping the program’’s essential elements s essential elements and examine the current and future resources and examine the current and future resources available to deliver the program. available to deliver the program.
Adaptation ActivityAdaptation Activity
Connections MapConnections Map
Funding Matrix WorksheetFunding Matrix Worksheet
5757
Step 3 OverviewStep 3 OverviewLongLong--term Sustainability: Putting it all term Sustainability: Putting it all
together with planning and periodic retogether with planning and periodic re--evaluation.evaluation.What are the action items?What are the action items?Who is responsible?Who is responsible?When will it be done?When will it be done?–– Producing Evidence and Materials to Promote the ProgramProducing Evidence and Materials to Promote the Program–– Funding Development Committee. Funding Development Committee. –– Fostering & Utilizing Program Champions. Fostering & Utilizing Program Champions. –– Keeping the Community Involved. Keeping the Community Involved. –– Planning for a sustainable program delivery team. Planning for a sustainable program delivery team.
Integration into Existing Operations. Integration into Existing Operations. –– After 6After 6--12 months review the evaluation criteria from each 12 months review the evaluation criteria from each
of the action plansof the action plans
5858
Active for LifeActive for Life
12 sites funded by RWJF to deliver one 12 sites funded by RWJF to deliver one of two evidenceof two evidence--based physical based physical activity programs for older adultsactivity programs for older adults
Completed the 3 step sustainability Completed the 3 step sustainability action planaction plan
Completed a final survey reflective of Completed a final survey reflective of perceptions of potential for the project perceptions of potential for the project to be sustained after grant award to be sustained after grant award periodperiod
5959
Likelihood of Sustainability
0
20
40
60
80
100
Definite Hope to Littlechance
No Chance
Perc
ent o
f Gra
ntee
s
6161
Projected AdaptationsProjected AdaptationsMajor:Major:
–– Use program as a basis for expansion (e.g., nutrition; Use program as a basis for expansion (e.g., nutrition; stress)stress)
–– Shift telephone counseling principles to current inShift telephone counseling principles to current in--person staff contacts. person staff contacts.
–– Offer through worksites to all adults over 18 years oldOffer through worksites to all adults over 18 years old–– Change program length Change program length –– Combine with or use as an adjunct to ongoing Combine with or use as an adjunct to ongoing
programsprograms
Minor:Minor:–– Fee for use programFee for use program–– Reduce handouts or replace with personal diariesReduce handouts or replace with personal diaries–– Use volunteer instructorsUse volunteer instructors–– Use program resources that are on lineUse program resources that are on line–– Reduce participant incentivesReduce participant incentives
62626262
Funding Proportions for Funding Proportions for SustainabilitySustainability
Internal Organizational Funds:Internal Organizational Funds:–– Average 45%; Range 0Average 45%; Range 0--100%100%
GrantsGrants–– Average 25%; Range 0Average 25%; Range 0--80%80%
External sourcesExternal sources–– Average 11%; Range 0Average 11%; Range 0--30%30%–– Includes sources such as state funds Includes sources such as state funds
or foundation supportor foundation supportFees for ServiceFees for Service
–– Average 19%; Range 0Average 19%; Range 0--80%80%
63636363
Funding Proportions for Funding Proportions for SustainabilitySustainability
Future Financial Support by Likelihood of Sustainability
0
2040
6080
100
Definite Hope to Definite Hope to Definite Hope to Definite Hope to
External Grant Internal Fees
Perc
ent o
f Fun
ding
6464
Board of Directors or ChampionsBoard of Directors or Champions
About 50% of About 50% of ““definitesdefinites”” compared to 25% of compared to 25% of ““hope hope toto’’ss”” have such a grouphave such a group
Examples of members of champion groups:Examples of members of champion groups:–– Advisory board of champions who provide Advisory board of champions who provide
support and advocacysupport and advocacy
–– Executive directors of organization who Executive directors of organization who provide assistance in grant seeking, hosting of provide assistance in grant seeking, hosting of program, and support marketing/recruitment program, and support marketing/recruitment effortsefforts
–– Regional health services board that is Regional health services board that is responsible for those who administer AFL.responsible for those who administer AFL.
6565
Infrastructure and Plans for Evaluation Infrastructure and Plans for Evaluation
Infrastructure For Program Delivery
0
20
40
60
80
100
Definite Hope to Definite Hope to Definite Hope to Definite Hope to
Manage Facilitators Evaluation Eval Plans
Perc
ent
* With one exception, all sites indicated the pilot evaluation data was important in the sustainability process
6666
Examples of InfrastructureExamples of InfrastructureCommitment from public health leadership to Commitment from public health leadership to continue the servicescontinue the servicesExisting staff that can continue to support the Existing staff that can continue to support the project beyond the Grant period. project beyond the Grant period. ALED program is in line with mission and vision, ALED program is in line with mission and vision, management in place to continue to directmanagement in place to continue to directCreated health manager position to oversee this Created health manager position to oversee this programprogramAFL coordinator integrated into regular staff with AFL coordinator integrated into regular staff with additional dutiesadditional dutiesProgram Management integrated into current Program Management integrated into current staff rolesstaff roles
6767
Mission, Vision, and PolicyMission, Vision, and PolicyThe majority of sites have components of the The majority of sites have components of the organizational mission that addressed physical organizational mission that addressed physical activity promotion for older adults.activity promotion for older adults.
Only 2 sites implemented policy directives Only 2 sites implemented policy directives related to AFLrelated to AFL
–– Board of Trustees recommendation for Board of Trustees recommendation for organizational administration to develop organizational administration to develop comprehensive plan to address obesity, poor comprehensive plan to address obesity, poor nutrition and sedentary lifestyles.nutrition and sedentary lifestyles.
–– Redirected Title III recreational funding to agencies Redirected Title III recreational funding to agencies offering evidence based programming. offering evidence based programming.
–– Health District added ALED facilitation and trainer Health District added ALED facilitation and trainer training to fitness focused Health Educators job training to fitness focused Health Educators job description. description.
6868
Changes in MissionChanges in Mission1/3 of grantee organizations had shifts in mission 1/3 of grantee organizations had shifts in mission or structureor structurePositives:Positives:
–– New focus on disease prevention/self managementNew focus on disease prevention/self management–– New focus on obesityNew focus on obesity
NegativesNegatives–– Change in focus from service delivery to broader Change in focus from service delivery to broader
supporting role for larger geographic regionssupporting role for larger geographic regions–– Culture of organization no longer supportive of Culture of organization no longer supportive of
psychosocial service offeringspsychosocial service offerings
6969
Grantee Perceptions of the Grantee Perceptions of the Sustainability Planning ProcessSustainability Planning Process
Organization related:Organization related:–– Proactive planning and increased sense of Proactive planning and increased sense of
accountabilityaccountability–– Template to generate a shared organizational Template to generate a shared organizational
vision on sustainabilityvision on sustainability–– Highlighted elements that may not have been Highlighted elements that may not have been
addressed with the time crunch when addressed with the time crunch when recruiting participantsrecruiting participants
Partnership related:Partnership related:–– Helped identify and coordinate key players, Helped identify and coordinate key players,
their strengths and roles, sharing our visionstheir strengths and roles, sharing our visions–– Formalized processFormalized process——gave some leveragegave some leverage–– Some felt partnerships were less involved with Some felt partnerships were less involved with
this than they expected this than they expected
70707070
What Worked Well for GranteesWhat Worked Well for GranteesWritten plans focused our strategiesWritten plans focused our strategiesFunding Matrix and OutcomesFunding Matrix and OutcomesValues Clarification ExerciseValues Clarification ExerciseIdentifying potential resources for supportIdentifying potential resources for supportBudget developmentBudget developmentTemplates in generalTemplates in generalPushed for collaborationPushed for collaborationIdeas around fostering program and Ideas around fostering program and organizational championsorganizational championsContinual review of action planContinual review of action planNeeds assessment tools & Connections mapNeeds assessment tools & Connections mapOpportunities to share with other granteesOpportunities to share with other granteesAdaptable processAdaptable process
71717171
What DidnWhat Didn’’t Work Well for Granteest Work Well for Grantees
Getting the partners involvedGetting the partners involvedFormality of reportingFormality of reportingSome components werenSome components weren’’t relevant for each t relevant for each granteegranteeFelt difficult to focus on sustainability when Felt difficult to focus on sustainability when recruitment was ongoingrecruitment was ongoingConnections map and funding matrixConnections map and funding matrixSometime asked to provide detail on things Sometime asked to provide detail on things that we didnthat we didn’’t have enough information ont have enough information onDidnDidn’’t come to a true consensus on the best t come to a true consensus on the best approach to sustainabilityapproach to sustainability
7272
SummarySummarySites that have a strong belief the future Sites that have a strong belief the future sustainability of their programs have:sustainability of their programs have:
–– A more balanced plan for financial support from A more balanced plan for financial support from internal, grant, external, and fee for service sourcesinternal, grant, external, and fee for service sources
–– A greater likelihood of having a champion groupA greater likelihood of having a champion group
–– A greater likelihood of infrastructure for AFL A greater likelihood of infrastructure for AFL program management and facilitatorsprogram management and facilitators
Coalitions were dynamic and, on occasion, Coalitions were dynamic and, on occasion, difficult to navigatedifficult to navigate
The sustainability plan was viewed positively by The sustainability plan was viewed positively by grantees, but there was some concern related to grantees, but there was some concern related to focusing on sustainability so early in the focusing on sustainability so early in the process.process.