Diane M. Dowdy, Ph.D.Interventions PHSB 612

April 3, 2008

Sustainability

Outline of Topics for Outline of Topics for SessionSession

What is meant by “sustainability”?What is meant by “sustainability”?

What factors are related to sustaining What factors are related to sustaining interventions within an organization or a interventions within an organization or a community?community?

Funding strategies for sustaining Funding strategies for sustaining

What is “Sustainability?”What is “Sustainability?”

Definitions may differ; in general, it’s what Definitions may differ; in general, it’s what happens to a happens to a program or intervention program or intervention afterafter external program funding ends.external program funding ends.

What is a “program”? - “A set of resources What is a “program”? - “A set of resources and activities directed toward one or more and activities directed toward one or more common goals”common goals”

– Newcomer, et al., 1994Newcomer, et al., 1994

Why is Sustainability Why is Sustainability Important ?Important ?

Your thoughts? Your thoughts?

Is this something you should think Is this something you should think about? care about?about? care about?

Ethics? Creating a demand, identifying a Ethics? Creating a demand, identifying a need then…need then…

Keeping “good will” of community Keeping “good will” of community partners; their willingness to participate partners; their willingness to participate in future interventions/researchin future interventions/research

Sustainability is itself a research topic; Sustainability is itself a research topic; you may want to continue or fund you may want to continue or fund research into this phase of interventionsresearch into this phase of interventions

From “efficacy” to “effectiveness” in From “efficacy” to “effectiveness” in contextcontext

Context for Context for Sustainability: Sustainability: Program or Intervention Program or Intervention Life CyclesLife Cycles

Programs develop over time, thru stages:Programs develop over time, thru stages:– Initiation - Initiation - whywhy was the project started? was the project started? WhoWho

was/is involved?was/is involved?– Implementation – full or partial?Implementation – full or partial?– Sustained use or discontinued?Sustained use or discontinued?

Sustainability is affected by all earlier Sustainability is affected by all earlier stages – start planning for it early!stages – start planning for it early!

Program Life CycleE

xten

t of

Im

plem

enta

tion

wit

hin

each

org

aniz

atio

n

Time in Months

0 6 12 18 24 30

SustainedUse,

DiscontinuedOr

Replaced

Initiation

Impl

emen

tatio

n

Incomplete Use

Full Use

WhatWhat is to be Sustained? is to be Sustained?

A.A. Sustaining Sustaining benefitsbenefits or outcomes for or outcomes for consumers/clients/participantsconsumers/clients/participants

B.B. Sustaining specific program Sustaining specific program activitiesactivities

C.C. Sustaining enhanced community or Sustaining enhanced community or organizational organizational capacitycapacity (coalitions) (coalitions)

D.D. Sustaining Sustaining attentionattention to the issue or to the issue or problem; related to problem; related to replicationreplication of the of the interventionintervention

Sustaining the Sustaining the Benefits for ConsumersBenefits for Consumers

Intended patient/client outcomes are Intended patient/client outcomes are maintainedmaintained

– can the participant outcome measures be can the participant outcome measures be continued?continued?

DangerDanger - Sustaining a program that does - Sustaining a program that does notnot achieve intended outcomes achieve intended outcomes

Sustaining the Sustaining the program site program site does not does not necessarily sustain the same level of necessarily sustain the same level of benefitsbenefits

Sustaining Program Sustaining Program ActivitiesActivities

Usually occurs within a delivery Usually occurs within a delivery organization organization

Important: what Important: what componentscomponents were were implemented? Which sustained?implemented? Which sustained?

May be called “institutionalization” or May be called “institutionalization” or “routinization”“routinization”

““Sustaining the Sustaining the program site program site does does not necessarily sustain the same not necessarily sustain the same level of benefit”level of benefit”

Example: Breast Cancer screening Example: Breast Cancer screening program, implemented in 28 hospitals in program, implemented in 28 hospitals in Maryland.Maryland.– During 6 months when state funds were cut off During 6 months when state funds were cut off

for those under age 40, 24 of 28 (86%) for those under age 40, 24 of 28 (86%) hospitals stated they sustained “the program”hospitals stated they sustained “the program”

– ButBut – Only 44 % as many mammograms were – Only 44 % as many mammograms were provided by these hospitals during this period, provided by these hospitals during this period, compared to same period, prior yearcompared to same period, prior year

Does this example show 86% or 44% Does this example show 86% or 44% “sustainability”?“sustainability”?

Sustaining Community or Sustaining Community or Organizational CapacityOrganizational Capacity

Working with coalitions fits here Working with coalitions fits here

Coalitions can have benefits for members Coalitions can have benefits for members beyond the outcomes achieved for clients - beyond the outcomes achieved for clients - participation, networking, informationparticipation, networking, information

Does coalition have an organizational Does coalition have an organizational “home”? “home”?

Should “spin-off” organization be created?Should “spin-off” organization be created?

Should the coalition Should the coalition itselfitself be sustained or be sustained or emphasize another type of sustainability?emphasize another type of sustainability?

Example: Coalition Example: Coalition Changing or Evolving? Changing or Evolving?

In a follow-up to NCI’s COMMIT In a follow-up to NCI’s COMMIT community trial on smoking cessation, community trial on smoking cessation, 9 of 11 intervention cities (82%) still 9 of 11 intervention cities (82%) still had a had a coalitioncoalition one year after COMMIT one year after COMMIT funding ended, funding ended, but but they averaged about they averaged about 68% of their former scope of activities.68% of their former scope of activities.

--Lichtenstein, et al., 1996Lichtenstein, et al., 1996

AFL Partnerships…Generations AFL Partnerships…Generations PartnershipsPartnerships

Sustaining Attention to Sustaining Attention to Issue; Moving to Issue; Moving to ReplicationReplication

May involve social marketing and other May involve social marketing and other communicationscommunications

Might relate to theory and literature on Might relate to theory and literature on “social movements” “social movements”

Do the ideas survive as part of the on-Do the ideas survive as part of the on-going “fabric” of the community?going “fabric” of the community?

Not often addressed in research literature Not often addressed in research literature about sustainabilityabout sustainability

Methods: MA Scheirer’s Methods: MA Scheirer’s Review of 19 Studies Review of 19 Studies of Sustainabilityof Sustainability

Searched databases for relevant Searched databases for relevant empiricalempirical studies, for “health” or “healthcare” studies, for “health” or “healthcare” programs.programs.

Required data studying “sustainability”, Required data studying “sustainability”, “institutionalization” or “routinization.”“institutionalization” or “routinization.”

19 studies located, coded and reviewed19 studies located, coded and reviewed

Coding categories derived from conceptual Coding categories derived from conceptual framework of Shediac & Bone –framework of Shediac & Bone –

in in Health Education Research, 1998Health Education Research, 1998

Lit Review: Methods Lit Review: Methods (Continued)(Continued)

Number of sites or projects studied per Number of sites or projects studied per paper: paper: – Ranged from 5 to 787Ranged from 5 to 787

Their data collection methods:Their data collection methods:– Mail/telephone surveys – 11 studiesMail/telephone surveys – 11 studies– Case studies – 5 studiesCase studies – 5 studies– Other/multiple – 8 studiesOther/multiple – 8 studies

Many used only 1 respondent per siteMany used only 1 respondent per site

Results from Review:Results from Review:ExtentExtent of Sustainability of Sustainability

14 of 17 studies (82%) reported that 60% or 14 of 17 studies (82%) reported that 60% or more of their more of their sitessites sustained their sustained their “program”: “program”:

– Usually defined as the continued operation of at Usually defined as the continued operation of at least least somesome program activities program activities

2 other studies 2 other studies selectedselected sites having a range sites having a range of sustainability – for case studiesof sustainability – for case studies

ButBut – Programs that continued were not – Programs that continued were not necessarily at same level of client necessarily at same level of client benefitsbenefits

0 2 4 6 8 10

Number of Studies

Sustainability Is Possible!

Some Caveats…Some Caveats…

Most studies used telephone interviews – Most studies used telephone interviews – positive bias likelypositive bias likely

2 case studies using site visits found 2 case studies using site visits found lower extent of “institutionalization” of full lower extent of “institutionalization” of full programs programs

Few studied continuation of benefits for Few studied continuation of benefits for clientsclients

Few studies built on results of prior Few studies built on results of prior researchresearch

DiscussionDiscussion

Be Strategic!Be Strategic! What types of sustainability would What types of sustainability would

be relevant for be relevant for youryour project? project?

How to Sustain? How to Sustain?

What Factors Influence What Factors Influence Sustainability?Sustainability?

Variety of factors potential Variety of factors potential influences on sustainabilityinfluences on sustainability

Not a uniform set of factors across Not a uniform set of factors across all all programs/organizations/communitiprograms/organizations/communitieses

Factors influencing Factors influencing sustainability?sustainability?

Project design and characteristicsProject design and characteristics If intervention is If intervention is modifiedmodified from its original from its original

form; intervention is flexible, adaptable form; intervention is flexible, adaptable

Uses no paid staff; uses volunteers; Uses no paid staff; uses volunteers; low low costcost

Evaluation available & showed Evaluation available & showed effectivenesseffectiveness

Factors influencing Factors influencing sustainability?sustainability?

B. Factors in Organizational SettingB. Factors in Organizational Setting Good fit between program & host agency’s Good fit between program & host agency’s

missionmission, objectives, and operating routines, objectives, and operating routines Presence of an internal “Presence of an internal “championchampion”” Organization has strong existing capacityOrganization has strong existing capacity Benefits Benefits feltfelt by agency &/or staff by agency &/or staff

Factors influencing Factors influencing sustainability?sustainability?

C. Community Involvement & EnvironmentC. Community Involvement & Environment

Presence of Presence of external external funding, type of funding, type of fundingfunding

Non-monetary support by Non-monetary support by other other community organizations community organizations

12

5

4

13

4

12

12

12

9

0 5 10 15Number of Studies

Factors Related to Program Sustainability

Funding from other sources

Support from other organizations

Benefits visible to staff or clients

Fit with mission and procedures

Exising organizational capacity

Role of program champion

Use of evaluation data

Low cost/uses volunteers

Program is modifiable at local level

Aspects of Community Environment

Aspects of Organizational Setting

Program Design & Characteristics

Others mentioned: Use of TA; planning in advance; staff & ownership changes; strong implementation.

Working with Partners to Working with Partners to Plan for Sustainability – Plan for Sustainability – Relevance to You?Relevance to You?

Program components modifiable or low Program components modifiable or low costcost

Presence of a local program “champion”Presence of a local program “champion”

““Fit” with agency mission & goalsFit” with agency mission & goals

Benefits for staff/clients are visibleBenefits for staff/clients are visible

Non-monetary support from other Non-monetary support from other organizationsorganizations

Sustaining Sustaining Interventions from Interventions from Research ProjectsResearch Projects

A dilemma: intervention A dilemma: intervention efficacyefficacy may may not be known until end of funding, but not be known until end of funding, but planning for sustainability often needs to planning for sustainability often needs to start earlier.start earlier.

Ideally, funder Ideally, funder continuescontinues transition transition funding for effective interventions.funding for effective interventions.

Or, use interim outcome data to guess Or, use interim outcome data to guess probable efficacy, to plan for probable efficacy, to plan for sustainability before end of funding.sustainability before end of funding.

FundingFunding

“ “ Whenever anyone says it is Whenever anyone says it is not about the money, it not about the money, it isis about the money.”about the money.”

– Attributed to H.L MenkenAttributed to H.L Menken

Strategies: Strategies: InstitutionalizationInstitutionalization

Transfer from “special” project status Transfer from “special” project status to funding as an on-going agency to funding as an on-going agency activityactivity

Often desirable, if feasibleOften desirable, if feasible

Requires that agency Requires that agency hashas annual annual budget from continuing sourcesbudget from continuing sources

Also called “routinization”Also called “routinization”

External Grants or Fund-External Grants or Fund-raisingraising

May be needed, if institutionalization May be needed, if institutionalization not possiblenot possible

Start early in project to identify Start early in project to identify potential external funding sourcespotential external funding sources

Use networking contactsUse networking contacts

Consider both external grants and Consider both external grants and in-kind resourcesin-kind resources

External Grants or Fund-External Grants or Fund-raising (cont.)raising (cont.)

Funding sources:Funding sources:– Gov’t grantsGov’t grants– Local foundationsLocal foundations– CorporationsCorporations– Budget shared by Budget shared by

partnerspartners– Fund raising eventsFund raising events– Bequests Bequests – Individual donationsIndividual donations

In-kind In-kind resourcesresources– Personnel timePersonnel time– Recruit partners w/ Recruit partners w/

needed skillsneeded skills– Meeting/office spaceMeeting/office space– SuppliesSupplies– Lists of volunteersLists of volunteers– Networking contactsNetworking contacts– Recruit retired Recruit retired

professionals w/ professionals w/ needed skillsneeded skills

Health Insurance/Health Insurance/Fee for ServiceFee for Service

For example, obtain coverage under For example, obtain coverage under HMO/MedicaidHMO/Medicaid

Applicable for client services, training, other ?Applicable for client services, training, other ?

Usually requires extensive negotiations to Usually requires extensive negotiations to obtain change in coverage policiesobtain change in coverage policies

Start early to begin decision-makingStart early to begin decision-making

Fees from clients – What would they be Fees from clients – What would they be willing/able to pay?willing/able to pay?

System ChangeSystem Change

Implies that change becomes part of on-Implies that change becomes part of on-going operations – not a distinct going operations – not a distinct “program”“program”

Further funding might not be needed, if Further funding might not be needed, if full behavior change has occurredfull behavior change has occurred

Some funding may be needed – for Some funding may be needed – for training new personnel, for maintenancetraining new personnel, for maintenance

Beware of incomplete implementationBeware of incomplete implementation

Some Implications Some Implications For Working With Local For Working With Local ProjectsProjects

Does the intervention relate strongly to Does the intervention relate strongly to agency mission and culture?agency mission and culture?

Explore/consider ways to foster sustainability Explore/consider ways to foster sustainability internallyinternally

Identify local “champion” - someone with Identify local “champion” - someone with links with both top management & program links with both top management & program staff staff

Emphasize benefits for agency staff & Emphasize benefits for agency staff & prioritiespriorities

AFL: AFL: Addressing SustainabilityAddressing Sustainability

Part of grant applicationPart of grant application

Discussed at site visitDiscussed at site visit

Identified definition of Identified definition of successsuccess

Worked through strategic Worked through strategic planning processplanning process

Reviewed & revised Reviewed & revised annually annually

Sustainability ABC’sSustainability ABC’s

A A Identify & address factors that Identify & address factors that influence sustaining influence sustaining

BB Examine current & future Examine current & future resources resources & desired program modifications & desired program modifications

CC Long-term action planning Long-term action planning

ResultsResults

11 of 12 communities reported they 11 of 12 communities reported they would sustain AFL programs after would sustain AFL programs after the grant funding period…the grant funding period…

ExamplesExamples:: Oasis making ALED a regular course offeringsOasis making ALED a regular course offerings YMCA combining AC with existing exercise YMCA combining AC with existing exercise

programsprograms FirstHealth utilizing ALED as part of disease FirstHealth utilizing ALED as part of disease

management effortsmanagement efforts City of Berkeley adapting AC and continuing City of Berkeley adapting AC and continuing

with grant fundingwith grant funding

And now…the rest of the And now…the rest of the storystory

And now for the rest of And now for the rest of the story…the story…

What is reality…What is reality…

One year after the grant funds One year after the grant funds endend

And, later?And, later?