Making Changes Last- Sustainability for the OTP Project Thomas R Zastowny, PhD

Reduce Waiting & No-Shows Increase Admissions & Continuation www.NIATx.net

Making Changes Last-Sustainability for the OTP

Project

Thomas R Zastowny, PhDNIATx Coach & Healthcare Consultant

& all OTP coaches

Reduce Waiting & No-Shows Increase Admissions & Continuation All authors share equally in this presentation


PRESENTATION

1. Background & Theory2. A Field Study3. Work-Sheets &

Methods4. British Sustainability

Model5. Summary6. Bonus Slides


CONCEPTS

NIATx has achieved “first stage validity”


CONCEPTS

1. Diffusion, spread and sustainability are inter-related and inter-dependent

2. A specific plan, customized to organization, culture and system, is required for maximal stability & portability and sustainability

3. NIATx has achieved “first stage validity”

4. Precise Definitions, Planning and Methods are required for sustainability


Background

• As we have witnessed the significant and powerful performance improvement changes associated with NIATx, our attention has turned to sustainability of improvements. Webster has defined sustainability in many ways but including these two succinct and clear elements: “A characteristic of a process or state that can be maintained indefinitely; to keep in existence, to maintain or prolong” What family of strategies then can help us maintain the important changes we have made?


Background

• Concept & Definition

Spread Diffusion

Sustainability


METHOD

• Concept & Definition

Spread

Diffusion

Sustainability

Scatter

Share

Switch

Stretch

A P

S D


Four NIATx AimsReduce Waiting Times

Reduce No-Shows

Increase Admissions

Increase Continuation Rates


Hunches Theories

Ideas

Changes That Result

in Improvemen

t

A P

S D

APS

D

A P

S DD S

P ADATA

Change CyclesPD-SA-Sustain

Rapid cycle changes should be do-able in 2 weeks !!!!!Maybe…..


ASK: Sustain What?

The Projects--PDSAs

The Culture of Improvement

Evidence Based Practice

Practice Based Evidence


Qualitative & Quantitative

99

Not everything thatNot everything that can be counted counts, can be counted counts,and not everything thatand not everything thatcounts can be counted.counts can be counted.

-Albert Einstein-Albert Einstein

Bonus Slide


The importance of sustainability

• Important to not only implement a change but also to see whether it continues – Pluye, et al. 2004

• Most implementation is able to be sustained for at least a limited amount of time - Porowski, et al. 2004; Scheirer, 2004

• Activities become routine when they reflect the collective values and beliefs of members – Capoccia, et al., 2007; Pluye, et al., 2004

NIATx 200 - March 25, 2009

12


Sustainability and Institutionalization-Business Case

• Effectively maintaining an operation that improves services

• Maintaining financial feasibility and producing positive outcomes through consistent interagency collaboration

• Programs ability to generate continuation, growth, and support

NIATx 200 - March 25, 2009 13


Oregon Health & Science University

Sustaining Change in NIATx

Katherine (Katie) Riley, Ed.D.Traci Rieckmann, Ph.D.

A Field Study


2 major Research questions

• NIATx goals: Increase access and retention– 1. Was this accomplished? – 2. Were the NIATx components sustained?

– Quantitative & Qualitative assessment to determine if this was accomplished.

NIATx 200 - March 25, 2009


Evaluation process• Qualitative data collected from 38 agencies

– Quarterly calls – 302 interviews– Site visits – 121 interviews + 119 focus

groups– Evaluator observations – 111 summaries

• Time period– Base grant funding – 18 mo. RWJ, R1 & R2; 36

mo. STAR (2003-2006)– NIDA follow-up – additional 18 mo. (2006-

2008)

NIATx 200 - March 25, 2009


Which core components remained?

• Change leader – 35 of 38 agencies• Top leadership support – 34 + 3 partial• Client focus – 34 +1 to some extent• Data collection – 33 + 3 some sources• Data review – 34 + 1 somewhat• Change teams – 28 agencies• PDSA’s – 22 agencies

NIATx 200 - March 25, 2009


BONUS SLIDE

Fidelity = SustainabilityUse the Tools-Project Charter

Data is your Friend- collect,aggregate,analyze;-Decision Support

PDSA your way to improvementTHE BUSINESS CASE & MISSION CASE


Other components still in use

• Strategy techniques:– Walkthroughs – 8 + agencies– Nominal group – 2

• NPO ongoing learning components:– Website – 1– Interest circle participation – 8 ~ others– Learning collaboratives – well liked– Monthly conference calls – mostly not

mentioned– Coaches – no longer under contract

NIATx 200 - March 25, 2009


Change Initiatives

• Most were sustained - key influences:

• Roadblocks and barriers:– Staffing turnover - 25 agencies“Change team meetings have been suspended due to staffing difficulties

and the PDSA cycles are not being used.”

– Staff resistance – 18 agencies“They abandoned the attempt to integrate the Session Rating Scale and the Outcomes Rating Scale into clinical sessions. Counselors never were very supportive. But, they have used the experience to make their treatment planning process more client centered.”

– Unforeseen consequences – 11 agencies– Too many changes at one time – 2 agencies

NIATx 200 - March 25, 2009


Change Initiatives

• Wings beneath your feet– Leadership support – 34 +3 partial– Data review – 34 + 1 (ongoing

monitoring)– Internal coaching - 16“I sent out emails to clinical staff, reminded them to give

feedback. I didn’t do that kind of stuff with the second team so staff just forgot.”

Another Change Leader said:

“We need to be refreshed, reminded, and then implement [changes] again.”

NIATx 200 - March 25, 2009


Nourish the Learning Community/Collaborative

Bonus Slide

• Use & Dissemination of the “Walk Through” as an accepted and powerful methodology

• Real Use of the Rapid Cycle Performance Improvement -Real people, Real data, Real improvement, Real fast

• The development and creation of the Business Case- first combinations of Quality, Cost, Improvement and Impact

• Creation and accelerated use of “Learning Collaborative (s) & Communities”


Experience speaks

Monitoring prevents negative spirals:“Participants noted that it is important to continue to monitor

changes after implementation. Otherwise, staff will tend to revert to old practices. Participants provided examples of previous changes undertaken in the outpatient programs that had not been sustained.”

Regular meetings allow important data review:“The agency was reminded of the importance of looking at data every

single month and having good indicators in place to flag problems.”

NIATx 200 - March 25, 2009 23


Culture change

• Implementation of policies and procedures; another mechanism to change and sustain culture:

“Staffing changes give challenges to sustainability. One good thing is that those changes that were working became standard so new staff didn’t know anything but the changed practices. It’s not extra; it’s just what we do!”

NIATx 200 - March 25, 2009 24


Not every change is a keeper!

• Data revelations!“We’ve sustained most of the changes but when something hasn’t

worked, we abandon it.”

• Client revelations!“We tried [making reminder calls] for group [attendance] but it was

too much: clients didn’t like all of the phone calls—they could be getting 4 phone calls a week. We amended the change because of their feedback.”

NIATx 200 - March 25, 2009 25


Inquiry (1) What organizational and

operational structures (e.g., policies, procedures, work guidelines for providing service, ownership of the process?) are in place to hold the gain?


Inquiry (2) What are the clinical, business and

oversight processes that assure continuation? (e.g., stability, integration into the organization culture, allowance of fluctuations within statistical control, sustain plans and staff ownership)


BONUS SLIDE-Project Charter/Change Form

Sustainability Plan (only complete if you are sustaining the changes) A. Who is the SUSTAIN LEADER?B. What CHANGES do you want TO SUSTAIN?C. What SUSTAIN STEPS are being taken to ensure that the changes stay

in place and that it is not possible to revert back to the old way of doing things?

D. What is the TARGET SUSTAIN MEASURE, i.e. the point at which the Change Team would intervene to get the project back on track?

E. What system is in place to effectively MONITOR the SUSTAIN MEASURE?


Inquiry (3) What are the outputs, impacts and

outcomes we continue to want to see to know sustainability is working? (e.g., definitions-, i.e. same day service, continuation & client participation, and intervals of measurement to gauge stability, increases in access and retention).


Inquiry (4) What is the Business Case for the

Improvement and Sustainability? ( e.g. cost benefit, dollars and sense, new revenue)


One Story ▲One organization’s plan for sustainability for access

included the following…(a) two policy changes to ensure same day treatment, (b) a mandate to revisit the process if same day treatment was unsuccessful for 2 contiguous days, (c) continuous measurement in the first year after the improvement using a step down approach (e.g., measure weekly, then monthly, then quarterly ,then yearly). A well crafted plan for sustainability is a must for longevity. Inclusion of these dimensions can help improve the precision of the sustainability plan and cover a wide reaching set of strategies to “hold the gain” across the organization. This strategy works equally as well within single and complex organizations, and state wide collaboratives.


British National Health Services Sustainability Model

The NHS Sustainability Model is a diagnostic scoring model designed to be used by each member of the core project team. The information collected provides the whole team with a really good understanding of areas that need improving in order to maximize the potential of sustainability within the project. An accompanying Sustainability Guide provides a range of practical ideas about how to actually make those improvements.

http://www.institute.nhs.uk/sustainability_model/general/welcome_to_sustainability.html



http://www.institute.nhs.uk/sustainability_model/general/welcome_to_sustainability.html

The Model

Interactive Assessment Version

Examples & Sustainability Videos

Bonus



Service Sustainability Rating for ____________________________

(Name of Program, Project or Improvement Initiative)

Select the level of each factor that best describes your situation and circle the corresponding number. Your complete

honesty is appreciated.The closer your score is to 100, the better chance of successful

sustainability. A score of 55 or higher offers reason for optimism while a score of 45 or lower suggests reason for concern



The NHS Sustainability Model provides a useful framework which helps you:

• Assess your improvement initiative against a number of key criterion for sustaining change

• Recognize and understand key barriers to sustainability relating to the specific local context

• Identify your current strengths in sustaining improvement

• Monitor progress over time



1. Benefits Beyond Helping Patients

2. Credibility to Affected Staff of Benefits From Improvement

3. Adaptability of Improved Process

4. Staff Involvement & Training to Sustain Process

5. Staff Attitude Toward Sustaining the Improved Process

6. Senior Leadership Responsibility for the Process



7. Clinical Leadership responsibility for the Process

8. Effectiveness of the System to Monitor Progress & Process

9. Fit with Organization’s Strategic Aims & Culture

10. Infra Structure to Sustain- e.g. Staff, Facilities, Equipment,Time


WORKSHEET(1) What organizational and operational structures (e.g.,

policies, procedures, work guidelines for providing service, ownership of the process?) are in place to hold the gain?

(2) What are the clinical, business and oversight processes that assure continuation? (e.g., stability, integration into the organization culture, allowance of fluctuations within statistical control, sustain plans and staff ownership)

(3) What are the outputs, impacts and outcomes we continue to want to see to know sustainability is working? (e.g., definitions-, i.e. same day service, continuation & client participation, and intervals of measurement to gauge stability, increases in access and retention).

(4) What is the Business Case for the Improvement and Sustainability? ( e.g. cost benefit, dollars and sense, new revenue)


WORKSHEETSustaining the Gains

Describe the change you want to sustain

What are the organizational structures that can be put in place to help preserve the process changes you have made?

What are the ongoing data needs that will help the organization know if the desired change is being sustained? Who will gather this data? Who will review it and when? Is there a standard meeting that could own this responsibility?

What is the business case for the improvement you want to sustain?


SUMMARYSustaining the Gains

1. PDSA SUSTAIN

2. Fidelity

3. Worksheets

4. Business Case

5. Sustainability Model

6. Plan for Attrition-Staff & Leadership

7. Build Culture of Sustainability

8. A simple plan for sustainability

9. Build it in-not added on-Quality By Design

10.Nourish the learning Collaborative


References

Akerlund, K. M. (2000). Prevention program sustainability: The state’s perspective. Journal of Community Psychology, 28, 353–362.

. Backer, T.E. (2000). The Failure of Success: Challenges of Disseminating Effective Substance Abuse Prevention Programs. Journal of Community Psychology, 28 (3), 363-373.

Capoccia, V.A., et al. (2007). Making “Stone Soup”: Improvement in Clinic Access and Retention in Addiction Treatment. Journal on Quality and Patient Safety, 33 (2), 95-103.

Commons, M., McGuire, T.G., Riordan, M.H. (1997). Performance Contracting for Substance Abuse Treatment. HSR: Health Services Research, 32 (5), 631- 650.

Fitzgerald, M. (2000). Operator assistance with process improvement. Addictions Management, 21-22. Gustafson D.H.: Designing systems to improve addiction treatment: The foundation. Alcoholism and Drug

Abuse Weekly 14, Nov. 4, 2002.Hoffman, , K.A., Ford, J.H., Choi, D, Gustafson, D.H., McCarty, D (2008). Replication and sustainability of

improved access and retention within the Network for the Improvement of Addiction treatment, Drug and Alcohol Dependence 98 (1-2) 63-69.

Johnson, K., Hays, C., Center, H., Daley, C. (2004). Building Capacity and sustainable prevention innovations: a sustainability planning model. Evaluation and Program Planning, 27, 135-149.

Lake, B., Walker R. (2005). Report on Sustainability and Expansion of North Carolina’s Drug Treatment Courts. Administrative Office of the Courts, 1-57

NIATx 200 - March 25, 2009


References

McCarty, D. et al. (2007). The Network for the Improvement of Addiction treatment (NIATx): Enhancing Access and Retention. Drug Alcohol Depend, 88(2-3), 138-145

Porowski, A.W., Burgdorf, K., Herrell, J.M. (2004) Effectiveness and sustainability of residential substance abuse programs for pregnant and parenting women. Evaluation and Program Planning, 27, 191

Pluye, P., Potvin, L., Denis, J. L.(2004). Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning, 27, 121-133

Pluye, P., Potvin, L. Denis, J.L., Pelletier, J. (2004). Program Sustainability: focus on organizational routines. Health Promotion International. 19 (4), 489- 500

Scheirer, M.A., (2005). Is sustainability Possible? A review commentary on Empirical studies of program sustainably. American Journal of Evaluation, 26, 3, 320- 347

Thompson, B., Lichtenstein, E., Corbett, K., Nettekoven, L., & Feng, Z. (2000). Durability of tobacco control efforts in the 22 Community Intervention Trial for Smoking Cessation (COMMIT) communities 2 years after the end of interventions. Health Education Research, 15, 353–366.

Thompson, B., &Winner, C. (1999). Durability of community intervention programs: definitions, empirical studies, and strategic planning. In N. Bracht (Ed.), Health promotion at the community level (pp. 137–154). Thousand Oaks: Sage

Wisdom, J. P., Ford, J. H., Hayes, R. A., Edmondson, E., Hoffman, K., & McCarty, D. (2006). Addiction treatment agencies' use of data: a qualitative assessment. Journal of Behavioral Health Services & Research, 33(4), 394-407.

NIATx 200 - March 25, 2009

Making Changes Last- Sustainability for the OTP Project Thomas R Zastowny, PhD

Documents

Transcript of Making Changes Last- Sustainability for the OTP Project Thomas R Zastowny, PhD