© 2006 CARF International. All rights reserved.
Performance Improvement: A Performance Improvement: A Quality Framework for Adult Quality Framework for Adult
Day ServicesDay Services
NADSA Annual ConferenceSeattle, Washington * October 10, 2009
Janice Nestler, Executive Director, Elder and Adult Day ServicesZoie Nikov, Business Development Executive, CARF International
2 © 2006 CARF International. All rights reserved.
Session ObjectivesSession Objectives
Discuss elements of an effective measurement system to help determine your ROI in key operational areas
Discover standards that support implementation of a performance measurement framework focused on quality improvement
Learn from providers that have successfully measured ROI and implemented quality improvement initiatives
3 © 2006 CARF International. All rights reserved.
Commission on Accreditation of Commission on Accreditation of Rehabilitation FacilitiesRehabilitation Facilities
International accreditation and standards setting organization Founded in1966 Private, non-profit, independent Customer Service Units
Aging Services Behavioral Health Child and Youth Services Durable Medical Equipment Prosthetic Orthotics & Supplies (DMEPOS) Employment and Community Services Medical Rehabilitation
Accredits more than 5,500 organizations in U.S., Canada, Europe, South America, South Pacific
Over 7.2 million people served annually Field driven Peer Review Process Moral owners – persons served Process emphasizes:
Persons served Performance improvement Business aspects of human service delivery Good management process
4 © 2006 CARF International. All rights reserved.
Commission on Accreditation of Commission on Accreditation of Rehabilitation FacilitiesRehabilitation Facilities
CARF International Tucson, Arizona
CARF Canada Edmonton, Alberta
CARF-CCAC and Aging Services
Washington, DC
5 © 2006 CARF International. All rights reserved.
Aging Services Aging Services Customer Service UnitCustomer Service Unit
Adult Day Services Aging Services Networks Assisted Living Continuing Care Retirement Communities Home and Community Services (coming in
2010) Person Centered Long Term Care
Communities Dementia & Stroke Specialty
6 © 2006 CARF International. All rights reserved.
ADS CARF Accreditation TodayADS CARF Accreditation Today
2008 revised ADS standards NADSA is CARF International
Advisory Council participant Tracking Medicare ADS Act of 2009 VA explorations Currently accredit programs in 22
states in US and in Ontario, Canada
7 © 2006 CARF International. All rights reserved.
State Acceptance of AccreditationState Acceptance of Accreditation
Acceptances of accreditation in lieu of regulatory requirements:
Iowa Maine Maryland Massachusetts Missouri Montana Nebraska New Mexico North Carolina Rhode Island Wyoming
Contact us if your state is developing, updating, revising statutory provision or regulations
© 2006 CARF International. All rights reserved.
The Accreditation ProcessThe Accreditation Process
9 © 2006 CARF International. All rights reserved.
The Accreditation ProcessThe Accreditation Process Intent to survey Use standards for 6 months – self
assessment Survey is scheduled – 2-3 day survey Peer surveyors - Administrative,
Program Orientation Tour Interviews with persons served, staff,
leadership, families Observation of processes Review of documentation Daily “Where we are” meeting Exit Conference Very consultative throughout
10 © 2006 CARF International. All rights reserved.
Cost of AccreditationCost of Accreditation
Intent to Survey = $950 Survey Fee = $1375 per surveyor per
day Average survey: 2 surveyors for 2
days = $1375 x 4 = $5500 Total = $6450 for 3-year term
Average cost: $2150 per year
for three years
11 © 2006 CARF International. All rights reserved.
Why is Accreditation Important?Why is Accreditation Important?
CARFAccreditation
12 © 2006 CARF International. All rights reserved.
Why: Big Picture ValueWhy: Big Picture Value
Preemptive: regulations can be invasive, disjointed, excessive
Standards: cover best practices for business and service
Process: Self assessment with third-party peer review – learn your strengths and areas for improvement
Bottom line value Public accountability
13 © 2006 CARF International. All rights reserved.
Why: Standards ConformanceWhy: Standards Conformance Rights are communicated, procedure
in place for formal complaints, review of formal complaints
Emergency procedures Safety during violent or
threatening situations Tests of procedures Procedure for responding to
critical incidents Evaluation of critical incidents
Evacuation routes are accessible in all areas where people receive services
Accessibility planning Performance measurement:
Data collection system Performance analysis Sharing performance
information Technology and systems plan
Human Resources: Performance management Students and volunteers Training on diversity issues,
customer service, outcomes measurement, performance improvement
Procedures for responding to subpoenas, search warrants, other legal actions
Review of billing Risk management plan Written procedures for receiving
counseling and appealing involuntary transfer, discharge, contract termination
Written agreement includes refund policies
Policies on protection of records from damage due to water damage or fire
14 © 2006 CARF International. All rights reserved.
Why: Bottom Line ValueWhy: Bottom Line Value
2009 study found average business increases in CARF-accredited organizations:
26% increase in persons served annually 37% increase in annual budget dollars
2008-2009 surveyed accredited provider findings: 98% stated their business was improved due to CARF 88% said CARF standards were relevant to direct
quality 93% said CARF’s surveyors provided valuable
consultation 88% said on-site survey was beneficial 88% said survey report guides improvement
© 2006 CARF International. All rights reserved.
Measuring for Measuring for SuccessSuccess
Determining Your
Return on Investment
16 © 2006 CARF International. All rights reserved.
Questions to Consider Questions to Consider For The Organization...For The Organization...
Who are your stakeholders?
Do you know if you are meeting their needs?
Can you provide information in simple and useful way?
How do you analyze if you have met strategic goals or targets?
Do you use all the data you gather?
17 © 2006 CARF International. All rights reserved.
Questions to Consider Questions to Consider For The Organization...For The Organization...
How do you make decisions in your organization now?
What role does data play in those decisions?
Can you accurately speak about your “performance”?
18 © 2006 CARF International. All rights reserved.
Thought Provoking QuestionsThought Provoking Questions
Where is the information to respond to questions from persons served, payers, leadership and others currently available?
Who collects the information and how are they trained? What system and/or tool is used to collect the
information? Where is the information stored? Who ensures that the data has been entered correctly? How do you know how different groups respond to
questions? New vs. long-term personnel Persons served who have been part of the organization
for <1 year vs. >5 years
19 © 2006 CARF International. All rights reserved.
Data Collection StandardsData Collection Standards
Collect data Needs of persons served Needs of other stakeholders Business needs of the organization
Allow for comparative analysis Used to manage and improve service
delivery
20 © 2006 CARF International. All rights reserved.
Data Collection StandardsData Collection Standards
Fact-based, decision making model Organization demonstrates how it
addresses data: Reliability Validity Completeness Accuracy
21 © 2006 CARF International. All rights reserved.
Demonstrating ReliabilityDemonstrating Reliability
The organization plans to take steps to ensure that data are collected in a way that could be reproduced at another time or by other data gatherers. What actually happens in practice? How are people trained to administer
tools? If standardized tools are used are
they administered properly?
22 © 2006 CARF International. All rights reserved.
Demonstrating ValidityDemonstrating Validity
The organization chose indicators, measures, and data elements that reflect what it intends to measure.
Tool (s) ask appropriate questions of the population to get at information needed
Did this happen? Have things changed and do the changes reflect the indicators actually used?
Has the population changed in such a way to invalidate the tools used because tools don’t get at the right information?
23 © 2006 CARF International. All rights reserved.
Demonstrating CompletenessDemonstrating Completeness
The organization takes steps to ensure that the data used for decision making are as complete as possible No groups of persons served are omitted
from the data gathering or analysis: representative sample
No data elements or fields are constantly missing
Database is checked for completeness of records before final analyses are run and decisions made
24 © 2006 CARF International. All rights reserved.
Demonstrating AccuracyDemonstrating Accuracy
The organization takes steps to ensure that data are recorded properly and that errors are caught and corrected
Spot checks of records of persons served are done to ensure that data taken from the record are correctly entered into database
The person responsible for the database routinely reviews the data to ensure that it appears correct
25 © 2006 CARF International. All rights reserved.
Service Delivery StandardsService Delivery Standards
Data collection system: Includes characteristics of persons
served Collects data at:
— Beginning of services— Appropriate intervals— End of services— Point (s) in time following services
26 © 2006 CARF International. All rights reserved.
Service Delivery StandardsService Delivery Standards
Measures of indicators of: Effectiveness of services Efficiency of services Service access Satisfaction and other feedback
from:– Person served– Other stakeholders
27 © 2006 CARF International. All rights reserved.
Measuring the Big Four Measuring the Big Four Domains of IndicatorsDomains of Indicators
EffectivenessOutcomes or results achieved for
persons served
Efficiency Resources used to get a result
AccessGetting into services
Feedback/satisfactionProcess and outcome
28 © 2006 CARF International. All rights reserved.
Service Delivery StandardsService Delivery Standards
Each indicator: Whom applied to How is data collected What is the performance goal Extenuating/ influencing factors
considered for analysis
29 © 2006 CARF International. All rights reserved.
Example: FrameworkExample: Framework
Expected Outcome
Data Source/ Obtained By
Time of Measure
Who Applied To
IndicatorObjectiveType of Measure
30 © 2006 CARF International. All rights reserved.
Premises for MeasurementPremises for Measurement
Information (converted data) is available in your organization.
Information is used to Review the implementation of mission and
core values of the organization, Improve the quality of programs and services Facilitate organizational decision making and
strategic planning. Measurement is woven throughout your
organization
31 © 2006 CARF International. All rights reserved.
Information Management Information Management DefinitionDefinition
Philosophy + processes = improvement to better meet needs and expectations of persons served
Designs or activities which
improve quality or reduce
costs Eliminate waste Reduce non-value-added tasks
32 © 2006 CARF International. All rights reserved.
Don’t let your data be like socks into a dryer—
What goes in must come out!
The CycleThe Cycle
© 2006 CARF International. All rights reserved.
Case Studies:Case Studies:ROI Measured and Quality ROI Measured and Quality
Improvements Improvements ImplementedImplemented
© 2006 CARF International. All rights reserved.
Case Study #1Case Study #1
Elder and Adult Day Serviceswww.eadscares.org
35 © 2006 CARF International. All rights reserved.
Elder and Adult Day ServicesElder and Adult Day Services
Founded Feb 1984 private free standing non-profit
First Adult Day Health serving East King County Expanded from 1 to 4 centers 300 participants
Purchased Bellevue Center 1989 East King County – Issaquah 1996 South King County – Des Moines 2003 Kitsap County – Bremerton 2004 to
Bainbridge 2008 2001 first program in Wa State to achieve CARF
accreditation/currently only accredited ADH
36 © 2006 CARF International. All rights reserved.
EADS - Key MessageEADS - Key Message
Elder and Adult Day Services provides relief to caregivers by offering day health programs focused on frail seniors and
developmentally disabled adults
37 © 2006 CARF International. All rights reserved.
EADS AccreditationEADS Accreditation
Commitment to deliver Adult Day Health Services in accordance with industry standards established nationally
Dedicated to being good stewards for our resources to ensure continuous quality improvement
EADS philosophy includes persons served not only the participant and their family but also staff, vendors, funders and the community at large
History of collaboration and successful partnerships for the benefit of each person served
Mission and values driven
38 © 2006 CARF International. All rights reserved.
Implementing CARF StandardsImplementing CARF Standards
Care Process for the Person Served The program has a written philosophy
of health and wellness for the person served that:
a. Is implemented to:(1) Address: (a) Function (b) Quality of life
(2) Promote healthy aging and well being
39 © 2006 CARF International. All rights reserved.
EADS ProcessEADS Process
Talked with participants Reviewed care plans/assessed acuity Met with Parents/Caregivers/physicians Talked with Staff Review/evaluate notes from all of the above Assessed EADS current program
What’s missing? What’s needed?
Research possible programs Evidence based exercise programs Appropriate, affordable and effective
40 © 2006 CARF International. All rights reserved.
What Next?What Next?
Participants needed an improved form of exercise Increase core strength and mobility Proactive form of fall prevention instead of
reactive Increase individual self esteem
Goals Improve each persons independence Reduce the risk of falls Increase individual dignity and self reliance Secondary gain of function in their ADL’s
41 © 2006 CARF International. All rights reserved.
Staying Active and Independent for Staying Active and Independent for Life (SAIL)Life (SAIL)
SAIL = Developed by geriatricians and Physical Therapists specializing in balance disorders by the WA Department of Health with funds from CDC
SAIL first evidence based exercise program specifically designed for Adult Day Health Centers
SAIL has become a part of the public domain SAIL P.T. specialist visited EADS centers and
met with the board for final approval EADS was selected to pilot this program in WA EADS staff were trained to deliver SAIL EADS OTR trained to train trainers
42 © 2006 CARF International. All rights reserved.
SAIL begins at EADSSAIL begins at EADS
SAIL began at one center plan to include all centers
Staff collect and provide data Fitness checks every 6 months using PT
standardized assessment completed by OTR Grant provide minimal equipment for
optimum benefit SAIL participants receive a pre-test
evaluation SAIL added to participants care plan SAIL progress is charted weekly
43 © 2006 CARF International. All rights reserved.
SAIL ROI at EADSSAIL ROI at EADS
Early data indicates after almost one year using SAIL Most significant gain from seniors who
were the most frail Anecdotally congenital balance
disorders have improve Traced lower blood sugar for person
with diabetes Seniors with dementia have improved
participation Caregivers report greater independence
and sense of well being when their loved one is at home
44 © 2006 CARF International. All rights reserved.
CARF Standards for AssessmentCARF Standards for Assessment
Participant benefits related to CARF Standard are as follows: Positively affecting multiple systems that
lead to increase in health and wellness Increase in overall function Increase in general health and well being Improved self esteem In line with state initiative for falls
prevention
45 © 2006 CARF International. All rights reserved.
Implementing CARF StandardsImplementing CARF Standards
Leadership Standard
CARF accredited organizations identify leadership that embraces the values of accountability and responsibility to the individual organizations stated mission. The leadership demonstrates corporate social responsibility
Standard: The identified leadership guides the following:
a. Establishment of the mission and direction of the organization
b. Promotion of value and achievement of outcomes in the programs and services offered
46 © 2006 CARF International. All rights reserved.
EADS ProcessEADS Process
EADS Strategic Plan Foundation for direction and sets priorities for EADS
— Tactic: Increased awareness of EADS through key message and brand awareness
Development staff researched grants In line with EADS mission, values and goals Consistent with EADS partnership philosophy
Taproot Foundation grant received $50,000 in professional staff for defined scope of
work Blueprint contract approved with deliverable
outcomes EADS staff /board worked with industry professionals Process includes environmental assessment
47 © 2006 CARF International. All rights reserved.
What Next?What Next?
Taproot Team produces several versions of EADS key branding messages for various print and verbal means Presented to EADS board for final approval All staff and board given written and in person
training on appropriate use of all key branding messages
Printed on newsletter, brochure, invitations giving envelopes, website, employee ads/orientations participant and family handbook
Used in verbal presentations
48 © 2006 CARF International. All rights reserved.
Key Message ROI for EADSKey Message ROI for EADS
EADS measures effectiveness after 1 year: More appropriate referrals from
advertizing E-Newsletter # of hits “looks” and click
through Increased level of contributors at
fundraisers Recognition at United Way/Day of
Caring events Differentiate from other adult day health
centers
49 © 2006 CARF International. All rights reserved.
CARF Standard AssessmentCARF Standard Assessment
Leadership Standard: Promotion of value and achievement of outcomes and achievement in the programs and services offeredDelegation of activities,
community assessment, using the feedback and collaborating on several levels of EADS leadership
50 © 2006 CARF International. All rights reserved.
EADS Survey ExpensesEADS Survey Expenses
Survey preparations (third time around) Purchase CARF-ADS standards CARF Lead takes an online refresher
course Complete online application for survey Inform all staff and board important dates Begin by reviewing standards with key
staff Evaluate Performance Improvement
Plans Let the fun begin
51 © 2006 CARF International. All rights reserved.
EADS AccreditationEADS Accreditation
WHY?
52 © 2006 CARF International. All rights reserved.
CARF ROI at EADSCARF ROI at EADS
Overall positive effect on EADS bottom line by: National Compass for our mission and purpose Enhanced participant health, independence and
well being Improved staff retention with enhanced esteem Higher caliber Board members/easier
recruitment Identify, demonstrate and measure benefits Additional key marketing message Third party independent quality assurance for
all vendors and persons served
© 2006 CARF International. All rights reserved.
Case Study #2Case Study #2
Senior Resource Groupwww.srgseniorliving.com
54 © 2006 CARF International. All rights reserved.
Measuring Staff TurnoverMeasuring Staff Turnover
Senior Resource Group Headquarters in Solana Beach, CA 15 communities in Arizona, California,
Florida and Oregon 1,000 employees, 2,800 persons served,
2,000 units Objective: Minimize staff turnover Domain: Efficiency Measure Target: Less than 50% annual turnover
(historical turnover rate 100%)
55 © 2006 CARF International. All rights reserved.
Measuring Staff TurnoverMeasuring Staff Turnover
Result: 100% to 60% in 5 years = 40% reduction in annual employee turnover
Learnings from analysis: implemented standards for tracking/trending, employee satisfaction survey, new employee orientation, longevity recognition program
56 © 2006 CARF International. All rights reserved.
Return on InvestmentReturn on Investment
*Track Data Before and After Implementing Standards*
Annual salary $25,000 ($10/hour) Turnover cost $37,500
Vacancy/replacement costs Recruitment costs Orientation/training costs Lost productivity costs New hire costs
Number of employees 50 Average turnover rate is 30% 15 employees X $37,500 = $562,500 annual cost Cost of accreditation $6450/3 year term of
accreditation or $2150 per year for 3 years
57 © 2006 CARF International. All rights reserved.
Bottom Line ValueBottom Line Value
Marketing: 1,450 more leads were received and 1,118 more tours given
Falls decreased by 2% Medication errors decreased by 34% Elopements decreased by 15% Vehicle maintenance decreased by $831
despite the addition of 10 vehicles Supply costs decreased on average, by
$2,700 per campus
© 2006 CARF International. All rights reserved.
Case Study #3Case Study #3
Youngstown Ohio Area Easter Sealswww.mtc.easterseals.com
59 © 2006 CARF International. All rights reserved.
Measuring Client SatisfactionMeasuring Client Satisfaction
Youngstown Ohio Area Easter Seals Boardman Adult Day Services Warren Adult Day Services
Objective: Maintain high client satisfaction
Domain: Effectiveness Target: Overall Satisfaction 90%
(historical satisfaction high 80%)
60 © 2006 CARF International. All rights reserved.
Measuring Client SatisfactionMeasuring Client Satisfaction
Result: 90% and higher client satisfaction Feedback is a major driver of
programmatic changes Feedback provides support for
additional resources, equipment, space and new services
61 © 2006 CARF International. All rights reserved.
Measuring Client SatisfactionMeasuring Client Satisfaction
Learnings from analysis: Implemented standards in key operational
areas Low satisfaction rates for parking,
accessibility and outside space led to decision to move programs to new facilities in order to meet client needs
Responding to the feedback has a direct positive impact on client satisfaction
— Consistent 90% and higher client satisfaction
62 © 2006 CARF International. All rights reserved.
Return on InvestmentReturn on Investment
*Track Data Before and After Implementing Standards*
Reduction in marketing costs to recruit new participants
Seattle Times: $1806 for one weekday ad (black and one standard color)
Invitation/announcement costs Food and beverage cost for one open house $250 Staff cost to plan, manage and attend event
Reduction in cost for print marketing materials Reduction in staff time spent orienting participant
and family members Cost of accreditation $6450/3 year term of
accreditation or $2150 per year for 3 years
© 2006 CARF International. All rights reserved.
Closing TipsClosing Tips
64 © 2006 CARF International. All rights reserved.
Nothing is less productive
than to make more efficient what should not be done at all.
-Peter Drucker
65 © 2006 CARF International. All rights reserved.
Closing TipsClosing Tips
Keep it very simple Get staff on board with how this can help Measure before you begin, during and after
you implement quality improvement efforts Don’t collect more data – take a hard look at
what you collect to ensure that it brings value Integrate business and service delivery Set realistic/achievable targets
66 © 2006 CARF International. All rights reserved.
Thank YouThank You
Janice Nestler, Executive Director, Elder and Adult Day Services [email protected]
Zoie Nikov, Business Development Executive, CARF Aging Services [email protected]
67 © 2006 CARF International. All rights reserved.
Contact CARFContact CARF
CARF Aging Services866-888-1122, www.carf.org
/aging
Rebecca Best, Resource Specialist
[email protected] Amanda Birch,
Administrator of Operations [email protected]
Susan Ganson, Resources Specialist [email protected]
0
50
100
1st
Qtr
4th
Qtr
East
West
North
Sue Matthiesen, Managing Director [email protected]
Zoie Nikov, Business Development Executive [email protected]
68 © 2006 CARF International. All rights reserved.
Insights from ColleaguesInsights from Colleagues
“CARF is an educational experience. The surveyor’s attitude was one of cooperation and support rather than roadblocks. We enjoyed working with CARF because of the precise nature of CARF’s philosophy. I am very excited working with CARF, as I believe CARF focuses on quality, competence, and character.”
- an accredited provider “CARF has helped us focus on the needs of our
customers. By defining ourselves clearly, evaluating ourselves professionally, we can locate funding sources, and articulate our staffing needs. CARF gave us many suggestions to generate needed income for our programs.”
- an accredited provider
69 © 2006 CARF International. All rights reserved.
Insights from ColleaguesInsights from Colleagues
“You set the standards…the way standards should be set – by professional field and consumer input. I have always appreciated that surveyors work full time in the field and understand the realities of trying to do a good job (and stay in business). Thank you for all the work that you do!”
- an accredited provider
“We were very happy that the expertise of the survey team matched our agency. This is the best thing we’ve ever done as an organization to improve our system of care.”
- an accredited provider
70 © 2006 CARF International. All rights reserved.
Insights from ColleaguesInsights from Colleagues “The opportunity for an objective and standardized review
of our programs validated our focus for service delivery. The opportunity for feedback, dialogue, and exchange of ideas provided the impetus for further creativity and identified areas for improvement.”
- an accredited provider “Our involvement with CARF has benefited the people we
serve beyond belief. We have become even more customer driven and outcomes based. CARF has demonstrated that you can be rural and deliver quality care.”
- an accredited provider “The standards were very supportive of the services,
philosophy, and mission of our organizations. The standards assisted us in fine-tuning our client-oriented program and in establishing systems for accountability.”
- an accredited provider
Top Related