1

Public Reporting of Long Public Reporting of Long Term Care Quality:Term Care Quality:The US ExperienceThe US Experience

Vincent Mor, Ph.D. Brown University

2

BackgroundBackgroundLong history of scandals regarding long term care quality, particularly nursing homes

While preference for and supply of “community based” alternatives have grown in US, all acknowledge residentially based long term care must be part of any system

Home Health less scrutinized but many worry about care adequacy since hard to inspect

3

BackgroundBackgroundInstitute of Medicine Report in 1987 served as basis for nursing home reform also adopted by home care

Uniform Resident Assessment Instrument created in 1991 and became the basis for the creation of performance measures designed to stimulate quality competition through public reporting

Home Health Outcome and Assessment Information Set (OASIS) emerged independently

4

Background (cont.)Background (cont.)Using RAI Nursing Home Quality Measures tested, revised and published as “Nursing Home Compare” since 2002

More recent efforts to create composite measure incorporating Inspection results, Staffing Levels and Quality Measures have been widely promulgated

Home Health Quality Measures developed and tested and published as Home Health Compare since 2004

5

PurposePurposeSummarize US Experience with Development of Long Term Care Quality Measures

Review Conceptual and Technical Issues Facing the Construction of Long Term Care Quality Measures

Review Literature on Effects of Public Reporting of Quality Measures in Long Term Care

6

The Nursing Home Resident The Nursing Home Resident Assessment Instrument Assessment Instrument (RAI)(RAI)1986 Institute of Medicine Report on Nursing Home Quality Recommended a Uniform RAI to Guide Care Planning --MDSOBRA ‘87 Contained Nursing Home Reform Act Including RAI RequirementA 300 Item, Multi-Dimensional RAI Tested for 2 Years Mandated Implementation in 1991

7

Clinical Planning Basis of Clinical Planning Basis of the MDSthe MDS

Assessment Profile in Given Domain “Triggers” Potential “Risk” Status

Resident Assessment Protocol Reviewed to Determine Presence of Problem or High Risk of Problem

Care Planning and Treatment Directed to the Problem

Data Quality Contingent upon conduct of Clinical Care Planning Process

8

MDS BackgroundMDS BackgroundMDS Version 2.0 Introduced in 1996

Admission, Short Term and Quarterly Reassessments done on all Residents

Inter-State Variation with some requiring additional data

Since 1998 all MDS records are computerized and submitted to Centers for Medicare & Medicaid

9

10

11

MDS: Putting Practice into MDS: Putting Practice into ResearchResearch

12

CMS Quality MeasuresCMS Quality Measures“The quality measures, developed under CMS contract to Abt Associates and a research team led by Drs. John Morris and Vince Mor, have been validated and are based on the best research currently available. These quality measures meet four criteria. They are important to consumers, are accurate (reliable, valid and risk adjusted), can be used to show ways in which facilities are different from one another, and can be influenced by the provision of high quality care by nursing home staff.” CMS Web Site

13

CMS Quality Measures - Long CMS Quality Measures - Long TermTerm

Percent of Long-Stay Residents Given Influenza Vaccination During the Flu Season Percent of Long-Stay Residents Given Pneumococcal VaccinationPercent of Residents Whose Need for Help With Daily Activities Has Increased Percent of Residents Who Have Moderate to Severe Pain Percent of High-Risk Residents Who Have Pressure Sores Percent of Low-Risk Residents Who Have Pressure Sores Percent of Residents Who Were Physically Restrained Percent of Residents Who are More Depressed or Anxious (Looks back 30 days)Percent of Low-Risk Residents Who Lose Control of Their Bowels or BladderPercent of Residents Who Have/Had a Catheter Inserted and Left in Their Bladder Percent of Residents Who Spent Most of Their Time in Bed or in a Chair Percent of Residents Whose Ability to Move in and Around Their Room Got WorsePercent of Residents with a Urinary Tract Infection (Looks back 30 days)Percent of Residents Who Lose Too Much Weight (Looks back 30 days)

14

Physical Functioning- Physical Functioning- October/December 2009October/December 2009

State ADL Worse

Bed Bound

Move Worse

Decline in ROM

National 14.9%  4.7%  14.7%  6.6%  AK 17.4%  5.3%  14.6%  10.9%  AL 11.6%  6.5%  11.7%  5.4%  AR 14.2%  4.2%  12.6%  5.5%  AZ 13.9%  4.9%  14.6%  5.5%  CA 10.4%  7.1%  11.6%  6.2%  CO 15.3%  3.0%  14.9%  6.4%  CT 15.4%  2.5%  16.3%  4.8%  DC 13.4%  1.7%  13.2%  5.4%  DE 13.8%  4.1%  15.5%  8.1%  FL 12.7%  4.6%  12.5%  5.2% 

15

Psychotropic Drug Use- Psychotropic Drug Use- October/December 2009October/December 2009

State Anti-Psychotics Overall

Anti-Psychotics LOW Risk

Anti-Anxiety Agents

National 18.6%  15.6%  23.1% 

AK 11.2%  4.7%  21.5% 

AL 15.9%  14.0%  27.2% 

AR 17.9%  15.6%  21.1% 

AZ 19.2%  15.8%  21.5% 

CA 16.8%  14.0%  20.4% 

CO 18.6%  15.1%  18.1% 

CT 23.7%  21.2%  22.7% 

DC 13.6%  12.4%  13.4% 

DE 20.2%  17.8%  23.3% 

FL 12.2%  10.1%  27.5% 

16

CMS Quality Measures – CMS Quality Measures – Short stayShort stayPercent of Short-Stay Residents Given Influenza Vaccination During the Flu Season Percent of Short-Stay Residents Who Were Assessed and Given Pneumococcal VaccinationPercent of Short-Stay Residents With Delirium Percent of Short-Stay Residents in Moderate to Severe Pain Percent of Short-Stay Residents With Pressure Sores

17

Home Health Quality Home Health Quality MeasurementMeasurementOASIS began as a cooperative effort between home health agencies and researchers to develop simple “outcome” measures to track patients’ rate of improvement while in care

University of Colorado researchers worked with large Visiting Nurse Services to develop and test

CMS then funded multiple large demonstrations to implement the tool and use for quality measurement and case-mix reimbursement

18

Outcome Based Quality Outcome Based Quality ImprovementImprovement

Distinct measures of change in patient functioning, resolution of symptoms and ability to manage independently collected at the start and “end” of care (OR every 60 days)

Most Medicare home health is short term

Measures tested and revised with extensive case mix adjustment to allow for comparison across agencies and states

19

Risk-adjusted Home Health Outcome Risk-adjusted Home Health Outcome Report for Improvement of Activities of Report for Improvement of Activities of Daily LivingDaily LivingEXAMPLE:

Percent of Patients in Home Health Care whose ability to [Groom, Bathe, Dress Upper and Dress Lower Body] themselves improves between start of care and discharge

20

CMS OASIS Report – 2009CMS OASIS Report – 2009Rates of Improvement in ADLRates of Improvement in ADL

State Grooming Upper Dressing

Lower Dressing

Bathing

Alabama 71.0%  73.0%  75.0%  68.0%  Alaska 67.0%  66.0%  56.0%  59.0%  Arizona 67.0%  69.0%  67.0%  64.0%  Arkansas 68.0%  69.0%  70.0%  65.0%  California 71.0%  72.0%  70.0%  67.0%  Colorado 70.0%  71.0%  70.0%  64.0%  Connecticut 69.0%  69.0%  68.0%  62.0%  Delaware 68.0%  69.0%  70.0%  62.0%  District of Columbia 75.0%  79.0%  79.0%  72.0%  Florida 69.0%  69.0%  68.0%  66.0%  Georgia 71.0%  73.0%  74.0%  67.0% 

21

Risk-adjusted Home Health Risk-adjusted Home Health Outcome Report for Utilization Outcome Report for Utilization OutcomesOutcomes

Percent of patients who have received emergency care prior to or at the time of discharge from home health care.

Percent of patients who are discharged from home health care and remain in the community

Percent of patients who are admitted to an acute care hospital for at least 24 hours while a home health care patient.

22

Risk-adjusted Home Health Risk-adjusted Home Health Outcome ReportOutcome Report

State Any Emergent Care

Discharged to Community

Acute Care Hospital

Alabama 23.0%  64.0%  33.0%  Alaska 20.0%  71.0%  25.0%  Arizona 25.0%  67.0%  29.0%  Arkansas 24.0%  64.0%  32.0%  California 18.0%  72.0%  25.0%  Colorado 23.0%  69.0%  26.0%  Connecticut 27.0%  65.0%  32.0%  Delaware 20.0%  70.0%  26.0%  District of Columbia 21.0%  71.0%  26.0%  Florida 18.0%  70.0%  26.0%  Georgia 22.0%  68.0%  29.0% 

23

Conceptual Issues Inherent Conceptual Issues Inherent in Applying Quality in Applying Quality IndicatorsIndicatorsRequires “shared” interpretation of Quality

Assumes all Providers have same goals

Assumes Measured Quality Domains are Important

Indicators are NOT Quality per se, BUT often used as evidence in and of themselves

Assumes Facilities Accountable for most of the variation in the Indicator (e.g. outcomes)

Assumes Facilities Know how to Change Practice

24

Technical Issues That Can Technical Issues That Can Compromise Validity of QI’sCompromise Validity of QI’s

•Reliability & Validity of the data

•Multi-dimensionality of Quality & Indicators

•Stability of Estimates Sensitive to Sample Size

•Ranks can Overestimate Differences

•Patient Level Risk Adjustment Complex

•Differences in Assessment Practices Influence QI Scores & Comparisons

25

Reliability Studies: NHReliability Studies: NH

219 of 462 (47.4%) facilities approached chose to participate in full study (52.4% for HB and 45.6% for non-HB);

Non-participants were more likely to be for-profit, less well staffed and with more regulatory deficiencies

5758 patients (ave. 27.5/facility) included in reliability analyses;

119 patients assessed twice by research nursesPatients resemble traditional US nursing home patient

26

Reliability of “Gold Standard” Reliability of “Gold Standard” NursesNurses

Of 100 items, only 3 didn’t reach Kappa>.450%+ items had Kappa >.75

Pct. Agreement high even for ordinal items with variance

Item % Agree Kappa

DNR 91% .83

Memory 88% .63

Decisions 97% .89

Understood 96% .82

Understand 96% .80

Fears 97% .76

Wander 99% .85

Walk 95% .86

Pain Fx. 93% .78

27

Reliability of Facility RNs to Reliability of Facility RNs to “Gold Standard”“Gold Standard”

Of the 100 data items 28 had Kappa <.4 and 15 had Kappa >.75

Worst Kappa items were rare binary items like “end stage”, didn’t use toilet, recurrent lung aspirations, etc.

ADLs and other Functioning items had Kappa values above .75

28

Reliability of Constructed Reliability of Constructed Quality Indicators: NHQuality Indicators: NH

Quality Indicators are composites of several RAI items; a definition of the denominator and of the conditions required to meet the QI definition

The inter-rater reliability of a QI is a function of the reliability of all the component items defining the algorithm

29

Prevalence and Inter-Rater Agreement and Prevalence and Inter-Rater Agreement and Reliability of Selected Facility Quality Reliability of Selected Facility Quality

Indicators  [N=209 homes]Indicators  [N=209 homes]   Avg. Avg. QI QI

Prev Prev raterate

FacilitFacility Avey Ave

SD of SD of QI QI

Prev Prev raterate

Ave Ave Kappa Kappa

for for Items Items

used in used in QIQI

% % Agree Agree Resch Resch

& & facility facility RNs on RNs on

QIQI

QI QI KappKapp

aa

Behavior Problems Behavior Problems High & Low Risk High & Low Risk Combined Combined

.20.20 .10.10 .71.71 89.889.8 .61.61

Little no activitiesLittle no activities .12.12 .12.12 .28.28 65.365.3 .23.23

Catheterized Catheterized .07.07 .05.05 .71.71 92.592.5 .67.67

Incontinence Incontinence .62.62 .13.13 .88.88 91.491.4 .78.78

Urinary Tract Urinary Tract InfectionInfection

.08.08 .05.05 .53.53 89.189.1 .45.45

Tube FeedingTube Feeding .08.08 .05.05 .73.73 98.198.1 .83.83

Inadequate Pain Inadequate Pain ManagementManagement

.11.11 .08.08 .85.85 86.586.5 .87.87

30

Facility QI Reliability Facility QI Reliability Variation: Bladder/Bowel Variation: Bladder/Bowel IncontinenceIncontinence

kappa Bladder/Bowel Incontinence (High and Low Risk)

1.00

.94

.88

.81

.75

.69

.63

.56

.50

.44

.38

.31

.25

.19

.13

.06

0.00

Nu

mb

er

of

Fa

cilit

ies

70

60

50

40

30

20

10

0

Std. Dev = .21

Mean = .78

N = 209.00

31

Facility QI Reliability Facility QI Reliability Variation: Inadequate Pain Variation: Inadequate Pain ManagementManagement

kappa Inadequate Pain Management

1.00

.94

.88

.81

.75

.69

.63

.56

.50

.44

.38

.31

.25

.19

.13

.06

0.00

-.06

-.13

Nu

mb

er

of

Fa

cilit

ies

30

20

10

0

Std. Dev = .30

Mean = .50

N = 209.00

32

Reliability Studies: Home Reliability Studies: Home HealthHealth

Fewer inter-rater reliability studies of OASIS

More expensive to send two nurses at separate times on the same day to do the same assessment

Largest Reliability Study done as part of research to develop case-mix reimbursement system

ADL and other function items yield high levels of reliability; symptoms achieve “ok” reliability

33

Selected Inter-Rater Reliability Selected Inter-Rater Reliability Results from OASIS testResults from OASIS test

Signs & Symptoms Sample Size

Percent Agreement

Kappa

1. Diarrhea 304 93.4% 0.44

2. Difficulty urinating or >=3x/night 304 91.5% 0.45

3. Fever 304 96.7% 0.63

4. Vomiting 304 97.4% 0.49

5. Chest Pain 304 95.4% 0.51

6. Constipation in 4 of last 7 days 304 92.1% 0.53

7. Dizziness or lightheadedness 304 89.1% 0.46

8. Edema 304 81.3% 0.50

9. Delusions 304 99.0% 0.66

10. Hallucinations 304 98.4% 0.44

34

OASIS Reliability Results: OASIS Reliability Results: FunctionFunction

Variable Sample Size

Percent Agreement

Kappa

Grooming: Current ability to tend to personal hygiene needs

304 74.7% 0.83

Dressing: Current ability to dress upper body with or without dressing aids

304 71.1% 0.83

Dressing: Current ability to dress lower body with or without dressing aids

304 77.0% 0.85

Bathing: Current ability to wash entire body 304 64.8% 0.80

Toileting: Current ability to get to and from the toilet or bedside commode

304 82.6% 0.86

Transferring: Current ability to move from bed to chair, on/off toilet or commode, tub, …

304 74.3% 0.88

Ambulation/Locomotion: Current ability to safely walk, use a wheelchair…

304 77.6% 0.87

35

Validity of the Data & Validity of the Data & MeasuresMeasuresValidity of the data shown by the extent to which items and measures behave as expected relative to “gold standard” variables or “hard” outcomes

Compared MDS diagnoses to Hospital discharge diagnoses

Looked at MDS predictors of survival

Related to MDS measures to research scales

36

MDS vs. CMS Hospital MDS vs. CMS Hospital diagnosesdiagnosesNeurological

Cerebrovascular disorders (ICD-9: 432, 434, 436, 437)� PPV = 0.73

Parkinson’s disease (ICD-9: 332)� PPV = 0.86

Alzheimer’s disease (ICD-9: 331)� PPV = 0.68

Brain degeneration (ICD-9: 331.0, 331.2, 331.7, 331.9)� PPV = 0.84

37

Men (CPS 0-1)

Women (CPS 2-4)

Months

One Year Survival by Gender & Cognition Level

38

39

Construct Validity: Cognitive Construct Validity: Cognitive Performance Scale & Performance Scale & CorrelatesCorrelates

Cognitive Performance Scale (CPS) Derived from 5 MDS Items

Strong (>.85) Correlation with MMSE

High Kappa with Global Deterioration Scale (.76)

Percent Patients with Dementia Increases as CPS Declines

MDS Communication Correlated (.85) with MMSE

ADL, CPS Symptoms & Select Diagnoses Related to Survival

40

Sample Size and QI StabilitySample Size and QI Stability

Providers and Consumers want QI to reflect not just what WAS but what WILL BE; SOQI stability is desiredQI must be based upon minimum # observationsCorrelation between quarters among QIs variesCorrelation among prevalence based QIs is high because same individuals assessed each quarterCorrelation between quarters among incidence and change based QIs lower and VERY sensitive to sample size

41

Residents’ Expected Rates Residents’ Expected Rates of Change on Quality of Change on Quality IndicatorsIndicatorsOver 90 day period 77.1% of residents still in facility do not change on ADL, 14.7% decline and 8.2% improve. Over 12 months 58% of residents in home don’t change and 30.2% decline.Similar pattern for Communication, Cognition and individual ADL itemsMeans that rates of decline are low and many residents are needed to estimate a home’s rate of ADL decline with confidence.

42

Estimated Sample Size for Estimated Sample Size for ChangeChange

     

Facility Size

Number Residents Decline Estimate

Residents Expected to Decline

20th Pctile Expected Residents Declining

80th Pctile Expected Residents Declining

 

  20 Beds 

12  

1 

  <1 

  1  

30 Beds 

16 

1 

  <1 

3  

50 Beds 

28 

 2 

  1 

  4 

  80 Beds 

45 

  4 

2 

  6 

100 Beds 

 56 

  5 

2 

7 

 150 Beds 

83 

  7 

  4 

 11  

200 Beds 

 

 117  

9   

5  

14

43

Facility QI Trend: Incidence of Late-Loss ADL Worsening(Stratified by Quality at Baseline)

0.000

0.050

0.100

0.150

0.200

0.250

199

9Q

3

199

9Q

4

200

0Q

1

200

0Q

2

200

0Q

3

200

0Q

4

200

1Q

1

200

1Q

2

200

1Q

3

200

1Q

4

200

2Q

1

200

2Q

2

200

2Q

3

200

2Q

4

200

3Q

1

200

3Q

2

200

3Q

3

200

3Q

4

200

4Q

1

200

4Q

2

200

4Q

3

200

4Q

4

Best-quality at baseline Good-quality at baseline

Mixed-quality at baseline Worst-quality at baseline

Long Term Predictability of Long Term Predictability of QualityQuality

44

1.5

2

2.5

3

3.5

Flu

De

aths

/10

0,00

0 P

opul

atio

n

.105

.11

.115

.12

.125

.13

AD

L D

eclin

e R

ate

2000

Q1

2000

Q2

2000

Q3

2000

Q4

2001

Q1

2001

Q2

2001

Q3

2001

Q4

2002

Q1

2002

Q2

2002

Q3

2002

Q4

2003

Q1

2003

Q2

2003

Q3

2003

Q4

2004

Q1

2004

Q2

2004

Q3

2004

Q4

2005

Q1

2005

Q2

2005

Q3

2005

Q4

ADL Decline Flu Mortality

Figure 2. Quarterly ADL Decline in Nursing Home Residents &Flu Mortality Rates in 122 CDC Monitored Cities: 2000-2005

Quality Fluctuation: Quality Fluctuation: SeasonalitySeasonality

45

Transforming QI Scores into Transforming QI Scores into RanksRanks

Many QI score distributions are skewed; many facilities with little or no problem and few facilities with many residents experiencing the problem.Median facility might be very similar to the “best” (the one with fewest problems)Transforming to ranks means saying there is a difference between the 10th and 40th percentile when there is little difference

46

Pressure Ulcer Prevalence Pressure Ulcer Prevalence Facility Distribution: Facility Distribution: Meaning of RanksMeaning of Ranks

47

598 Facilities

Me

dia

n R

an

ks

600

500

400

300

200

100

0

80% Confidence

Intervals

Median

598 Facilities

Me

dia

n R

an

ks

600

500

400

300

200

100

0

80% Confidence

Intervals

Median

Persistent Pain: Median RanksPersistent Pain: Median Ranks Anti-psychotics: Median RanksAnti-psychotics: Median Ranks

Variability in Ranking DistributionsVariability in Ranking Distributions

48

Complexity of Determining Complexity of Determining Appropriate Risk Appropriate Risk AdjustmentAdjustment•Risk Factors May not be Measured Independent of the Provider (tx) Effect•Potential for Over Adjustment as Great as Under Adjustment•How to Adjust for Socio-Economic Differences Known to Affect Health Behavior or Clinical Characteristics (e.g. PU not “seen” on African American NH pts until at Stage 2 OR Pain Harder to “see” in Cognitively Impaired & Oldest pts)

49

Risk Adjustment ComplexityRisk Adjustment Complexity

50

Why Adjust QIsWhy Adjust QIsFacilities should be compared on ‘level playing field’, acknowledging differences in

Types of residents admitted

Ability to ameliorate clinical characteristics thought to predispose to poor outcomes irrespective of care quality

Variability in measurement acumen of assessors

51

Average Admission Prevalence Average Admission Prevalence of Pressure Ulcers Across All of Pressure Ulcers Across All States, 1999States, 1999

LouisianaDistrict of Columbia

New JerseyMississippi

CaliforniaGeorgia

New YorkWest Virginia

MarylandAlabama

TennesseeSouth Carolina

OklahomaPennsylvania

NevadaIllinoisTexas

FloridaVirginia

KentuckyMichigan

Rhode IslandNorth Carolina

ArkansasOhio

ArizonaDelaware

IndianaHawaii

New MexicoAlaska

MassachusettsMissouri

WashingtonNew Hampshire

ColoradoConnecticut

KansasUtah

OregonWisconsin

IowaMaineIdaho

South DakotaMontana

NebraskaVermont

North DakotaMinnesotaWyoming

Admission Prevalence of Pressure Ulcers in 1999

.3.2.10.0

52

Hospitalization Rate in a 6-Month Period in 2000 Among Long-Stay NH Residents (Who Spent 90+ Days in the Facility)

24.9

23.9

23.2

21.5

21.3

21.2

20.9

20.7

20.1

20.0

19.7

19.4

18.8

18.7

18.3

17.7

17.3

16.9

16.6

16.5

16.3

16.3

16.2

16.1

16.0

15.6

15.5

14.9

14.9

14.6

14.2

14.0

13.7

13.4

12.6

12.3

12.3

11.3

10.8

10.4

10.1

9.5

9.2

9.1

8.9

8.7

8.5

8.3

0.0

5.0

10.0

15.0

20.0

25.0

30.0

LA MS

NJ

OK

TX

KY

AR

WV

GA IL FL

MO AL

TN

MD

OH

PA IN MI

NY IA SD

CA

VA

MA

NC

SC RI

NV

DE

KS

NE

MN

WY

CT

WI

AZ

CO

WA

ND

MT

VT ID OR

NH

ME

NM UT

% R

e-H

osp

italiz

ed

Source: MDS 2000; Medicare inpatient claims 2000.

53

54

Multi-dimensionality of QIsMulti-dimensionality of QIs

Consumers want to know “Best” nursing home & Regulators want to know where to focus their survey energies & Purchasers want to buy best.

If Quality is multi-dimensional no such thing as the “best”; most valuable dimension is a preference and will be individualized

Combining QIs that aren’t highly correlated may mask differences between facilities on important individual QIs

55

Does Poor Performance on Does Poor Performance on One Measure Mean NF is One Measure Mean NF is Poor?Poor?

•Average Correlation Among QIs is Low; •Anti-Psychotics and Restraints Correlated .04•What is a “Good” Home if QIs not Related?•Can Performance Measures Help Pick Good Homes?• Are Some Measures More Meaningful?•Should Users of Performance Measures Select the Measures they Value Most?

56

Summary Results of Summary Results of FactoringFactoring

Functional Decline

Mood/Behavior

Pressure Ulcers

Treatment & Condition[No Factor]

Worsening Bladder Poor Mood State [Prevalence]

Worsening Pressure Ulcers

Prevalent Catheter

Worsening Bowel Worsening Mood Prevalent Pressure Ulcer (Hi Risk)

Prevalent Restraint

ADL Decline Poor Mood w/o Anti-depressants

Prevalent Pressure Ulcer (Lo Risk)

Prevalent Anti-Hypnotic Use

Mobility Decline Behavior Problems [Prevalence]

Prevalent Anti-Psychotic Use

Cognitive Decline Worsening Behavior Weight Loss

Communication Decline

Worsening Relationships

Falls

Worsening Pain

57

Regression Modeling ResultsRegression Modeling Results

Results of relating each QI to all others revealed very low R2 for all Treatment & Conditions

While R2 higher for QIs within other factors, many conceptually unrelated QIs found to weakly predict other QI

Many QIs “load” (related to) on multiple factors QI “type” (e.g. prevalence, longitudinal, change) as

influential as QI content in factor Factor structure sensitive to which QIs included Many QIs totally uncorrelated with others

58

Provisional Test of Provisional Test of Combining Unlike Quality Combining Unlike Quality IndicatorsIndicators

Use 1999 MDS 2.0 from OH, NY & CA

Create Risk and Admission adjusted QIs for Pressure Ulcers, Anti-Psychotic Use and Pain

Correlate Measures: PU & Pain<.05; PU & Anti-Psych = -.15; Pain & Anti-Psych = .16

Only 13% of facilities in bottom half on all 3 QI’s; 5% if use bottom third on all measures

59

Public Reporting of QualityPublic Reporting of QualityNURSING HOME COMPARE allows consumers and advocates to identify facilities in their geographic area and to select using a “Five Star” global rating OR based upon global domains OR specific measures.

HOME HEALTH COMPARE allows consumers and advocates to identify agencies in geographic area and presents detail of many different Quality Indicators

60

61

62

63

64

65

66

Effect of Public ReportingEffect of Public ReportingResearch to date all done on nursing homes

Broken down by market served; long stay residential vs. short stay, post-acute

First wave of studies surveyed administrators to find out how they were responding

More recent studies use MDS data to examine changes in outcomes and admission patterns

67

Facility Response to Facility Response to ReportingReportingCastle (2005) initially found Administrators were skeptical and unconvinced that reporting mattered

Zinn & colleagues (2005) surveyed leaders and found they were aware of their scores and those of closest competitors; concluded spurred quality improvement

Castle (2007) concurred in a separate survey that more competitive markets affected response

68

Reporting Improve Quality?Reporting Improve Quality?Werner & colleagues (2009) found significant improvement in BOTH measured and unmeasured quality measures following public reporting – BUT general improvement trend

Mukamel et al (2007) looked carefully at initial response relative to prior quality patterns and also found improvement on most but not all measures

Werner, et al, 2010 also found improvement in post-acute quality scores

69

Reporting Alter Admissions?Reporting Alter Admissions?

Werner & Colleagues have examined whether facilities with worse quality scores in competitive markets manifest reductions in admissions

Very complicated; must infer from the data why someone entering a facility; should affect those entering to stay more, but hard to know who

However, evidence suggest small but significant changes in referral patterns favoring better quality

70

SummarySummary

Public Reporting of long term care providers’ quality performance is possible;

All measures are flawed, but no more than acute and ambulatory care

Pre-requisite is to have uniform data collected with relevant clinical detail AND should be able to be audited with penalties to minimize bad data

71

Summary (cont.)Summary (cont.)Constructing quality measures can be complex

Sample size, seasonality, risk adjustment are all important to assure the “fairness” of the system

Like case mix reimbursement, don’t want incentives for providers to limit access to sickest

Still at the infancy of understanding how consumers & advocates use the data

72

Issues for the FutureIssues for the Future

Preferable to have common items, measures and metrics across different types of long term care options, technically AND for consumers

Challenge of Creating Composite Scores consumers want that are technically less sensitive than domain specific measures

However, Movement to “Pay for Performance” requires we develop a solution