HCQI PSI SUBGROUP MEETING HCQI PSI SUBGROUP MEETING HCQI PSI SUBGROUP MEETING HCQI PSI SUBGROUP MEETING –––– 9 MAY 20129 MAY 20129 MAY 20129 MAY 2012 1

Conditions of Use & Reporting Methods Conditions of Use & Reporting Methods Conditions of Use & Reporting Methods Conditions of Use & Reporting Methods

of Patient Safety Indicators of Patient Safety Indicators of Patient Safety Indicators of Patient Safety Indicators

in OECD Countriesin OECD Countriesin OECD Countriesin OECD Countries

State of knowledgeState of knowledgeState of knowledgeState of knowledge

Dr M.A. Le Pogam, MD, MPH

Consortium CLARTE

Pôle IMER, Hospices Civils de Lyon

CCCCONSORTIUM LLLLOIRE-ATLANTIQUE AAAAQUITAINE RRRRHÔNE-ALPES

POUR LA PRODUCTION D’INDICATEURS EN SANTETETETE

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 2

• French Ministry of Health (DGOS) and National Authority for Health

(HAS) program of “Development and Validation of French Health Care

(Hospital) Quality Indicators”

• Endorsement by the Steering Committee for National Implementation of

Hospital Care Quality Indicators

• Integration into the National Quality Improvement F ramework

• Mandatory public reporting

• Comparative benchmarking

• Hospital accreditation

• P4P ( future project)

Context (1)

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 3

• CLARTE (Consortium Loire-Atlantique-Aquitaine-Rhône-Alpes pour la

production d’indicateurs de qualité en sanTE)

• Human Resource Management (HRM )

• Turn over and absenteeism of non-medical staff

• Medical and non-medical staff satisfaction

• Patient Safety (PS)

• Patient Safety Culture

• Patient Safety indicators

• French PSIs : Set of AHRQ PSIs

• Adapted from ICD-9-CM to ICD-10 by IMeCCHI (International Methodology Consortium for Coded Health Information)

• Adapted to French coding methodology and procedure classification (CCAM)

• Refined and validated

Context (2)

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 4

• Report on “Conditions of Use & Reporting Methods of Pa tient Safety

Indicators in OECD Countries » between 2001 and 2010

• Systematic review of PSI literature published between 2001 and 2010

• Semi-structured interviews and grey literature review

• Guidelines on

• Conditions of selection, adaptation and refinement of French PSIs

• Methods of estimation, validation and reporting of PSI rates at hospital- and area-level

• Institutional uses of PSIs for hospital care security assessment and enhancement

Context (3)

HCQI PSI SUBGROUP MEETING HCQI PSI SUBGROUP MEETING HCQI PSI SUBGROUP MEETING HCQI PSI SUBGROUP MEETING –––– 9 MAY 20129 MAY 20129 MAY 20129 MAY 2012 5

Conditions of Use & Reporting Methods Conditions of Use & Reporting Methods Conditions of Use & Reporting Methods Conditions of Use & Reporting Methods

of Patient Safety Indicators of Patient Safety Indicators of Patient Safety Indicators of Patient Safety Indicators

in OECD Countriesin OECD Countriesin OECD Countriesin OECD Countries

Qualitative approachQualitative approachQualitative approachQualitative approach

Le Pogam M.A, Januel J.M, Colin C.

Consortium CLARTE

CCCCONSORTIUM LLLLOIRE-ATLANTIQUE AAAAQUITAINE RRRRHÔNE-ALPES

POUR LA PRODUCTION D’INDICATEURS EN SANTETETETE

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 6

Method (1)

• Semi-structured telephone interviews with 10 intern ational experts in

the field of PSIs

Countries Organizations ExpertsCanada Canadian Institute for Health Information / IMECCHI Dr Chantal CourisCanada University of Calgary / IMECCHI Pr William GhaliGermany University of Niederrhein / IMECCHI / OECD HCQI Pr Saskia DröslerUSA University of California (UC Davis) / IMECCHI / OECD HCQI Pr Patrick RomanoBelgium Federal Public Service (FPS) - Public Health / OECD HCQI Dr Margareta HaeltermanUnited Kingdom Imperial College London - School of public Health / Dr Foster Unit Dr Paul AylinUnited Kingdom Imperial College London - School of public Health / Dr Foster Unit Pr Alex BottleSwitzerland University of Lausanne - Institute of Social and Preventive Medicine / IMECCHI Pr Bernard BurnandOECD OECD HCQI Pr Niek KlazingaOECD OECD HCQI Pr Vladimir StevanovicEU European Commission - DG Health and Consumers - Healthcare Systems Unit Dr Agnieszka Daval-Cichon

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 7

Method (2)

• Questionnaire

• Historical development of PSIs

• Routinely used PSIs

• PSIs selection, adaptation, validation and reporting processes

• Stakeholder uses of PSIs for patient safety improvement in hospital

• Grey literature

• Selected government and patient safety organization websites, journal articles

and reports, technical and working papers …

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 8

Results (Updated) – Canada (1)

• PSIs routinely used in Canadian Hospital Reporting Project (CHRP)

• In-Hospital Hip Fracture in Elderly (65+) Patients (rate per 1000 discharges) �modified PSI#8

• Obstetric Trauma - Vaginal Delivery with Instrument (rate per 100 instrument-assisted vaginal deliveries ) � PSI#18

• Obstetric Trauma - Vaginal Delivery without Instrument (rate per 100 unassisted vaginal deliveries ) � PSI#19

• Nursing-Sensitive Adverse Events for Medical Patients (per 1,000 medical discharges)

• Urinary tract infections, Pressure Ulcers (PSI#3), In-hospital fractures, Pneumonia

• Nursing-Sensitive Adverse Events for Surgical Patients (per 1,000 surgical discharges)

• Urinary tract infections, Pressure Ulcers (PSI#3), In-hospital fractures, Pneumonia

• No Pediatric Safety Indicator (AHRQ PDI)

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 9

Results (Updated) – Canada (2)

• Results reporting

• Hospital and peer group reports

• Risk-adjusted rates (logistic or Poisson regression model) and 95%CI limits

• Hospital level

• Health administration region level

• Provincial/territorial level

• Risk factors

• Age, gender and selected pre-admit comorbid diagnoses

• Bullet and trend graphs

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 10

Results (Updated) – Canada (3)

• Validation work

• Assessment of health administrative data quality• Development of Internal Consistency Tools based on logic checks

• Influence of coder characteristics

• Effect of physician alternate payment plans

• Assessment of chart documentation quality• Impact on administrative data quality

• Criterion validation studies : medical record revie ws as « gold standard »• PSI#5, #7, #12, #13, #15

• Effect assessment of Canadian coding standards on P PVs (diagnosis type)

• Construct validation studies• LOS, Re-admission rates

• Validation of a risk-adjusted model for national co mparisons

• Review of the differences across country-specific I CD-10 clinical modifications

• Effect on the international comparability of morbidity data

• International cross-validation of ICD-10 coding alg orithms• PSIs, Charlson index, Elixhauser index

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 11

Results (Updated) – Germany (1)

• PSIs not used yet for mandatory external quality as surance

(AQUA-Institut quality reports and hospital “benchmar k reports”)

• 3 modified patient safety indicators based on clini cal data

• modified PSI#3

• Pressure ulcer grade I-IV in older hospitalized patients (age ≥ 75 years, LOS ≥ 5 days, PU

POA or acquired after admission)

• Pressure ulcer grade II-IV in older hospitalized patients (age ≥ 75 years, LOS ≥ 5 days, PU

POA or acquired after admission)

• modified PSI#12

• Postoperative DVT (selected surgical procedures)

• Postoperative PE (selected surgical procedures)

• modified PSI#9

• Wound hematomas / postoperative bleeds (selected surgical procedures)

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 12

Results (Updated) – Germany (2)

• German health care policy ( Hospital Remuneration Law, KHEntgG ) plans

to “produce national quality assurance indicators ba sed on hospital

inpatient administrative data” by 2014

• PSI #2 – #19 adapted by 3M* company to German coding system

• Hospital care security self assessment and manageme nt

• PSI#1”complications of anesthesia” : not coded

• PSI#20 “Obstetric trauma – cesarean section” : clinica lly irrelevant

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 13

Results (Updated) – Germany (3)

• Results reporting

• Case-based (patients) and Hospital-based aggregate results

• Risk-adjusted rate (95% CI), median, range

• Time trend

• Between hospitals comparison

• Comparison with a reference range (target range, tolerance range)

• Risk-adjusted rates (95%CI)

• Stratification

• Logistic or Poisson regression model

• Risk factors

• Age, gender, selected pre-admit comorbid diagnoses, health related behaviours, socio-economic factors

• Sentinel indicators

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 14

Results (Updated) – Germany (4)

• Validation work

• Public reporting : multi-stage assessment procedure• Modified Delphi method• Statistical test• Recommendations for public reporting

• Criterion validation studies• Medical record reviews• Cross-validation with extensive hospital databases

• Construct validation studies• Validation of a risk-adjusted model for national co mparisons • Validation of a risk-adjusted model for internation al comparisons

• Stratification on age, gender, number of SDx, LOS

• Test-retest reliability of PSIs• Impact of national coding standards on the variabil ity of PSIs

• Country-specific ICD versions• PDx definition• POA flag for SDx• Admission type

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 15

Results (2011) – USA (1)

• PSIs routinely used in USA

• Hospital individual measures : PSI#2 - PSI#20• Modified PSI#3 “Pressure Ulcer stage 3 or 4”• Modified PSI#4 “Death among surgical inpatients with serious treatable complications”• Modified PSI#7 “Central venous catheter-related bloodstream infections”• Modified PSI#15 “Accidental puncture or laceration”

• Hospital composite measure : PSI#90 “ Patient safet y for selected indicators”• Composite of PSI#3, PSI#6, PSI#7, PSI#8, PSI#9, PSI#10, PSI#11, PSI#12, PSI#13,

PSI#14, PSI#15

• Area level : PSI#21 – PSI#27• Analogs of PSI#5, PSI#6, PSI#7, PSI#9, PSI#14, PSI#15, PSI#16

• NQF-endorsed PSIs• PSI#2, PSI#4, PSI#5, PSI#6, PSI#11, PSI#12, PSI#14, PSI#15, PSI#16, PSI#17• PSI#90

• PSIs adopted for CMS RHQDAPU• PSI#90, PSI#4, PSI#6, PSI#14, PSI#15

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 16

Results (2011) – USA (2)

• Results reporting

• Individual measures

• Provider observed rates

• Provider stratified observed rates (age, gender, payer, user-defined patient characteristics)

• Provider expected rates (risk-adjusted rates)

• Time trends

• Comparison with a reference population rate

• Risk-adjusted rates

• Stratification

• Logistic or Poisson regression model

• Risk factors

• Age, gender, DRG and comorbidity categories

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 17

Results (2011) – USA (3)

• Validation work

• PSIs Identification, selection and evaluation : mul ti-stage assessment procedure

• Step 1 : Concepts and Evaluation Framework • Step 2 : Literature review to identify potential PSIs • Step 3 : Candidate list of PSIs • Step 4 : Structured review of each PSI (face validation using modified Delphi method)• Step 5 : Empirical analysis

• Many criterion validation studies (PPV, Se, Sp)• Medical record reviews • Clinical data from VHA, CMS, CDC…• Volontary AEs reporting

• IHI Global Trigger Tool

• Construct validation studies• LOS, mortality rates, charges

• Validation of a risk-adjusted model for national co mparisons• Complex sampling method allowing sensitivity estima tion• Risk-adjustment method incorporating data on Presen t on Admission (POA)

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 18

Results (2010) – UK (1)

• PSIs routinely used in United-Kingdom (Dr Foster Hos pital Guide)

• Hospital individual measures

• PSI#2, PSI#7 - PSI#15, PSI#18, PSI#19

• Modified PSI#2 “Deaths in low-risk conditions”

• Modified PSI#7 “Central venous catheter-related bloodstream infections”

• Modified PSI#15 “Accidental puncture or laceration”

• Hospital composite measure

• Composite of PSI#3, PSI#4, PSI#9, PSI#11, PSI#13, PSI#15

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 19

Results (2010) – UK (2)

• Results reporting

• Individual measures

• Crude rates

• Expected rates (risk-adjusted rates)

• Mean number of SDx coded (“coding depth”)

• Real time monitoring tool (CUSUM control charts)

• Comparison with national average rate and 99,8%CI (banding)

• Funnel plots

• Risk-adjusted rates

• Logistic or Poisson regression model

• Risk factors

• Age, gender

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 20

Results (2010) – UK (3)

• Validation work

• Hospital Guide Consultation (Dr Foster Intelligence)

• Criterion validation studies (PPV, Se)

• medical record reviews

• Construct validation studies

• LOS, mortality rates, non-elective hospital re-admission rates

• Validation of a risk-adjusted model for national co mparisons

• 2012 : Campaign for a POA flag (Dr Foster Intelligen ce)

• Impact of POA flag on criterion validity and on trust rates

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 21

Results (2010) – Belgium (1)

• PSIs routinely used by the Belgian Ministry of Heal th (SPF-SPSCAE)

• PSI#1 - PSI#20

• Modified PSI#4.1 “Failure to rescue –acute kidney failure”

• Modified PSI#4.2 “Failure to rescue – sepsis”

• Modified PSI#4.3 “Failure to rescue – DVT/PE”

• Modified PSI#4.4 “Failure to rescue – cardiac shock or arrest”

• Modified PSI#4.5 “Failure to rescue – pneumonia”

• Modified PSI#4.6 “Failure to rescue – digestive hemorrhage or acute gastric ulcer”

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 22

Results (2010) – Belgium (2)

• Results reporting

• Rate-based indicators : PSI#3, PSI#4, PSI#11, PSI#1 2, PSI#13, PSI#15

• Hospital observed rates

• National observed rates (benchmarks)

• Sentinel indicators

• AEs lists

• Validation work• Criterion validation study (PPV, NPV) : PSI#3, PSI# 12, PSI#13

• medical record review

HCQI PSI SUBGROUP MEETING – 9 MAY 2012 23

Conclusion• Variability across countries

• Choice• Specifications• Risk-adjustment

• PSIs can be routinely used for national comparisons or public reporting after:• A standardized selection procedure

• Administrative data quality assessment

• Chart documentation quality assessment

• Criterion and construct validation studies

• Implementing diagnosis and procedure classifications modifications

• Validation of a risk-adjusted model

• Coding standards should be modified• PDx

• POA

Thank you for your attention

www.clarte-sante.fr