Are you good enough? Ryutaro Hirose MD UCSF Transplant QI committee, Chair UCSF Dept of Surgery QI...
-
Upload
john-barton -
Category
Documents
-
view
212 -
download
0
Transcript of Are you good enough? Ryutaro Hirose MD UCSF Transplant QI committee, Chair UCSF Dept of Surgery QI...
Are you good enough?
Ryutaro Hirose MD
UCSF Transplant QI committee, ChairUCSF Dept of Surgery QI committeeUCSF Clinical performance improvement committee, ChairUCSF Patient Safety CommitteeFormer Chair, ASTS Standards and Quality committee
How do you know if you’re good?
• Look in the mirror– You ask people– Pt satisfaction surveys – important for CMS…but correlate with
quality??• CGCAHPS, HCAHPS
• National rankings• You don’t make errors – pt safety
– You don’t make errors that hurt people or kill people– Failure Mode Effect Analysis (FMEA)– RCA
• Processes and Outcomes– Benchmarks and comparators
Patients tell you that you are good?
Archives of Internal Medicine, Fenton et al 2012
Patients with highest satisfaction had increased mortality, worse outcomes
Comparison of Four National Rating Systems
National Hospital Rankings• US News and World Report, HealthGrades, Leapfrog
group, Consumer Reports• Also CMS Hospital Compare• NO hospital was ranked a high performer by all 4
rankings• ONLY 10% of hospitals (844) ranked high performer
by one ranking was ranked high performer by another
• Different rating methods, different focus, stresses different measures of quality
Quality Assessment and Performance Improvement
• Institute of Medicine– To Err is Human• Released 2000• Focused attention on medical errors, preventable
deaths• Systems approach to patient safety• Improving processes, not blaming individuals• Individuals must be held accountable and held
responsible
– Crossing the Quality Chasm
How do you judge quality? Team metric?
• 2014 – 2015 U of Kentucky Men’s Basketball• Overall 34-0• #1 Seed in NCAA tournament• National Championship?
Are you good enough?
• New England Patriots (2014-2015)• Super bowl championships– 2001, 2003, 2004, 2014
Are you good enough
• Individual provider metrics are confused with team metrics
• Baseball –– Pitchers are judged by W-L record– More appropriate metrics• ERA• WHIP
Individual metrics
• Michael Jordan– 6 time NBA champion– 10 x scoring title, PPG: 30.2 (career)
• Tom Brady– 4 Superbowl rings– 63.5 completion %, 53,258 yrds, 392 TDs, 143
INTs
Effect of environment on quality
• Domains of quality– Patient safety/safe care– Practice consistent with current medical
knowledge– Customization, ability to meet customer-specific
values• External forces to drive quality– Regulatory/legislative pressures– Economic and other incentives
Professional societies
• Set norms, standards of practice• Expectation that delivery of safe, high quality
care is standard • Promote culture of safety and improvement
Lapses in quality
• Misuse - Avoidable complications• Overuse – excess provision of service, not
supported by evidence• Underuse - failure to provide a service that
would have provided a favorable outcome
• First addressed by pt safety initiatives• Second and third addressed by evidence based
practice.
Quality Assurance and Performance Improvement
• QAPI programs– Mandated by CMS– Industry models– PDSA (plan-do-study-act)– Six Sigma (DMAIC)
Define/Measure/Analyze/Improve/Control– Lean Six Sigma
• LEAN – preserving value with less work• Waste reduction, increase efficiency, improve work flow• Production time/costs reduced
Cycle of continuous improvementPlan – Do – Study – Act
• Plan – Identify goal/purpose, define metrics of success
• Do– Implement components of plan
• Study– Monitor outcomes, test validity – ID areas for improvement
• Act– Close cycle, adjust goal– Change methods, reformulate theory
Six Sigma/Total Quality Management
• Seeks to improve qulaity by identifying and removing errors and minimizing variability
• DMAIC– Define, Measure, Analyze, Improve, Control
• Combined with lean manufacturing• LEAN Six Sigma– Address flow and waste
QAPI programs
• Mandated to have process AND outcome measures – for three phases
• Pre transplant• Transplant• Post transplant
18
How is quality measured at liver transplant centers
• Mandated data collection and submission to UNOS• Transplant Centers– Pretransplant – Transplant Candidate Registration form– Transplant – Transplant Recipient Registration form– Post transplant – Transplant Followup forms
• Organ Procurement Organizations– Deceased donor data– Performance data
What do we do now?
SRTR PSR’s– Waitlist mortality– Transplant rates– 1 yr and 3 yr patient and graft survival
• Used by UNOS/OPTN to flag programs, used by CMS as well
Center specific results
• Candidates– ‘shot selection’
• Referring physicians• Nephs, hepatologists• Selection committee/selection
critieria• Surgeon
• Donors– Donor demographics– Donor management
• Affects outcome
– Donor selection• In general• For specifc recipient
• Donor/recip interactions– Matching– Size
• Operative course– Redo, blood loss– Technical performance– Anesthesia
• Post op management– ICU– Surgeons/medical– ID
Variation in outcomes
• UNOS/MPSC• CMS– National coverage decisions e.g.• Liver - 1991 - if meets survival minimum
– (1-Yr: 77%, 2-yr: 60%)
– Two consecutive reporting periods of worse than expected outcomes
– O-E > 3.0; O/E ratio of events > 1.5, p<0.05– Results in review, possible SIA
Center specific results
• Integrated signals– We play a team sport
• Wait list mortality– Referral patterns
• Who are referred to us
– Selection criteria/behavior• Who are listed
– Wait list management• PCPs, referring nephs, heps• De-listing• Workup• Accurate documentation of comorbidities
26
Scientific Registry of Transplant Recipients
• Current Contract is held by the Chronic Disease Research Group of the Minneapolis Medical Research Foundation (Used to be held by the U of Michigan group)
• Responsible for designing and carrying out rigorous scientific analyses of data and disseminating information to the transplant community – including– transplant programs– organ procurement organizations– policy makers– transplant professionals– transplant recipients– organ donors and donor families– and the general public.
27
SRTR roles
• Policy development – evidence based• Collaborative efforts between the transplant
community, the SRTR, and the OPTN. • Policy-making is the OPTN's responsibility, the SRTR
plays a critical role in policy development through ongoing data analyses designed to provide policy makers with the information necessary to make informed decisions– LSAM modeling to predict changes in allocation policy, and
effects on wait list mortality, transplant rates, overall outcomes
28
SRTR – Scientific Registry of Transplant Recipients
• Risk stratified results– Wait list mortality– Transplant rates– Post transplant
• 1 month, 1 year, 3 year• Graft survival• Patient survival
• CUSUM charts– Real time analysis to trend events over time– Signals potentially concerning event rates
29
How is the expected pt and graft survival derived?
• Risk stratification model• E.g 1 yr graft survival model• Multiple factors• Donor factors • Recipient factors• Interaction between donor and recipient
30
Model for 1 year graft survival
• ABO compatible• Cold ischemia time• Local vs Regional• Recip factors
– Diagnosis of recip– PVT– Previous surgery– Alb, INR– Functional status– Race– Cr/dialysis– ICU/life support
• Donor age• Donor ht• Donor race• Donor use of drugs
(cocaine)• Donor hx of cancer• Hx of HTN• CDC high risk• Donor pressors/ddAVP• Split vs whole
Liver – adult 1 yr pt and graft survival
• Only one center had lower than expected 1 year patient survival on the last SRTR PSR
• Not a sensitive metric for quality
32
Risk stratification
• Too little?• Risk averse behavior• Inhibits innovation
• Too much?• Risky behavior• Futile transplants• Not best use of organs?
Should there be an absolute non risk stratified standard?
• Observed/Expected ratios• Limited/scarce resources– Lower absolute rates of success– Best use of limited resources?
• Argument that innovation is stifled
34
Quality control
• Available Statistical Techniques • Basic techniques
– Average mortality rate – Rolling average (last 10 cases) – Adjusted average mortality rates – # of consecutive failures
• Cox proportional hazard models – Adjusted for clinical characteristics – Incorporates risk adjustment
• Statistical process control – Continuous monitoring techniques (e.g. CUSUM)
CUSUM technique
• Statistical control charts developed to study industrial processes– Designed to ‘signal’ if there is a deviation from
accepted production standards• CUmulative SUM – used in quality control in
industry to trend events over real time• CUSUM techniques recently being used in
medicine and to track surgical outcomes
36
CUSUM – cumulative sum
Cumulative Sum (CUSUM) is technique of monitoring outcomes – Began as a method to monitor industrial processes – Produces a graphical output that can be tracked over time.
• CUSUM monitoring utility in health care recognized in 1970s – Limited by poor data collection – Lack of risk adjustment techniques
• Recent innovations let to expanded interest – Steiner et al. Incorporated risk adjustment – Axelrod et al. Effective in multi-center assessment in a retrospective
study – Kalbfleisch and Biswas- now utilizes survival analysis rather than a
binary (alive or dead) logistic analysis
37
CUSUM charts
• Graphical representation of outcomes for process – Can be risk-adjusted charts for important donor and recipient
characteristics – Plot outcomes over time to compare the results with expected
outcomes based on a national model of mortality or graph failure – 2 types: O - E charts and One-sided charts – Trends in the plot line suggest improving or declining outcomes – Once the trend line reaches a certain predefined level (one-sided
charts) or exceeds a certain slope (O - E charts) the CUSUM signals
CUSUM chart
• Graphical representation of observed vs expected events over time
• Risk adjusted CUSUM chartgs• One-sided CUSUM– Control limit is set, defines signaling threshold
• Two-sided or O-E CUSUM
43
A kidney transplant center
44
CUSUM conclusions
• CUSUM charting provides a reliable, risk adjusted method of tracking outcomes of a clinical process
• Can be “tuned” to balance the need for sensitively to detect clinical failures with the requirement to limit the number of false positive signals
• Graphical output is easily interpretable with a minimal amount of training
• Providing outcomes are promptly reported the CUSUM can provide real time insight into TC outcomes
National Transplant QI program
• Why?• 1 yr graft and pt survival – Gross outcomes– Team outcomes
• Provider specific outcomes• Need Benchmarks• Identify outliers• Identify areas for PI• Identify best practices
Surgical complications
• Identification of surgical complications is poor– In most cases, done retrospectively– By coders/billing personnel
• Poor risk adjustment• Lack of relevant benchmarks– Rate of SSI– Rate of thrombosis– Rate of biliary/ureteral complications– readmissions
Provider specific results
• Blood loss– Donor and recip characteristics
• Need for reoperation/intervention• Vascular complications• Bile duct/Ureter complications• Readmissions
TransQIP
• Accurate prospective data collection• Risk adjustment models to be developed and
tested• ACS/NSQIP tests inter-rater reliability and
accuracy• Establishment of national benchmarks,
comparison across centers/providers• Identification of best practices
The future of quality and reimbursement
• Value based purchasing• Expansion of PQRS program– From 0.5% incentive to 2% penalty– 3-8% penalty/incentives
• Maintenance of competency– Quality assurance activities
Summary
• The environment is changing• Quality will take center stage• Legislative/Regulatory environment• Reimbursement/Financial disincentives and
incentives• Value = Quality/Cost
Future directions• Patient safety
– Operative debriefings– Preventable errors in the OR – retained FB, errors of omission in the OR
• National TransQIP program– Collaboration between ASTS and ACS– Relevant outcomes– Benchmarks– Identify areas for improvement, identify best practices
• We can always do better, refine processes, increase pt safety• Patient satisfaction – modify Surgical Care CAHPS• Improve access, eliminate disparities• Composite Pre transplant Metric
– Waitlist mortality– Offer acceptance rate– Transplant rate (geographically adjusted)