Engaging with data
-
Upload
nuffield-trust -
Category
Healthcare
-
view
39 -
download
0
Transcript of Engaging with data
![Page 1: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/1.jpg)
Engaging with data
Martin Utley, Director of the Clinical Operational Research Unit,
University College London
Chair: Chris Sherlaw-Johnson, Senior Research Analyst,
Nuffield Trust
![Page 2: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/2.jpg)
Building tools with clinical teams for use in monitoring outcomes.
Martin Utley
UCL Clinical Operational Research Unit
![Page 3: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/3.jpg)
I received royalties from UCL Business in relation to co-authorship of the PRAiS software tool for monitoring 30-day survival following paediatric cardiac surgery.
Declaration
![Page 4: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/4.jpg)
An insentient machine
![Page 5: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/5.jpg)
? One can monitor outcomes to identify problems
One has absolute standards (some components to ¼ inch, others to a thousandth)
If there is a problem, the machine can be fixed.
To diagnose and fix problem, one has to stop production and open the box.
Stopping production comes at a cost, so one has to have a degree of
confidence that there is a genuine problem.
To what extent do these conditions apply to health care?
An insentient machine
![Page 6: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/6.jpg)
Survival to 30 days following
paediatric cardiac surgery
![Page 7: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/7.jpg)
Variable Life Adjusted Display (VLAD)
with Partial Risk Adjustment in Surgery (PRAiS)
![Page 8: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/8.jpg)
Variable Life Adjusted Display (VLAD)
with Partial Risk Adjustment in Surgery (PRAiS)
Pagel C, Utley M, Crowe S, Witter T, Anderson D, Samson R, McLean A, Banks V, Tsang V, Brown K, Real time
monitoring of risk-adjusted paediatric cardiac surgery outcomes using variable life-adjusted display: implementation
in three UK centres, Heart (2013) 99:1445-50.
In use for local monitoring of outcomes at UK surgical centres since 2013.
![Page 9: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/9.jpg)
In use for local monitoring of outcomes at UK surgical centres since 2013.
Dissemination to
wider clinical community
Clinical consultation / contribution / leadership
Development of risk
model
Design of data displays
![Page 10: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/10.jpg)
In use for local monitoring of outcomes at UK surgical centres since 2013.
Dissemination to
wider clinical community
Clinicians led
engagement with surgical units,
National audit body & other stakeholders
![Page 11: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/11.jpg)
All episodes in a 30% validation
sample within development set:
Mortality: 3.5%, N=7890
High risk diagnoses
Mortality: 8.8%, N=667
HLHS
Interrupted aortic arch
Pulmonary atresia (inc PA+IVS)
Comorbidity
Low risk diagnoses
Mortality: 1.4%, N=3196
Fallot/DORV-Fallot type
VSD
Pulmonary stenosis
Aortic arch obst +/- VSD/ASD
ASD
Subaortic stenosis (isolated)
Aortic regurgitation
Procedure
Medium risk diagnoses
Mortality: 4.2%, N=4027
PDA
Acquired
EMPTY/Unknown
Functionally uni-ventricular heart
TGA+VSD/DORV-TGA type
Pulmonary atresia+VSD (inc fallot type)
Tricuspid valve abnormality (inc Ebstein)
Mitral valve abnormality (inc supra, sub)
TAPVC
AVSD
Aortic valve stenosis
Misc Cong
TGA & IVS
Common arterial truncus
Normal
All episodes in a 30% validation
sample within development set:
Mortality: 3.5%, N=7890
High risk diagnoses
Mortality: 8.8%, N=667
HLHS
Interrupted aortic arch
Pulmonary atresia (inc PA+IVS)
Comorbidity
Low risk diagnoses
Mortality: 1.4%, N=3196
Fallot/DORV-Fallot type
VSD
Pulmonary stenosis
Aortic arch obst +/- VSD/ASD
ASD
Subaortic stenosis (isolated)
Aortic regurgitation
Procedure
Medium risk diagnoses
Mortality: 4.2%, N=4027
PDA
Acquired
EMPTY/Unknown
Functionally uni-ventricular heart
TGA+VSD/DORV-TGA type
Pulmonary atresia+VSD (inc fallot type)
Tricuspid valve abnormality (inc Ebstein)
Mitral valve abnormality (inc supra, sub)
TAPVC
AVSD
Aortic valve stenosis
Misc Cong
TGA & IVS
Common arterial truncus
Normal
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Year
Percentage episodes with empty co-morbidity fields
New ways of using diagnostic codes
clinically led / meaningful
then simplified for purposes of
risk model
Any old way of including comorbidity
knowingly included poor quality data
acknowledging clinical nous
commitment to long game
Development of risk
model Old ways of using operation codes
accepted existing schemes
impact on model “performance”
impact of model use
Year
% missing
comorbidity
data
![Page 12: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/12.jpg)
PRAIS 1:
• 29 Specific procedures
• 3 diagnosis groups (low, medium, high)
• UVH
• Yes/no comorbidity
• Bypass/non-bypass
• Continuous weight
• Continuous age
• Age band (neonate/infant/child)
• Epoch effect
38 parameters
PRAIS 2:
• 16 spec proc groupings
• 11 diagnosis groupings
• UVH
• 4 “other illness” indicators
• Bypass/non-bypass
• Continuous nonlinear weight
• Continuous nonlinear age
• Epoch effect
36 parameters
PRAiS2 – developed on 21838 surgical episodes 2009-2014 from NCHDA data.
Recent update
![Page 13: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/13.jpg)
VLADs do not give an evaluation of the
effectiveness / benefit of a surgical
programme to the patient population.
VLADs are not evidence of good or bad
performance.
DATA
patient case-mix
structure / resources
care processes
professionals
Queensland
pyramid model of
investigation
Worked with 3 sites on design of software
We improved ease of data entry / use, shared strengths and
limitations, listened to concerns.
![Page 14: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/14.jpg)
Consultation on design of data displays
Added after consultation with units
Secondary outcomes we would have discarded if not for discussions with clinical team
![Page 15: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/15.jpg)
Weeks following release of PRAiS 2013
![Page 16: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/16.jpg)
Christina Pagel - How can we better support appropriate interpretation
and use of risk adjusted survival data?
PRAiS 1 launch
![Page 17: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/17.jpg)
What?
How?
Why? PRAiS2
Research team worked with clinicians, family representatives, press officers and journalists on resources to support understanding and use of risk-adjusted mortality data.
![Page 18: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/18.jpg)
childrensheartsurgery.info
![Page 19: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/19.jpg)
Launch childrensheartsurgery.info
![Page 20: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/20.jpg)
Launch childrensheartsurgery.info
![Page 21: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/21.jpg)
What?
How?
Why?
Funding NIHR HSDR PI Victor Tsang
Health Foundation PI MU
NICOR
Funding NIHR HSDR PI Christina Pagel
Funding NIHR HSDR PI Victor Tsang & Kate Brown
Funding GOSH Charity PI Kate Brown
None of the views expressed are those of DH, NIHR, Health Foundation, GOSH Charity etc.
![Page 22: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/22.jpg)
![Page 23: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/23.jpg)
2014/15 - 4,207 surgical episodes with 97 deaths (mortality rate 2.3%) – 89 deaths predicted Area under ROC curve 0.86 [0.82, 0.89]
PRAiS2 performance in test data
![Page 24: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/24.jpg)
Brown KL, Crowe S, Franklin R, McLean A, Cunningham D, Barron D, Tsang V, Pagel C, Utley M, Trends in 30-day mortality
rate and case mix for paediatric cardiac surgery in the UK between 2000 and 2010, Open Heart. 2015 Feb 14;2(1):e000157.
doi: 10.1136/openhrt-2014-000157.
Absolute survival rates completely absent from debate in 2013
Kate Brown
![Page 25: Engaging with data](https://reader030.fdocuments.us/reader030/viewer/2022021418/5872c7d81a28ab0c718b6959/html5/thumbnails/25.jpg)
PRAIS 2:
• 16 spec proc groupings
• 11 diagnosis groupings
• UVH
• 4 “other illness” indicators
• Bypass/non-bypass
• Continuous nonlinear weight
• Continuous nonlinear age
• Epoch effect
36 parameters
Performance assessed using cross-validation in main data set and prospective use in 2014/15 national data set.
PRAiS risk model recently updated
Close collaboration with an expert panel including of intensive care consultants, surgeons, cardiologists, data experts, analysts