Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6%...

31
©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D Informatics Research Seminars We ask that all participants mute their microphones Please note that the session is being recorded.

Transcript of Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6%...

Page 1: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

Informatics Research Seminars

We ask that all participants mute their microphonesPlease note that the session is being recorded.

Page 2: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

Health Care IT Advisor

The Decision MachineAnalytics and the Rise of AI

Carolina Health Informatics ProgramNovember 7th, 2018

Greg KuhnenSenior Research [email protected]

Page 3: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

3

Sources: “Analytics,” Wikipedia, last modified March 5, 2018, https://en.wikipedia.org/wiki/Analytics; English Oxford Living Dictionaries. Oxford: Oxford University Press, 2018 (also available at https://en.oxforddictionaries.com/); Health Care IT Advisor research and analysis.

AnalyticsThe discovery, interpretation, and communication of meaningful patterns in data.

–Wikipedia

Artificial Intelligence (AI)The theory and development of computer systems able to perform tasks normally requiring human intelligence…

–Oxford English Dictionary

Machine Learning (ML)The field of study that gives computers the ability to learn without being explicitly programmed.

–Arthur Samuel, 1959

Page 4: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

ROAD MAP4

The Data Dilemma1

2 AI and the New Machine Age

Page 5: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

5

Health Economics: An Unsustainable TrajectoryAnalytics Can Be a Powerful Tool for Bending the Cost Curve

Sources: Cottarelli C, “Macro-Fiscal Implications of Health Care Reform in Advanced and Emerging Economies,” IMF Fiscal Affairs Department, December 2010; McKinsey & Company, “mHealth: A new vision for healthcare,” 2010; Office of Economic Co-operation and development, ‘Health Spending”; Health Care IT Advisor research and analysis.

1) Global Burden of Disease Health Financing Collaborator Network, “Future and potential spending on health 2015-40: development assistance for health, and government, prepaid private, and out-of-pocket health spending in 184 countries,” The Lancet, 389, No. 10083 (2016); 2) According to the Australian Institute of Health and Welfare (AIHW) calculations; 3) New Zealand Ministry of Health projections; 4) GDP = Gross domestic product.

Per Capita Health Care Spending1 Health Expenditures as Percentage of GDP4

2016 20302016 2030

2016 2030 2016 2030

2016 2030

Australia2

9.6% 14.4%

2010 2030

New Zealand3

6.2% 8.9%

10.6%15.9%

11.0%18.0%

FranceCanada

United States United Kingdom

17.2%24.9%

9.7%13.5%

$9,237

$5,234

$4,751

$4,576

$4,032

$4,050

$3,749

$3,096

$12,448

$7,799

$6,437

$5,926

$5,606

$5,496

$5,002

$4,245

United States

Netherlands

Belgium

Canada

Australia

New Zealand

UnitedKingdom

Spain

Health Expenditure per Capita 2014

Health Expenditure per Capita 2030

Page 6: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

6

Technology Brings a Flood of DataSmartphones, and Genomes, and Sensors, Oh My!

Source: Health Care IT Advisor research and analysis.

1) IoT = Internet of things.2) RTLS = Real-time locating system.3) ROI = Return on investment.

Common Barriers to Adoption of New Sources• Privacy / security concerns• Immature technology• Lack of budget• Unclear benefit / ROI3

• Staff already overwhelmed with data

A Sampling of New Data Sources

Integrated Partners (e.g., social services, cross-continuum care)

Data from Outside Organizations (e.g., social)

Environmental Data (e.g., IoT,1 RTLS,2temperature, air

quality)

Genomes, Proteomes, Microbiomes, Metabolomes

Clinical Surveillance (e.g., public health, outbreak reports)

Remote Patient Monitoring(e.g., bedside monitors,

remote patient monitoring, consumer wearables)

Page 7: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

7

New Data SourcesMainstream Systems Cover Only a Fraction of Predictive Power

Acquire Data

Sources: Nash DB, “Population Health: Why It Matters,” Essentials in Population Health, Children’s Hospital Association, Thomas Jefferson University, October 2016; Health Care IT Advisor research and analysis.

1) RPM = Remote patient monitoring.

EMR(s)

Lab Systems

Pharmacy Systems

Imaging

Medical Claims

Prescription Drug Claims

Billing and Supply Chain

Scheduling Systems

Medical Care Data

10%

Sensors / RPM1

mHealth Apps

Patient-Reported Outcomes

Genetics

Socioeconomic Data

Patient Questionnaires

Human Biology

Data20%

Lifestyle and

Behavior Data50%

Social and Environmental

Data20%

Data Sources by Influence on Health Outcomes

Page 8: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

8

Cognitive Overload and Clinician Burnout

Sources: Millett D, Exclusive: “Three GPs a day seeking help for burnout, official data show,” GP, Jan 2018; Health Care IT Advisor research and analysis.

1) Obermeyer Z, Lee T, “Lost in Thought – The Limits of the Human Mind and the Future of Medicine,” N Engl J Med; 377, no. 13; (2017):1209-1211.

Tim

e R

equi

red

“Medical thinking has become vastly more complex, mirroring changes in our patients, our health care system, and medical science. The complexity of medicine now exceeds the capacity of the human mind.1”

Ziad Obermeyer, M.D. and Thomas Lee, M.D.

Number of Considerations for Decision Making

Significant opportunity for AI and analytics

Augmented Intelligence Reduces Cognitive Burden

Reliance on heuristics, rules of thumb, “good-enough”

Page 9: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

9

Data Power the Decision Machine

Source: Health Care IT Advisor research and analysis.

Four Levels of Analytic Capabilities

Data

Inform

Recommend

Decision Automation

Decision Support

DescriptiveWhat happened?

How many of our patients were readmitted last month?

What is likely to happen?Is this patient likely to readmit? What do we expect seasonally?

PrescriptiveRecommend or decide the best action.

What should we doto address this patient’s risks?

PredictiveDiagnosticWhy did it happen?

Rates spiked; was there a change in case mix? Demographics?

Decide

Hum

an

Dec

isio

n

Ana

lytic

s

Page 10: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

ROAD MAP10

The Data Dilemma1

2 AI and the New Machine Age

Page 11: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

11

AI: Are We There Yet?And if Not, When?

Source: Health Care IT Advisor research and analysis.

We May Be at the Start of an Explosive Growth in AI Capabilities

Incremental Growth:AI continues to make small advances, delivering value in niche applications.

AI Winters: AI has already gone through past phases of hype and troughs of disillusionment.

Exponential Growth: AI rapidly gains capability and becomes a mainstream part of most digital systems.

Unpredictable Timing: Some advances never seem to arrive (conversational systems), while others take off unexpectedly (smartphones).

60s 70s 80s 90s 2000 2010 2020 203050s 2040 2050

?

Page 12: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

12

Fuel for the AI Fire

Source: Health Care IT Advisor research and analysis.

1) EMR = Electronic medical record. 2) GPU = Graphics processing unit.3) TPU = Tensor processing unit.

AlgorithmsData• Internet Everywhere

• Broad EMR1 Adoption

• Wearables

• Real-Time Location

• IoT Sensors

• Genomics, Proteomics, Microbiome

• Automated Feature Extraction

• Cluster Compute Platforms (e.g., Spark, Hadoop)

• Deep Learning

• Off-the-Shelf Libraries for Common Tasks• Faster Processors

• Specialised Processors (GPUs,2 TPUs3)

• Inexpensive Storage

• Elastic Cloud Computing

Hardware

Combinatorial Advances DeliveringRadically Better Models

Page 13: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

13

The AI Winter Is Thawing

Source: Health Care IT Advisor research and analysis.

1) NASA = National Aeronautics and Space Administration.

Microsoft and Alibaba outscore humans in a Stanford reading comprehension test 2018

Google DeepMind’s AlphaZero convincingly leads at Chess, Shogi, Go 2017

Skype launches real-time voice translation 2014

Mass market voice assistants (Apple Siri, Google Now, Microsoft Cortana) 2012

IBM Watson conclusively beats Jeopardy game show champions 2011

NASA’s1 Mars Rovers operate semi-autonomously 2004

IBM’s Deep Blue beats world chess champion Gary Kasparov 1997

1950 Alan Turing proposes the “Turing Test” for conversational systems

1956 The term “Artificial Intelligence” is coined

1970s First AI Winter

1981 Digital Equipment Corporation order expert saves company $40M per year

Timeline of AI Successes

From Toys to Tools

Page 14: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

14

DeepMind’s AlphaZero

Novice to Superhuman in Just Four HoursDeepMind’s AlphaZero Uses ‘Zero Human Knowledge’

Sources: Vincent J, “DeepMind’s AI became a superhuman chess player in a few hours, just for fun,” The Verge, December 6, 2017; Gershgorn D, “DeepMind wants to find the next miracle material—experts just don’t know they’ll pull it off,” Quartz, October 25, 2017; Health Care IT Advisor research and analysis.

By not using this human data, by not using human features or human expertise in any fashion, we’ve actually rem

oved the constraints of human knowledge. It’s able to therefore create knowledge for itself.”

• AI product of Google subsidiary, DeepMind

• Provided no prior knowledge of the game; given only basic game rules and an objective (win)

• Learns through self-play at an accelerated pace (‘reinforced learning’); searches 80 thousand positions per second

• Variants of AlphaZero also defeated world champion Go and Shogi systems

AlphaZero outperformed chess world-champion, Stockfish,1 in just 4 hours (300k steps)

By not using this human data, by not using human features or human expertise in any fashion, we’ve actually removed the constraints of human knowledge. It’s able to therefore create knowledge for itself.”

David Silver, Lead ProgrammerDEEPMIND

1) Stockfish is the 2016 Top Chess Engine Championship (TCEC) world-champion.

AlphaZero Chess Performance

Page 15: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

15

Need a Radiologist?

There’s an App for That

Sources: Kubota T, “Stanford algorithm can diagnose pneumonia better than radiologists,” Stanford News, Nov 2017; Health Care IT Advisor research and analysis.

1) Stanford University Medical Center.2) NIH = National Institutes of Health.

Case in Brief: Stanford University1

CheXNet Algorithm

• Algorithm developed by researchers at Stanford University in the US can diagnose 14 common pathologies in chest x-rays

• Trained on ChestX-ray14, a public data set released by the NIH2 containing 112,120 frontal-view chest x-ray images labelled with the 14 possible pathologies

• Outperforms previous models from the same data set for all 14 conditions and diagnoses pneumonia at an accuracy exceeding the performance of four control radiologists

• Produces heatmaps that visualize the areas of an image most indicative of disease

Development of CheXNet

Sept. 26, 2017

ChestX-ray14 data set released along with a preliminary algorithm that could detect the labelled conditions

≈ One week later

CheXNet could diagnose 10 of the 14 pathologies more accurately than all previous algorithms

≈ One month later

CheXNet surpassed best published results for all 14 pathologies and outperformed Stanford radiologists in detecting pneumonia“I treat the model as a lazy resident. They’re not dumb, they’ll

just use every trick they can find to avoid doing hard work.”

Dr. Matthew Lungren, RadiologistStanford University Medical Center

Page 16: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

16

Lessons from Stanford’s CheXNet

AI Is Effective on Narrow Tasks

Recent advances in AI have focused on solving narrowly defined tasks. Eachsub-problem (e.g., pneumonia) is a model.

Handle “Normal” Explicitly

Identifying normal conditions versus anomalies can help recognize when human review may be needed.

Explainable1 AI (XAI)

Models that can be interpreted, at least partially, are easier to trust, deploy, and govern. Explainability is an area of rapid progress in AI research.

1) Also known as interpretable AI.

Training Requires a Teacher or Referee

Machine learning needs to be told what answers are correct or what actions produce positive outcomes. Well-curated data are critical to the learning process.

Source: Health Care IT Advisor research and analysis.

Page 17: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

17

The AI Value PropositionAI Excels at Decisions That Are Dull, Detailed, and Dear (Expensive)

Three Key Areas of Benefit

Dear:Provide speed, capacity, 24x7 availability, and consistency at a fraction of the cost

Although these are well known benefits of automation, when you apply this to cognitive and complex tasks, it’s a new ball game.

Detailed:Evaluate a vastly broader and deeper set of data to improve decision quality

The complexity of health care decision making is growing due to new data from genetics, IoT devices, wearables, better interoperability, and the growing number of drugs and treatments.

Unlimited capacity, instant response, disruptive economics

Perform at super-human levels, exceeding experts

Serves as an assistant(watches your back)

Dull:Free human decision makers to focus on more challenging “top of license” activities

Reduce effort spent on repetitive tasks, process large amounts of complex information, provide continuous monitoring and notifications, and ensure “stuff” doesn’t “fall through the cracks.”

Source: Health Care IT Advisor research and analysis.

Page 18: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

18

Where to Start, Where We’re Going

1) Narrow (Task, Weak) AI focuses on narrowly defined problems, such as predicting a patient’s risk of developing sepsis.2) General (Strong) AI attempts to match the flexibility and versatility of human intelligence.

Nar

row

(Tas

k, W

eak)

AI1

Clinical Decisions

Administrative Decisions

Gen

eral

(Stro

ng) A

I2

Full Diagnosisand Treatment Plan

Clinical Chatbots

RadiologyInterpretation

Acute ClinicalRisk

Recruiting

Safety Risk SmartMonitoring

Medication Dosing

AdministrativeChatbots (Concierge)

Precision Engagement

Precision Medicine

Ambient Documentation

Adaptive UserInterfaces

WorklistPrioritization

Inevitable with Time

Today’s Opportunity (growing)

The Computer Will See You Now

The Uber-ization ofHealth Care

An AI Application Landscape

Capacity / StaffingManagement

Population Health Risk

The Self-DrivingHospitalInformation

Security

Targeted Marketing

Source: Health Care IT Advisor research and analysis.

Page 19: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

19

The Self-Driving HospitalCombine Predictive Models with Front-Line Activation

Sources: “Virtual Air Traffic Control for Emergency Departments,” Qventus, https://qventus.com/content/uploads/2017/03/QVE-ED-Solution-Brief.pdf; Health Care IT Advisor research and analysis.

1) EVS = Environmental services.2) LWBS = Left without being seen.3) LOS = Length of stay.

Case in Brief: Mercy Hospital Fort Smith

• 336-bed acute care hospital in Fort Smith, AR

• Partnered with Qventus to improve emergency department (ED) patient flow and patient satisfaction

• Analytics applies ML algorithms to anticipate potential capacity shortfalls, process bottlenecks

• Application sends real-time notifications (nudges) to care teams via their mobile phones to trigger specific interventions

• Emergency department results:– LWBS2 rates dropped 30%

– Door-to-doc time reduced by 20%

– 24-minute reduction in average LOS3

– Annual case capacity increased by 6%

See that a radiology order is delayed for a patient within a few days of discharge and notify radiology to prioritize the reading

Activate more EVS1 staff when high churn of beds is anticipated

Send an early warning for difficult patient placements at discharge

Prioritize inpatient transports depending on expected bed needs

Sample “Decision Recipes”

Page 20: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

20

Applicants Like You Liked This Job

Predictive Models Reduce Turnover, Improve Engagement

Case in Brief: MultiCare Health System

• Non-profit system based in Tacoma, WA with 16,000 employees

• Partnered with Arena, a cloud-based HR solution, to predict the likelihood that a candidate will be retained in a role

• Arena’s machine learning algorithms evaluate applicant submissions, digital interactions, and externally sourced data on the organization (e.g., employer review sites)

• Achieved a 40% reduction in RN turnover at 180 days and a 28% reduction in overall turnover at 180 days

MultiCare’s Hiring Process

Reduced recruiting expenses

More predictable staffing levels

More experienced staff

Realized Benefits

Algorithms predict the likelihood the applicant will be retained and engaged in the target role and send that information to the recruiter

Applicant completes a 15-20 minute assessment process on Arena’s platform

Recruiters determine whether or not to send the candidate along to hiring managers

Hiring managers use Arena’s recommendation as an optional factor in the hiring decision

Source: Health Care IT Advisor research and analysis.

Page 21: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

21

Advanced Readmission Prediction at UNC1

“Simpler Isn’t Always Better, Sometimes Better Is Better.”

Case in Brief: UNC Hospitals• 853-bed academic medical center based Chapel Hill, NC• 77,000 annual ED visits and 30,000 annual surgeries• Post-discharge care coordination driven by a LACE-like2 readmissions model; UNC’s

Center for Innovation sought more powerful alternatives.• Partnered with Forecast Health to develop the Modern SDH3 model, resulting in

significant improvements in readmission prediction• 6 weeks from initial data access to proposed model

0%

20%

40%

60%

80%

BOOST LACE PARR Modern SDH Model

Readmissions Successfully Predicted4

5 6

1) UNC = The University of North Carolina.2) LACE = Length of stay, acuity, co-morbidities, and

emergency department visits.3) SDH = Social determinants of health.

Source: Advisory Board interviews and analysis.

4) Percentage of readmissions successfully predicted when the top 20% of predicted risk scores are selected. Metrics are representative of model performance, not specific to UNC.

5) BOOST = Better outcomes by optimizing safe transitions.6) PARR = Patient s at risk of readmission.

Page 22: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

22

Driving the Decision Machine

1Acquire Data

• Ensure all models have

a sponsor and regular

evaluation against goals

• All models require

tune-ups as the

environment changes.

Monitor Performance and Revisit

42Train or Refine

the Model

• The more the better,

but…

• High-quality, well-

governed data pay

enormous dividends

• Diverse, cross-continuum

sources improve

predictions and

robustness of models

• Decide what features

are important (e.g.,

explainability)

• Apply practical and

ethical constraints

• Evaluate under

realistic conditions

(silent mode, real-time,

incomplete data)

• Embed insights directly

into applications at key

decision points

• Reengineer processes

as warranted

• Consider the “5 rights”

of decision support

3Incorporate Models

into Workflow

Hint: The Analytics Is the Easy Part

Begin with:• Explicit leadership agreement on outcome goals, expected benefit mechanisms

• Participation from empowered representatives of impacted process

• External experience—learn from others

Source: Health Care IT Advisor research and analysis.

Page 23: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

23

Implement a single, coherent management platform across all enterprise data

Strengths: Highly flexible, consistent data management, best positioning for advanced analytics

Weaknesses: Longer time-to-value, larger staff investment

Application AnalyticsUse analytic capabilities bundled with primary systems of record

Strengths: Strong, out-of-the-box data integration and ability to embed analytics into workflows

Weaknesses: Difficulty integrating data from adjacent domains or competitors’ systems in mixed environments

Point SolutionsAcquire solutions targeting specific problem domains (e.g., population health, cost accounting, clinical quality)

Strengths: Deep domain knowledge, rapid time to value

Weaknesses: Narrow focus on individual domains, limited ability to stretch into adjacent areas for more advanced analysis

Choose Your BI Architecture DeliberatelyPick a Core Approach, But Mix in Others as Appropriate

Acquire Data

Enterprise Data Platform

On Premise or in the Cloud?Analytics workloads are especially suited to cloud deployment. Elastic compute resources can “surge” to handle difficult tasks, then rapidly scale down when demand falls.

Source: Health Care IT Advisor research and analysis.

Page 24: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

24

Evolution of the Data WarehouseData Lakes Deliver, But Avoid the Data Swamp

Acquire Data

1) PACS = Picture archiving and communication system.

EMRs

Claims

Laboratory

PACS1

Finance

Transactional Systems Data Lake Data

WarehouseCore Reports

Data Marts

ExplorationDiagnosticsHypothesis Generation

AcquisitionDiscoveryTraceability

Data Capture GovernanceReconciliationQualityEnrichment

Ad-hoc Analytic Tools

DeliveryMeasurementAccountability

Systems

Activities

Promotion

Enrichment

Source: Health Care IT Advisor research and analysis.

Page 25: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

25

How Machines LearnSimilar Methods, Differing Goals

Build the Model

Source: Advisory Board research and analysis.

Input Data• The more the better• Labeled with key indicators

Adjust the Model• Automated trial and error• Cross-breeding successful

candidates

Repeat

Supervised Learning

Evaluate models against the “correct” answersprovided with the data. The learning process focuses on minimizing errors.

Unsupervised Learning

Evaluate models against an abstract algorithmic goal. Example: Divide this population of diabetics into a handful of subpopulations that have similar future risk profiles.

Reinforcement Learning

Evaluate models by giving a reward for desirable outcomes and a penalty for negative outcomes. Models are trained to maximize their total rewards over time.

Ready for Review

Evaluate Performance• Against what goal?• Forces explicit

tradeoffs (e.g., costvs. quality)

Page 26: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

26

Workflow Is Key to Adoption

Incorporate into Workflow

The Right Information

To the Right Person

In the Right Form

By the Right Channel

At the Right Time

Case in Brief: Mayo Breast OncologyClinic Trials Recruitment

• Up to 100 concurrent drug therapy clinical trials are available to Mayo Clinic breast cancer patients, making it challenging to consistently identify candidates.

• New clinical trial matching technology developed in collaboration with IBM Watson Health evaluates charts to identify eligible patients.

• Launched in January 2016 as a point-of-care solution for oncology providers; initial adoption was low due to time pressures and fell to zero by February.

• Rework by health systems engineers resulted in a July 2016 relaunch; screening moved to occur in advance of patient visits so clinicians know of trial matches before opening a patient’s chart.

• New process enrolls significantly more patients in clinical trials than the prior manual process; success led to additional coordinator funding.

The Five Rights of Clinical Decision Support

Source: Health Care IT Advisor research and analysis.

Page 27: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

27

Who Watches the Machines?Treat Predictive and Prescriptive Models Like Junior Staff

Monitor Performance and Revisit

Governance is focused on quantifiable target

outcomes

Automated processes need human oversight

“Circuit breaker” rules can limit potential harm

Periodically evaluate processes

against goals

If you aren’t listening to what your data tells you about the present, you’re not ready to use it to predict the future.

Source: Health Care IT Advisor research and analysis.

Page 28: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

28

Advisors to Our Work

With Sincere AppreciationThe Health Care IT Advisor program would like to express deep gratitude to

the individuals and organizations that shared their insights, analysis, and time

with us. The research team would especially like to recognize the following

contributors for being particularly generous with their time and expertise.

Arena.IOBaltimore, Maryland

Michael FinnMichael Rosenbaum

AthenahealthWatertown, Massachusetts

Girish Venkatachaliah

AyasdiPalo Alto, California

Jonathan Symonds

Children’s Hospital of Eastern OntarioOttawa, Ontario, Canada

Christina Honeywell

CollibraNew York, New York

Chris CooperDan Scholler

Dignity Health San Francisco, California

Dr. Gurmeet Sran

Duke University Health SystemDurham, North Carolina

Dr. Mark Sendak

Forecast Health Durham, North Carolina

Michael Cousins

Geisinger Health SystemDanville, Pennsylvania

Kirk Hanson

Health Catalyst Salt Lake City, Utah

Dale SandersLevi Thatcher

HealthDataVizWaltham, Massachusetts

Sandy Lawson Kathy Rowell

Intermountain Healthcare Salt Lake City, Utah

Dr. Nathan DeanJeffrey Ferraro Lonny Northrup

Page 29: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

29

Advisors to Our Work

With Sincere AppreciationThe Health Care IT Advisor program would like to express deep gratitude to the individuals and organizations that shared their insights, analysis, and time with us. The research team would especially like to recognize the following contributors for being particularly generous with their time and expertise.

Jewish General Hospital Montréal, Québec, Canada Dr. Lawrence RosenbergPhil Troy

Mayo ClinicRochester, MinnesotaDr. Tufia Haddad

Memorial Sloan Kettering Cancer CenterNew York, New YorkAri CarolineIsaac Wagner

Microsoft Seattle, WashingtonJohn DoyleTom Lawry Dennis Schmuland

New York PresbyterianNew York, New YorkDr. Peter FleischutDr. David Tsay

PaxataRedwood City, CaliforniaNenshad BardoliwallaJayanta Bhowmik

Pieces Technologies Dallas, TexasDr. Ruben AmarasinghamShannon Kmak

QventusLos Altos, CaliforniaMudit GargVenkat Mocherla

Royal Free London NHS Foundation TrustSurrey, London, UKGlenn Winteringham

SalesforceSan Francisco, CaliforniaJayesh Govindarajan

Stanford UniversityMedical CenterStanford, CaliforniaDr. Matthew LungrenDr. Nigam Shah

Taunton and Somerset NHS Foundation TrustSomerset, England, UKAndrew Forrest

Page 30: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

30

Advisors to Our Work

With Sincere AppreciationThe Health Care IT Advisor program would like to express deep gratitude to

the individuals and organizations that shared their insights, analysis, and time

with us. The research team would especially like to recognize the following

contributors for being particularly generous with their time and expertise.

The MetroHealth SystemCleveland, Ohio

Dr. Robert FerguesonDr. David Kaelber

The University of Chicago MedicineChicago, Illinois

Mike Wall

University Hospital BirminghamBirmingham, England, UK

Steven ChiltonPaul JenningsBarnaby Waters

University Hospitals of LeicesterLeicester, England, UK

Andrew Carruthers

University of CambridgeCambridge, England, UK

Pietro Lio

University of LeedsLeeds, England, UK

Owen Johnson

University of North Carolina Healthcare Chapel Hill, North Carolina

Jason Burke

University of Pittsburg Medical Center Pittsburgh, Pennsylvania

Terri MikolDon RiefnerDr. Rasu Shreshta

Washington University Medical CenterSt. Louis, Missouri

Dr. Brad Racette

Page 31: Informatics Research Seminars · 2016 2030 2016 2030 2016 2030 2016 2030 2016 2030 Australia2 9.6% 14.4% 2010 2030 New Zealand3 6.2% 8.9% 10.6% 15.9% 11.0% 18.0% Canada France United

©2018 Advisory Board • All Rights Reserved • advisory.com • 36516D

LEGAL CAVEAT

Advisory Board is a division of The Advisory Board Company. Advisory Board has made efforts to verify the accuracy of the information it provides to members. This report relies on data obtained from many sources, however, and Advisory Board cannot guarantee the accuracy of the information provided or any analysis based thereon. In addition, Advisory Board is not in the business of giving legal, medical, accounting, or other professional advice, and its reports should not be construedas professional advice. In particular, members should not rely on any legal commentary in this report as a basis for action, or assume that any tactics described herein would be permitted by applicable law or appropriate for a given member’s situation. Members are advised to consult with appropriate professionals concerning legal, medical, tax, or accounting issues, before implementing any of these tactics. Neither Advisory Board nor its officers, directors, trustees, employees, and agents shall be liable for any claims, liabilities, or expenses relating to (a) any errors or omissions in this report, whether caused by Advisory Board or any of its employees or agents, or sources or other third parties, (b) any recommendation or graded ranking by Advisory Board, or (c) failure of memberand its employees and agents to abide by the terms set forth herein.

The Advisory Board Company and the “A” logo are registered trademarks of The Advisory Board Company in the United States and other countries. Members are not permitted to use these trademarks, or any other trademark, product name, service name, trade name, and logo of Advisory Board without prior written consent of Advisory Board. All other trademarks, product names, service names, trade names, and logos used within these pages are the property of their respective holders. Use of other company trademarks, product names, service names, trade names, and logos or images of the same does not necessarily constitute (a) an endorsement by such company of Advisory Board and its products and services, or (b) an endorsement of the company or its products or services by Advisory Board. Advisory Board is not affiliated with any such company.

IMPORTANT: Please read the following.

Advisory Board has prepared this report for the exclusive use of its members.Each member acknowledges and agrees that this report and the information contained herein (collectively, the “Report”) are confidential and proprietary to Advisory Board. By accepting delivery of this Report, each member agrees toabide by the terms as stated herein, including the following:

1. Advisory Board owns all right, title, and interest in and to this Report. Except as stated herein, no right, license, permission, or interest of any kind in this Report is intended to be given, transferred to, or acquired by a member.Each member is authorized to use this Report only to the extent expressly authorized herein.

2. Each member shall not sell, license, republish, or post online or otherwise this Report, in part or in whole. Each member shall not disseminate or permit the use of, and shall take reasonable precautions to prevent such dissemination or use of, this Report by (a) any of its employees and agents (except as stated below), or (b) any third party.

3. Each member may make this Report available solely to those of its employees and agents who (a) are registered for the workshop or membership program of which this Report is a part, (b) require access to this Report in order to learn from the information described herein, and (c) agree not to disclose this Report to other employees or agents or any third party. Each member shall use, and shall ensure that its employees and agents use, this Report for its internal use only. Each member may make a limited number of copies, solely as adequate for use by its employees and agents in accordance with the terms herein.

4. Each member shall not remove from this Report any confidential markings, copyright notices, and/or other similar indicia herein.

5. Each member is responsible for any breach of its obligations as stated herein by any of its employees or agents.

6. If a member is unwilling to abide by any of the foregoing obligations, thensuch member shall promptly return this Report and all copies thereof to Advisory Board.

Health Care IT Advisor

Research DirectorGreg [email protected]

Research TeamJacqueline Beltejar

Bethany JonesNouran Ragaban

Sophie Ranen

Andrew Rebhan

Program LeadershipJim Adams

Design ConsultantKevin Matovich