Implementing a new Machine Learning Algorithm to Predict

CRC- MHS Roll Out of MeScore

Ran Goshen, MD, Ph.D., Chief Medical Officer,

Medial EarlySign Ltd.

Possible Conflict of Interests

Ran Goshen is the Chief Medical Officer of Medial EarlySign Ltd (MES). MES is the developer of MeScore used by Maccabee

Health Services (MHS). MHS pilot implementation of MeScore will be described.

Ran Goshen

Our Goal for Today

• Brief background on MeScore, a machine learning based

algorithm, designed to flag individuals at risk of harboring

colorectal cancer

• Describe the implementation of MeScore within a large

HMO

• Discuss preliminary results

• Learn from you on your thoughts and plan ahead

MK

G-1

4-0

1-0

1-A

ColonFlag

• Based on:

Standard CBC values + age + gender

• Identifies trends and variations within

the normal ranges as well as exceptions

from the norm

• Provides a simple and easy to use

personal risk indication

MeScore No Flag Red Flag

Expedited Assessment

MeScore- A computed relative risk indicator

Follow Guidelines

1.2

1

0.8

0.6

0.4

0.2

0

-0.2

-1.5 -1 -0.5 0 0.5 1 -1.5 -1

1.2

1

0.8

0.6

0.4

0.2

0

-0.2

-0.5 0 0.5 1

Machine Learning vs. Regression Modeling

Machine Learning Regression Modeling

Clinical Rigor

* Maccabi Healthcare Services: IHCO covering 25% of Israel’s population (~2M individuals) ** The Health Improvement Network

Blindly validate the model (Validation set) 139,205 people

(698 CRC patients)

Blindly validate on independent data-set 25,613 people

(5,061 CRC patients)

Train & develop model (Derivation set) 466,107 people

(2,437 CRC patients)

THIN** (UK)

MHS* (Israel)

Create model

Check model performance on “blind” part of the same

population

Check model performance on different population

1

2

3

Worldwide Validation 4 Worldwide

Ongoing trials in leading institutions around the world

totaling nearly 5M individuals

2nd largest Integrated Healthcare Provider, serving

25% of the Israeli population (over 2 M

covered lives)

Fully integrated healthcare, 100%

penetration of EMR for 3 decades, single

centralized MegaLab

1615 GPs, 110 gastroenterologists, 9 GI

centers, performing 63,000 colonoscopies

annually

Maccabi Healthcare Services - Overview

MeScore Implementation at MHS

• Perform case finding on individuals not up to date with their CRC

screening.

• Extract the outpatient CBC data and demographics of non-

responders to CRC screening, who performed their last outpatient’s

blood count during the last working day.

• Run MeScore on the current (and historical, when available) CBC

test results data of the selected individuals.

• Flag in the GP EMR system individuals above a pre-defined

threshold.

• GPs are trained to refer their MeScore flagged patients as they do

with their FIT positive patients directly to colonoscopy

MK

G-1

4-0

1-0

1-A

MHS Inclusion Criteria

• 50-75 age range

• Outpatient CBC test was performed during the last day

• Has not been treated for cancer during the last 12 months

• Did not perform a FIT/gFOBT test in past 18 months

• Did not perform a colonoscopy in past 10 years

• Is not currently scheduled to perform CRC screening test

• Did not visit a gastroenterologist in past 3 months and is not

scheduled to see gastroenterologist during the coming 3 months

MK

G-1

4-0

1-0

1-A

Blood Count Ordered

Blood Count Performed and

Processed

Age 50-75?

Screening Up-To Date?

Under GI Examination

?

Lab Results

Report Result to GP and Smart

Alerts

Above Cut-Off ?

Calculate ColonFlag

New Results Blood Count

Referral

LAB SYSTEMS

EHR SMART ALERTS

Yes

No No

CS+

MHS Implementation Flow

Create Smart Alert and Reminders

Yes

ColonFlag+

MeScore Implementation Data Flow

CBC

Database

Extract CBC data for patients that

meet criteria

Transfer data to MeScore Server located

on customer site

MeScore software calculates MeScore

Transfer MeScore results back to main database

Alert GP

Population Management

Transfer flagged patients results

per cutoff to GPs

MK

G-1

4-0

1-0

1-A

A Sample Report for GP

MK

G-1

4-0

1-0

1-A

MHS implementation numbers

534 (~1%)High scores (≥ 99.6)

109* under GP supervision (21%)

56 have not visited GP yet (10%)

369 referred by GP (69%)

* Will not be referred to cs.

79 referred to CS. (21%)

290 * referred to gastro (78%)

106 have not visited the gastro yet (36%)

170* referred to CS. (59%)

14 under gastro supervision (5%)

* If there is referral to CS. or CS. was performed without referral from GP, our assumption is that the patient was referred to/by gastroenterologist.

162 performed CS. (65% compliance)

Total ‘high scores’ in MHS Mid 10/2015-08/2016 (10.5 months)

249 referred to CS.

• 47% of “high scores” are referred to colonoscopy • 30% of high scored patients performed colonoscopy (compliance to CS. is 65%)

~ 60,000 scores

Colonoscopy findings – Castells’ reference table

Reference: Postpolypectomy surveillance in patients with adenomas and serrated lesions: a proposal for risk stratification in the context of organized colorectal cancer-screening programs. Castells A, Andreu M, Binefa G, Fité A, Font R, Espinàs JA. Endoscopy. 2015 Jan;47(1):86-7.

(19) (5)

(11)

(17)

(14)

(90)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1

Portion of colonoscopy findings classification

Cancer High Risk

Intermidiate risk Low risk

No risk Clean colonoscopies

Colonoscopy findings

• Total number of colonoscopies: 162 • Unknown results: 6 • Total colonoscopies with known results: 156

• Total colonoscopies with findings: 66 • Clean colonoscopies: 90

(66)

(90)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1

Portion of colonoscopy results

Colonoscopies with findings Clean colonocopies

Medial EarlySign at a Glance

2009 2011

2010 2012

2016 2014

2015

2013

Development of first models Collection of Data

Initial model for CRC

2nd generation CRC Validation in UK

>1M Patient Records Initial model for Lung Cancer Models for real-time ICU

Predictor Engines Discovery workbench

Initial Models for Diabetes, Renal Failure Installation in MHS Peer-reviewed publication

EU Installations U.S., Asia validations GI Modelling >17M records

Company founding

for ColonFlag

THANK YOU