Digital & Com - Beck Belfast 06.19 - Path · PDF fileConvolutional Neural Network P(tumor) ......
7
21/07/2017 1 Deep Learning for pathology Andrew H Beck MD PhD CEO - PathAI Belfast Pathology June 19, 2017 Disclosures "I am the CEO and an equity holder in PathAI, Inc. Pathology is critical for precision cancer medicine Diagnostics Therapeutics Errors are common in pathology Elmore et al. (JAMA 2015) report an overall concordance rate of 75% on breast biopsies and a concordance rate of only 48% for the diagnosis of Atypia. Cases below were classified by an expert consensus panel as Denise Grady. Breast Biopsies Leave Room for Doubt, Study Finds. New York Times (3/17/2015) Patient symptoms Clinical signs Radiology impression Pathology diagnosis No Treatment Minimal Treatment Aggressive Treatment Pathologic diagnosis affects all subsequent medical decisions Errors in pathology have significant downstream effects Pro-K 25 mM EDTA HO 1 h, RT Gelation Expansion Pathology to Generate Massive Morpho-Molecular Data from Tiny Specimens Joint work with Ed Boyden, Yongxin Zhao, Octavian Bucur (2017, in press at Nature Biotechnology)
Transcript of Digital & Com - Beck Belfast 06.19 - Path · PDF fileConvolutional Neural Network P(tumor) ......
21/07/2017
1
Deep Learning for pathology
Andrew H Beck MD PhDCEO - PathAI
Belfast PathologyJune 19, 2017
Disclosures
• I am the CEO and an equity holder in PathAI, Inc.
Pathology is critical for precision cancer medicine
Diagnostics Therapeutics
Errors are common in pathologyElmore et al. (JAMA 2015) report an overall concordance rate of 75% on breast biopsies and a concordance rate of only 48% for the diagnosis of Atypia. Cases below were classified by an expert consensus panel as
Denise Grady. Breast Biopsies Leave Room for Doubt, Study Finds. New York Times (3/17/2015)
Patient symptoms
Clinical signs
Radiology impression
Pathology diagnosis
No Treatment
MinimalTreatment
AggressiveTreatment
Pathologic diagnosis affects all subsequent medical decisions
Errors in pathology have significant downstream effects
Pro-K25 mM EDTA H2O
1 h, RT
Gelation
Expansion Pathology to Generate Massive
Morpho-Molecular Data from Tiny Specimens
Joint work with Ed Boyden, Yongxin Zhao, Octavian Bucur (2017, in press at Nature Biotechnology)
21/07/2017
2
Expansion Microscopy (ExM)
Applications in cancer
pathology?
Expansion Microscopy (ExM)
Chen, Tillberg, Boyden (2015) Science 347(6221):543-548. http://expansionmicroscopy.org
Pre-expansion
Nuclei-Epithelium-Stroma-Stroma
FibroblastsBreastcancercells
Post-expansionScale bar: 50 µm
FibroblastsBreastcancercells
Expansion pathology of breast cancerExpansion pathology performed on a pre-invasive breast lesion
Blue –DapiGreen – Vimentin Red – anti Hsp60
Pre-expansion Blue – nuclei with DapiGreen – anti-Vimentin Ab (Stroma)Red – anti Hsp60 (Mitochondria)
Post-expansion
21/07/2017
3
3D Expansion Pathology –Nuclear Detection and Segmentation
Lightsheet MicroscopyForeground and nuclear
seed-point detection3D Nuclear Segmentation
3D Nuclear Classification
Expansion pathology enables visualization of renal podocytes
Pre-expansion Post-expansion
Red = Collagen IVBlue = Vimentin (Primary and Secondary foot processes)Green = Tertiary foot processes
Joint work with Astrid Weins MD PhD
Does expansion improve computational pathology classifiers?
Pre-expansion
Post-expansion
Image Processing
Pre-expansion classification models
Post-expansion classification models
Pre- vs. Post-Classification Performance
Expansion Pathology Produces More Accurate Classification Models
Pre-Exp Exp-Path
Normal vs Usual Ductal Hyperplasia 0.89 0.94
Normal vs Atypical Ductal Hyperplasia 0.89 1
Normal vs Ductal Carcinoma in Situ 0.74 0.81
Usual Ductal Hyperplasia vs Atypical Ductal Hyperplasia 0.75 0.94
Usual Ductal Hyperplasia vs Ductal Carcinoma in Situ 0.71 0.75
Atypical Ductal Hyperplasia vs Ductal Carcinoma in Situ 0.75 0.86
Area under the Receiver Operator Curve in Cross-Validation of L1-Regularized Logistic Regression Classifier
Expansion Pathology with DNA-FISH and Protein-IF
Blue = HER2 ProteinRed = HER2 AmpliconGreen = Centromeric probe
Negative for HER2 Amplification HER2 Amplified
21/07/2017
4
Expansion Pathology• New approach for physically expanding pathology
specimens
• Very high resolution analysis of morphology
• Multiplexed in situ molecular assays with very little autofluorescence
• Generates extremely large and complex morpho-molecular pathology data from tiny biopsy specimens, requiring AI-based approaches
Zhao, Bucur,…, Beck, Boyden. In Press at Nature Biotechnology (2017)
Deep learning is driving major advances in computer vision
0
5
10
15
20
25
30
2010 2011 2012 2013 2014 2015
erro
r ra
te (%
)
ImageNet performance over time
`
0
5
10
15
20
25
30
2010 2011 2012 2013 2014 2015
erro
r ra
te (%
)
ImageNet performance over time
Human error rate
Deep learning is driving major advances in computer vision
Deep learning is driving major advances in wide-ranging fields
Google DeepMindDefeats GO Champion Lee Sedol
(March, 2016)
Uber deploys autonomous driving taxis on the streets of Pittsburgh (September 2016)
Deep Learning for Pathology:Cancer Metastasis Detection
Dayong Wang PhD
Harvard Medical School
Aditya Khosla
PhDMIT
ISBI Grand Challenge on Cancer Metastases Detection in Lymph Node
21/07/2017
5
Camelyon16 (>200 registrants) H&E Image Processing Framework
Train
whole slide image
sample
sample
training data
norm
altu
mor
Test
whole slide imageoverlapping image
patches tumor prob. map
1.0
0.0
0.5
Convolutional Neural Network
P(tumor)
H&E Image Processing Results
Original image Tumor prediction
H&E Image Processing Results
Original image Tumor prediction
H&E Image Processing Results
Original image Tumor prediction
Pathologist Radboud EXB METU NLP LOGIXHMS & MIT
21/07/2017
6
Deep Learning vs Pathologist
7.5%
3.5%
0.5%
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Category 1Our model
Deep Learning vs Pathologist
7.5%
3.5%
0.5%
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Category 1Our model Pathologist
Deep Learning vs Pathologist
The combination of a pathologist and the Beck Lab deep learning system reduces error rate by 85% to 0.5%.
7.5%
3.5%
0.5%
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Category 1Our model Pathologist Our model + Pathologist
www.camelyon16.grand-challenge.org/results/
Artificial Intelligence Gets an A+ for Accurately Diagnosing Breast Cancer
- Breast Cancer News (Jun 29, 2016)
Our Team Won the 2016 ISBI Grand Challenge for Metastatic Cancer Detection
performance to humans is way beyond what I had anticipated. It is a clear indication that artificial intelligence is going to shape the way we deal with histopathological images in the years to come.
- Jeroen van der Laak, Radbound University Medical Center
Featured in the report Preparing for the Future of Artificial IntelligenceExecutive Office of the President of the United States
Clinical study on ISBI dataset
Error Rate
Pathologist in competition setting 3.5%
Pathologists in clinical practice (n = 12) 13% - 26%
Pathologists on micro-metastasis (small tumors) 23% - 42%
Beck Lab Deep Learning Model 0.65%
Beck Lab
Before AfterPathology Report
Patient Name: John DoeDiagnosis: Met. CancerSize: 2.3 mmpTNM Staging: pT2N1MX# of Pos. LN: 1# of Neg LN: 4
Confirm
Pathology Report
Patient Name: John DoeDiagnosis: __________Size: _______________pTNM Staging: _______# of Pos. LN: ________# of Neg LN: ________
Deep Learning in the Clinical Workflow
- microscope field of view
• Labor Intensive• Error-prone• Poor standardization
• Fast• Accurate• Standardized
21/07/2017
7
Immunohistochemistry Image Processing Framework
Region of Interest Classification
whole slide image
P(Stroma)
P(Normal)
P(Tumor)
P(Necrosis)
Nucleus detectionDAB Quantification
1.0
0.0
0.5
Data and image export:
• ROI
• Single-cell
• Sub-cellular
• Error metrics
Convolutional Neural Network
sample
sample
sample
sample
Deep Learning for Computational
Pathology
DL-Driven Computational Pathology at the center of bio-medicine and healthcare
Advanced AI for next generation computational
pathology
Advanced AI for next generation computational
pathology
Emerging methods enable generation of massive data from pathology specimens
Emerging methods enable generation of massive data from pathology specimens
Mission of pathology is to provide optimal data-driven
diagnoses
Mission of pathology is to provide optimal data-driven
diagnoses
AI-Powered Pathology
Genetics
Statistics
Computer Science
Molecular and
Cellular Biology
Systems Biology
Clinical Medicine
