Joint Classification and Localization of Critical Findings in Chest …€¦ · Epoch 1 Epoch 10...
Transcript of Joint Classification and Localization of Critical Findings in Chest …€¦ · Epoch 1 Epoch 10...
#CMIMI18#CMIMI18
Joint Classification and Localization of Critical Findings in Chest X-ray using Deep Multi-Instance Transfer Learning
Evan Schwab, PhDPhilips Research North America
#CMIMI183 CISS
1. CXR lack annotations (e.g. local bounding boxes).
2. Unlike pictures of dogs, difficult to validate correctness without experts.
3. Deep learning networks commonly non-interpretable
Objectives Constraints
1. Automatically classify CXR (No Finding vs Pneumothorax).
2. Automatically localize abnormality in CXR (if present).
3. Classify and localize jointly. (Want to end-to-end solution).
Medical images lack local annotations
#CMIMI184
Wang, Xiaosong, et al. "Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases." CVPR, 2017.
State-of-the-Art: Grad-CAM CXR Saliency Maps
#CMIMI186
1024 x 1024 224 x 224
Pre-Trained Network (VGG16)
224 x 224
7x7
** Images Drawn to Scale
DownSample
FeedInto
Grad-Cam Heatmap
UpSample
#CMIMI187
1024 x 1024 224 x 224
Pre-Trained Network (VGG16)
224 x 224
7x7
** Images Drawn to Scale
Extract FeedInto
UpSample
Grad-Cam Heatmap
#CMIMI189
Which patch contains Pneumothorax?
Multi-Instance Learning (MIL)
No Finding Pneumothorax Pneumothorax
#CMIMI1810
1024 x 1024
N x 224 x 224N Patches
Pre-Trained Network (VGG16)
.001 .92 .24
Sigmoid Patch Scores
Max ScoreOutput:
1. Predicted Bag Label: Pneumothorax with 92%2. Max Patch Location
Proposed Deep MIL with Transfer Learning
#CMIMI1811
Observed (+) Label
(+) (+) (+)
Observed (-) Label
(-) (-) (-)N Patches form 1 CXR fed as 1 training batch
N Patches form 1 CXR fed as 1 training batch
Sigmoid Layer Nx1
Max Layer 1x1
Nx224x224
Training Deep MIL with Transfer Learning
Pre-Trained Network (VGG16)
#CMIMI18
Data: – NIH CXR dataset– Subset of data has ground truth bounding box of Pneumothorax (PTX)– Binary Classification: PTX vs No Finding– 60 Subjects: 30 Subjects PTX, 30 Subjects No Finding– Ground truth bounding boxes only used for visual verification at the end– Divide data classes evenly into 2/3 Training 1/3 Validation
Setup:– Accuracy/AUC given by correct image classification– Stochastic Gradient Descent, Decay, Momentum– Transfer Learning: Freeze first 15 layers of VGG16– Add Sigmoid Patch Score Layer– Add Final Layer:
• Max
• Max Sum over Neighborhood
• Log Sum Exponential (LSE), (parameter r)
12
Preliminary Results
Final Layer AUC
Max 0.67
Max Sum 0.75
LSE (r=1) 0.89
LSE (r=2) 0.89
LSE (r=5) 0.94
LSE (r=10) 0.67