LDCT Lung Cancer Detection, Derek Onken, Computer Science...
1
PDE- BASED NEURAL NETWORKS FOR L UNG C ANCER D ETECTION USING 3-D LDCT I MAGES D EREK O NKEN PhD Candidate, Department of Computer Science, Emory University Data Science Intern, UnitedHealth Group R&D, Summer 2019 M OTIVATION Lung cancer: • is responsible for the most deaths of all cancers • begins localized in the lungs and spreads 5-year survival rates: – 19% (all stages) – 56% (still localized to the lungs) • can be caught and treated early, reducing mortality Incidence By Source [1] Symptomatic Incidental Screening 85% 13% 2% In 2018, • 541K people living with lung cancer (prevalence) • 234K new cases (incidence) Annual screening is recommended for the non- symptomatic high-risk population (smokers with >15 pack-years; age 55-74). Radiologists read these scans to determine cancer diagnosis. Actual True Actual False Total Predicted True 270 TP 6,911 FP 7,181 Predicted False 18 FN 19,043 TN 19,061 Total 288 25,954 26,242 Sensitivity TP TP + FN = 93.8% PPV TP TP + FP = 3.8% F 1 2TP 2TP + FN + FP = 7.2% Data from LDCT group baseline (T0) of Nat’l Lung Cancer Screening Trial (NLST) [2] Current process can be expensive for providers, and patients experience delays and high costs. Commercial Population [3] Of adult patients diagnosed with non-small cell lung cancer (2007-2011) • ~94% experienced a delay of 5-6 months • Mean per patient per month in total health care costs was $2,407 ± $3,364 False positives render many of these costs wasteful. ⇒ Reduction in false positives at initial screening saves patients and providers time and money. R EFERENCES [1] Disease Inception Accelerator. Patient Journey Research. 2018. [2] NLST Reseach Team. Results of Initial Low-Dose Computed Tomographic Screening for Lung Cancer. NEJM, 2013. [3] T Gildea et al. A Retrospective Analysis of Delays in Diagnosis. ClinicoEconomics and Outcomes Research: CEOR, 2017. [4] E Haber, L Ruthotto. Stable Architectures for Deep Neural Networks. Inverse Problems, 2017. [5] K He et al. Deep Residual Learning for Image Recognition. IEEE CVPR, 2016. [6] L Ruthotto, E Haber. Deep Neural Networks Motivated by Partial Differential Equations. JMIV, 2019. [7] F Liao et al. Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-or Network. IEEE TNNLS, 2019. [8] D Ardila et al. End-to-end Lung Cancer Screening with 3-D Deep Learning. Nature Med., 2019. P ROBLEM &A PPROACH Develop a model that classifies a 3-D low-dose computed tomography (LDCT) scan as cancerous or non- cancerous. Performance goal: fewer false positives than practicing radiologists while matching sensitivity. ! " Probability: that cube contains a cancerous nodule? no # 1 %&' ()*(&+,-' 0 *,/ ()*(&+,-' Model Max Probability Segment & pull out the nodules 300 slices, 512 x 512 each Neural Network ! " ! " ! " Input LDCT Output Inner workings of the model: • Segmenter pulls out smaller cubes with the most likely can- cerous nodules (for scalability) • PDE-based Classifier – Predict cancer likelihood for each cube – Use from the max likelihood cube for the patient O PTIMAL C ONTROL IN NN S Goal: Find a function f : R p × R n → R m and param- eters θ ∈ R p such that f (θ , y 0 ) ≈ c for every training input y 0 ∈ R n and its label c ∈ R m . We make this continuous, viewing weights θ and fea- tures y as functions of time t ∈ [0,T ]. The neural network f (θ , y 0 )= y(T ) performs a non- linear transformation of inputs y 0 satisfying the ordi- nary differential equation (ODE), ∂ t y(t)= ‘(θ (t), y(t)) , for t ∈ (0,T ] y(0) = y 0 , (1) where ‘ : R p × R n → R n is a neural network. A forward Euler discretization [4] of Eq. (1) is the N - layer ResNet y j +1 = y j + h‘ (θ j , y j ) , where j =0, 1,...,N -1, with step size h = T /N . Borrowing from optimal con- trol, the θ j are control layers and the y j are state layers. Training: Tune θ and linear layer W by solving min θ,W 1 s s X k=1 loss(f (θ , y k 0 )W, c k ) + regularizer(θ , W) across all s training inputs. D ECOUPLING W EIGHTS &L AYERS Goal: Improve training by maintaining few parame- ters but with many layers We develop a generalized ResNet for the classifier: Resnet-14 [5] Our Framework image 3x3 conv, 16, /2 pool, /2 3x3 conv, 16 3x3 conv, 16 3x3 conv, 16 3x3 conv, 16 3x3 conv, 32, /2 3x3 conv, 32 3x3 conv, 32 3x3 conv, 32 3x3 conv, 64, /2 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 avg pool fc 10 image 3x3 conv, 16, /2 pool, /2 3x3 conv, 16 3x3 conv, 16 3x3 conv, 16 3x3 conv, 16 3x3 conv, 32 pool, /2 3x3 conv, 32 3x3 conv, 32 3x3 conv, 32 3x3 conv, 32 3x3 conv, 64 pool, /2 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 avg pool fc 10 image Opening Layer 3 → 16 σ ◦N◦ K Dynamic Block, 16 RK1 scheme Double Layer t = [0, 2] h θ = h u =1 Connecting Unit 16 → 32 P ◦ σ ◦N◦ K Dynamic Block, 32 RK1 scheme Double Layer t = [0, 2] h θ = h u =1 Connecting Unit 32 → 64 P ◦ σ ◦N◦ K Dynamic Block, 64 RK1 scheme Double Layer t = [0, 2] h θ = h u =1 avg pool fc 10 Fixed-width portions of ResNet = a Dynamic Block Each block contains: • continuous ODE like Eq. (1) • ODE solver scheme (e.g., Runge-Kutta 4) • discretization for the solver • neural network layer ‘ For ‘, we experiment with: • Double Layer: σ 2 ◦N 2 ◦ K θ 2 ◦ σ 1 ◦N 1 ◦ K θ 1 (y) • Double Symmetric Layer [6]: -K > θ ◦ σ ◦N◦ K θ (y) for activation functions σ , normalizations N , and con- volution operators K defined by weights θ F UNDING Supported by the National Science Foun- dation awards DMS 1522599 and CAREER DMS 1751636, Binational Science Founda- tion Grant 2018209, NVIDIA Corporation, and UnitedHealth Group. C OLLABORATORS • Lars Ruthotto (Emory University) • Rachel Jennings (UnitedHealth Group) • Stephen Garth (UnitedHealth Group) • Simion Novikov (Ben Gurion, Israel) • Eran Treister (Ben Gurion, Israel) • Eldad Haber (UBC, Vancouver) R ESULTS • Use pre-trained segmenter [7] • Classifier uses Double Symmetric Layer • Minimize Focal Loss • Trained on a subset of 280 patient scans • Requires 1 week to train on a GPU with 16GB RAM Radiologists (NLST patients) Google AI [8] (NLST patients) Training Set (NLST patients) Validation Set (NLST patients) Actual True Actual False Actual True Actual False Actual True Actual False Actual True Actual False Predicted True 270 6,911 82 1,260 6 13 2 10 Predicted False 18 19,043 4 5,370 1 260 0 133 % cancerous (actual) 1.1% 1.3% 2.5% 1.4% Radiologists Google AI [1] Metric Training Validation 0.94 0.95 Sensitivity 0.86 1.00 0.04 0.06 PPV 0.32 0.17 0.07 0.11 ! " -score 0.46 0.29 0.73 0.81 Specificity 0.95 0.93 0.99 0.99 NPV 0.99 1.00 0.74 0.81 Accuracy 0.95 0.93 Ours Good results (but expensive) on small subset Need to increase training data and class imbalance I MPLEMENTATION IN P RACTICE Goal: Provide physicians some interpretation of the model’s output Each region has a cancer probability ! " The segmenter provides region of interest cubes Show those to the radiologist F UTURE D IRECTIONS • Scale current method to all 15, 000 patient scans • Apply PDE interpretation to the segmenter • Predict 5 classes (Lung-RADS) • Add a recurrent component to compare against past scans (nodule growth) S OFTWARE PyTorch implementation for CIFAR-10 and STL-10 available on Emory’s Machine Learning and Inverse Problems Github repository: github.com/EmoryMLIP/DynamicBlocks Lung-specific hyperparameters and data unavailable.
Transcript of LDCT Lung Cancer Detection, Derek Onken, Computer Science...
PDE-BASED NEURAL NETWORKS FOR LUNG CANCERDETECTION USING 3-D LDCT IMAGES
DEREK ONKEN PhD Candidate, Department of Computer Science, Emory UniversityData Science Intern, UnitedHealth Group R&D, Summer 2019
MOTIVATIONLung cancer:
• is responsible for the most deaths of all cancers
• begins localized in the lungs and spreads
5-year survival rates:– 19% (all stages)– 56% (still localized to the lungs)
• can be caught and treated early, reducing mortality
© 2019 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
LUNG CANCER IN THE U.S.
Incidence By Source [1]
Symptomatic
Incidental
Screening
85% 13%
2%
basic statistics
In 2018,• 541K people living with lung
cancer (prevalence)• 234K new cases (incidence)
2 / 30 Annual screening is recommended for the non-symptomatic high-risk population (smokers with >15pack-years; age 55-74). Radiologists read these scansto determine cancer diagnosis.
© 2019 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
LDCT PERFORMANCE
ActualTrue
ActualFalse
Total
PredictedTrue
270TP
6,911FP
7,181
PredictedFalse
18FN
19,043TN
19,061
Total 288 25,954 26,242
Data from baseline (T0) of NLST LDCT group.NLST Research Team. (2013). Results of initial low-dose computed tomographic screening for lung cancer. NEJM.
• 6,911 false positives even with practiced radiologists and good follow-up
participation by patients
• False positives lead to wasteful follow-up imaging and biopsy
Sensitivity = !"!"#$% = 93.8%
Specificity = !%!%#$" = 73.4%
Positive Predictive Value (PPV) = !"!"#$" = 3.8%
Negative Predictive Value (NPV) = !%!%#$% = 99.9%
F.-score = /!"/!"#$%#$" = 7.2%
NLST findings
9 / 30
Sensitivity(
TP
TP + FN
)= 93.8%
PPV(
TP
TP + FP
)= 3.8%
F1
(2TP
2TP + FN + FP
)= 7.2%
Data from LDCT groupbaseline (T0) of Nat’lLung Cancer ScreeningTrial (NLST) [2]
Current process can be expensive for providers, andpatients experience delays and high costs.
© 2019 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
PATIENT JOURNEY
1Gildea, et al. (2017). A retrospective analysis of delays in the diagnosis of lung cancer and associated costs.
delayed diagnoses + costs
First diagnostic test Definitive diagnosis
Ranges from one to six months
Commercial Population [3]
Of adult patients diagnosed with non-small cell lung cancer (2007-2011)• ~94% experienced a delay of 5-6 months• Mean per patient per month in total health care costs was
$2,407 ± $3,364
8 / 30 False positives render many of these costs wasteful.
⇒ Reduction in false positives at initial screeningsaves patients and providers time and money.
REFERENCES
[1] Disease Inception Accelerator. Patient Journey Research. 2018.
[2] NLST Reseach Team. Results of Initial Low-Dose Computed Tomographic Screening for Lung Cancer. NEJM, 2013.
[3] T Gildea et al. A Retrospective Analysis of Delays in Diagnosis. ClinicoEconomics and Outcomes Research: CEOR, 2017.
[4] E Haber, L Ruthotto. Stable Architectures for Deep Neural Networks. Inverse Problems, 2017.
[5] K He et al. Deep Residual Learning for Image Recognition. IEEE CVPR, 2016.
[6] L Ruthotto, E Haber. Deep Neural Networks Motivated by Partial Differential Equations. JMIV, 2019.
[7] F Liao et al. Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-or Network. IEEE TNNLS, 2019.
[8] D Ardila et al. End-to-end Lung Cancer Screening with 3-D Deep Learning. Nature Med., 2019.
PROBLEM & APPROACHDevelop a model that classifies a 3-D low-dose computed tomography (LDCT) scan as cancerous or non-cancerous. Performance goal: fewer false positives than practicing radiologists while matching sensitivity.
© 2019 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
!"
Probability:that cube contains a cancerous nodule?
no
#1 %&' ()*(&+,-'0 *,/ ()*(&+,-'
Model
Max ProbabilitySegment & pull out the nodules
300 slices, 512 x 512 each
Neural Network
!"
!"!"
15 / 30
Input LDCTOutput
Inner workings of the model:
• Segmenter pulls out smallercubes with the most likely can-cerous nodules (for scalability)
• PDE-based Classifier
– Predict cancer likelihood foreach cube
– Use from the max likelihoodcube for the patient
OPTIMAL CONTROL IN NNSGoal: Find a function f : Rp × Rn → Rm and param-eters θ ∈ Rp such that f(θ,y0) ≈ c for every traininginput y0 ∈ Rn and its label c ∈ Rm.
We make this continuous, viewing weights θ and fea-tures y as functions of time t ∈ [0, T ].
The neural network f(θ,y0) = y(T ) performs a non-linear transformation of inputs y0 satisfying the ordi-nary differential equation (ODE),
∂ty(t) = `(θ(t),y(t)) , for t ∈ (0, T ]
y(0) = y0,(1)
where ` : Rp ×Rn → Rn is a neural network.
A forward Euler discretization [4] of Eq. (1) is the N -layer ResNet
yj+1 = yj + h ` (θj ,yj) , where j = 0, 1, . . . , N−1,
with step size h = T/N . Borrowing from optimal con-trol, the θj are control layers and the yj are state layers.
Training: Tune θ and linear layer W by solving
minθ,W
1
s
s∑k=1
loss(f(θ,yk0)W, ck) + regularizer(θ,W)
across all s training inputs.
DECOUPLING WEIGHTS & LAYERSGoal: Improve training by maintaining few parame-ters but with many layers
We develop a generalized ResNet for the classifier:Resnet-14 [5] Our Framework
image
3x3 conv, 16, /2
pool, /2
3x3 conv, 16
3x3 conv, 16
3x3 conv, 16
3x3 conv, 16
3x3 conv, 32, /2
3x3 conv, 32
3x3 conv, 32
3x3 conv, 32
3x3 conv, 64, /2
3x3 conv, 64
3x3 conv, 64
3x3 conv, 64
avg pool
fc 10
image
3x3 conv, 16, /2
pool, /2
3x3 conv, 16
3x3 conv, 16
3x3 conv, 16
3x3 conv, 16
3x3 conv, 32
pool, /2
3x3 conv, 32
3x3 conv, 32
3x3 conv, 32
3x3 conv, 32
3x3 conv, 64
pool, /2
3x3 conv, 64
3x3 conv, 64
3x3 conv, 64
3x3 conv, 64
avg pool
fc 10
image
Opening Layer3 → 16
σ ◦ N ◦ K
Dynamic Block, 16
RK1 schemeDouble Layert = [0, 2]
hθ = hu = 1
Connecting Unit16 → 32
P ◦ σ ◦ N ◦ K
Dynamic Block, 32
RK1 schemeDouble Layert = [0, 2]
hθ = hu = 1
Connecting Unit32 → 64
P ◦ σ ◦ N ◦ K
Dynamic Block, 64
RK1 schemeDouble Layert = [0, 2]
hθ = hu = 1
avg pool
fc 10
Fixed-width portions of ResNet = a Dynamic BlockEach block contains:• continuous ODE like Eq. (1)• ODE solver scheme (e.g., Runge-Kutta 4)• discretization for the solver• neural network layer `
For `, we experiment with:
• Double Layer: σ2 ◦ N2 ◦Kθ2◦ σ1 ◦ N1 ◦Kθ1
(y)
• Double Symmetric Layer [6]: −K>θ ◦ σ ◦ N ◦Kθ(y)
for activation functions σ, normalizations N , and con-volution operators K defined by weights θ
FUNDING
Supported by the National Science Foun-dation awards DMS 1522599 and CAREERDMS 1751636, Binational Science Founda-tion Grant 2018209, NVIDIA Corporation,and UnitedHealth Group.
COLLABORATORS• Lars Ruthotto (Emory University)• Rachel Jennings (UnitedHealth Group)• Stephen Garth (UnitedHealth Group)• Simion Novikov (Ben Gurion, Israel)• Eran Treister (Ben Gurion, Israel)• Eldad Haber (UBC, Vancouver)
RESULTS• Use pre-trained segmenter [7]• Classifier uses Double Symmetric Layer• Minimize Focal Loss• Trained on a subset of 280 patient scans• Requires 1 week to train on a GPU with 16GB RAM
© 2019 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
Radiologists(NLST patients)
Google AI [8](NLST patients)
Training Set(NLST patients)
Validation Set(NLST patients)
ActualTrue
Actual False
ActualTrue
Actual False
ActualTrue
Actual False
ActualTrue
Actual False
Predicted True 270 6,911 82 1,260 6 13 2 10
Predicted False 18 19,043 4 5,370 1 260 0 133
% cancerous (actual) 1.1% 1.3% 2.5% 1.4%
19 / 30
Radiologists Google AI [1] Metric Training Validation
0.94 0.95 Sensitivity 0.86 1.00
0.04 0.06 PPV 0.32 0.17
0.07 0.11 !"-score 0.46 0.29
0.73 0.81 Specificity 0.95 0.930.99 0.99 NPV 0.99 1.000.74 0.81 Accuracy 0.95 0.93
Ours
Good results (but expensive) on small subsetNeed to increase training data and class imbalance
IMPLEMENTATION IN PRACTICEGoal: Provide physicians some interpretation of themodel’s output
© 2019 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
INTERPRETABILITY
Each region has a cancerprobability !"
potential utilization by physicians
The segmenter provides region of interest cubes
Show those to the radiologist
20 / 30
FUTURE DIRECTIONS• Scale current method to all 15, 000 patient scans• Apply PDE interpretation to the segmenter• Predict 5 classes (Lung-RADS)• Add a recurrent component to compare against
past scans (nodule growth)
SOFTWAREPyTorch implementation for CIFAR-10 and STL-10available on Emory’s Machine Learning and InverseProblems Github repository:
github.com/EmoryMLIP/DynamicBlocks
Lung-specific hyperparameters and data unavailable.
