Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep...
Transcript of Deep Learning for Medical Image Analysis - University of Oulu learning... · 2017-06-11 · Deep...
DeepLearningforMedicalImageAnalysis
AlekseiTiulpinResearchUnitofMedicalImaging,PhysicsandTechnology
UniversityofOulu
Outline
• WhatisDeepLearning• MachineLearning• Convolutionalneuralnetworks:computervisionbreakthrough• Applications:Images,Video,Audio• Interpretability• Transferlearning• Limitations
• MedicalImageanalysis• Segmentation• Skincancerdetectionatadermatologistlevel• DiabeticRetinopathy• Ownstudy:KneeOsteoarthritisdiagnosis
2A.Tiulpin,MIPT,UniveristyofOulu
WhatisMachineLearning?
3
MachineLearning
Space𝑿 (objects) Space𝒀 (labels)
Mapping𝑎(𝒙)
Object𝒙 Answer𝒚
4A.Tiulpin,MIPT,UniveristyofOulu
Learning(supervised)
𝑎( 𝒙 ≈ 𝑎(𝒙)
LearningLookingforoptimal
parameters𝜃
+
Newdata
Label
Prediction
5A.Tiulpin,MIPT,UniveristyofOulu
Artificialneuron
𝑥,
𝑥-
𝑥.
1
𝜃/𝜃,
𝜃-
𝜃.
𝜽1𝒙
Activation
Dendrite
Synapse
Axon
Cellbody
https://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/Complete_neuron_cell_diagram_en.svg/1280px-Complete_neuron_cell_diagram_en.svg.png
6A.Tiulpin,MIPT,UniveristyofOulu
ArtificialNeuralNetworks:non-linerfunctions
0 𝑥,
𝑥-
http://neuralnetworksanddeeplearning.com/images/tikz35.png
7A.Tiulpin,MIPT,UniveristyofOulu
Popularimagefeaturedescriptors
• HistogramofOrientedGradients- HoG• Detectionofeverything
• Person,car,roadsign,face,…
• Shift-InvariantFeaturetransform- SIFT• Key-pointmatching• …
• LBP• Face detection• Texture analysis
A.Tiulpin,MIPT 8
WhatisDeepLearning
Data Featureextraction Predictionmodel
Data Featureextraction Prediction
ConventionalMachineLearningpipeline
DeepLearning
Featurelearning
Deeppredictionmodel
9A.Tiulpin,MIPT,UniveristyofOulu
DeepLearningvs.Hand-craftedfeatures
Hand-craftingfeatures……
A.Tiulpin,MIPT,UniveristyofOulu 10
Whydoweneedit?
• ImageNet –generalimagerecognitionchallenge• 1000classes• Millions oftrainingimages
DeepLearning
https://www.quora.com/What-is-the-winning-top-5-error-rate-on-ImageNet-2016
Top-5%accuracy
11A.Tiulpin,MIPT,UniveristyofOulu
DeepLearningbreaktrough:AlexNet
• AlexNet• Krizhevsky,A.,Sutskever,I.,&Hinton,G.E.(2012).Imagenet classificationwithdeepconvolutionalneuralnetworks.In Advancesinneuralinformationprocessingsystems (pp.1097-1105).• Cited>11800times according toGoogleScholar
http://whatsnext.nuance.com/in-the-labs/convolutional-neural-networks-applied-to-natural-language-understanding-research/
Drasticerrorratedrop(>10%)achievedbylearningfeaturesdirectlyfromdata!
12A.Tiulpin,MIPT,UniveristyofOulu
ConvNet layerhierarchy:localconnections
13A.Tiulpin,MIPT,UniveristyofOulu
DeepLearning:conceptsoutofsimpleconcepts
Yann LeCun et.al,doi:10.1038/nature14539 IanGoodfellow et.al, DeepLearningBook
Deeplearningallowscomputationalmodelsthatarecomposedofmultipleprocessinglayerstolearnrepresentationsofdatawithmultiple
levelsofabstraction
Deeplearningallowsthecomputertobuildcomplexconceptsoutofsimplerconcepts.
https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/
14A.Tiulpin,MIPT,UniveristyofOulu
Convnets:model ofthe brain
• InferiorTemporal(IT)cortex• Visualrecognitiontasks
• ConvNet performsequivalentlywelltoSVMtrainedontopoftheactivationsfromITcortex
15A.Tiulpin,MIPT,UniveristyofOulu
Isitreallyablack-box?
DeepLearning
16A.Tiulpin,MIPT,UniveristyofOulu
Isitreallyablack-box?
ConvNets
RandomForest&GBM
KernelMethods
17A.Tiulpin,MIPT,UniveristyofOulu
Attentionmaps&GuidedBackpropagation
• Neuralnetisadifferentiablefunction• Wecanfindagradientwithrespecttotheimageforanyclass• Details:GradCAM
18A.Tiulpin,MIPT,UniveristyofOulu
Transfer learning
• Deepmodelslearnverygenericfeaturesatthefirstlayers• Iftheytrainedonalargedatasethavingmanycategories,e.g.ImageNet
• Canweusedpre-trainedfeaturesinthenewdomains?
ImageNetfeatures Breastcancerdiagnosis
19A.Tiulpin,MIPT,UniveristyofOulu
Limitations
• Data• Modelswithmanyparameterseasilyoverfit• Wisearchitecturedesignisneededtoperformwell• High-endequipmentisneededtotrainthemodels• Linearnatureofthemodelsallowsto“fool”them• See,Adversarialexamples
20A.Tiulpin,MIPT,UniveristyofOulu
MedicalImagesegmentation
21A.Tiulpin,MIPT,UniveristyofOulu
SkinCancer:transferlearningsuccess
• InceptionV3network• ImageNet
• Dermatologistlevelclassification• Canbesooninyourmobilephone
22A.Tiulpin,MIPT,UniveristyofOulu
DiabeticRetinopathy
• Kaggle competition• $100000prize• Kappacoefficient0.85wasachieved
23A.Tiulpin,MIPT,UniveristyofOulu
Summary
• WithDeeplearningwecandopracticallyanything• Classify• Regress• Segment
• Weneedalotofdatatotraindeepmodels• Wealsoneedpowerfulhardwaretotrainthenetworks• Pre-trainedmodelscanbeusedtoachieveareallygoodperformanceinclassificationandregression
A.Tiulpin,MIPT 24
DeepLearningforMedicalImageAnalysis
AlekseiTiulpinResearchUnitofMedicalImaging,PhysicsandTechnology
UniversityofOulu
Imagesused
• http://cdn2-www.superherohype.com/assets/uploads/gallery/terminator-genisys/terminator-genisys-018.jpg
• http://2.bp.blogspot.com/-7BaGjkSq6rc/VKBPTRYzNgI/AAAAAAAANyo/4VsIBTP-NVY/s1600/hog.jpg
• https://pdfs.semanticscholar.org/44bd/549524fb17e3e806723fef2a18ffe9cfba87.pdf?_ga=2.65825965.2083661991.1497171718-775385516.1497171718
• http://2.bp.blogspot.com/-WB8VomB7kLY/UTx6se-U1ZI/AAAAAAAAA5g/kEbN46LHty8/s1600/matched_sift.jpg
• http://www.cavemanlogic.com/i/caveman.jpg• https://commons.wikimedia.org/wiki/File%3AGray1197.png• https://arxiv.org/abs/1610.02391• https://devblogs.nvidia.com/parallelforall/deep-learning-computer-vision-caffe-cudnn/• https://blogs.nvidia.com/wp-content/uploads/2016/09/19-beck-image.jpg
A.Tiulpin,MIPT,UniveristyofOulu 26
Imagesused
• https://www.nature.com/article-assets/npg/srep/2016/160607/srep27327/images/w582/srep27327-f1.jpg• https://www.researchgate.net/profile/Marius_George_Linguraru/publication/262293298/figure/fig2/AS:296674879655946@1447744211805/Fig-2-Examples-of-liver-segmentation-overlayed-in-green-on-axial-views-of-3D-CT-data.png• http://www.jpathinformatics.org/articles/2016/7/1/images/JPatholInform_2016_7_1_29_186902_u11.jpg• https://deepsense.io/wp-content/uploads/2015/09/pred_2.jpg
A.Tiulpin,MIPT,UniveristyofOulu 27
Imagesused
• http://5047-presscdn.pagely.netdna-cdn.com/wp-content/uploads/2015/08/DR4_1.png• https://www.kaggle.com/c/diabetic-retinopathy-detection
A.Tiulpin,MIPT,UniveristyofOulu 28