Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper...
-
Upload
debdoot-sheet -
Category
Education
-
view
175 -
download
1
Transcript of Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper...
![Page 1: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/1.jpg)
Deep Learning of Tissue Specific Speckle
Representations in Optical Coherence
Tomography and Deeper Exploration for
In situ Histology
Debdoot Sheet
@ Department of Electrical Engineering, Indian Institute of Technology Kharagpur, India.
Sri Phani Krishna Karri, Jyotirmoy Chatterjee
@ School of Medical Science and Technology, Indian Institute of Technology Kharagpur, India
Amin Katouzian, Nassir Navab
@ Chair for Computer Aided Medical Procedures, TU Munich, Germany
Ajoy K. Ray
@ Electronics and Electrical Comm. Engg., Indian Institute of Technology Kharagpur, India.
1ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet
![Page 2: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/2.jpg)
Motivation• Soft tissues – e.g. skin
– Epithelial
– Connective
– Muscular
– Adipose
• Pathological markers– Extracellular matrix deposition
– Cellular atypia and dysplasia
– Loss of histo-architecture
– Proliferative changes
• Conventional histology– Patient discomfort
– 48-72 hours delay in processing
• Alternatives – Subsurface imaging
• Optical coherence tomography (OCT)
• Challenges with the alternative– Hard to interpret
– Stochastic uncertainty of speckles
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 2
Epithelium, Papillary
dermis, Dermis, Adipose
![Page 3: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/3.jpg)
Where do we stand now?
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 3
This Paper
Text books
R. K. Das (2012), PhD Thesis
A. Barui (2011), PhD Thesis
D. Sheet et.al., ISBI 2014
![Page 4: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/4.jpg)
State of the Art• In situ Histology with OCT
– G. van Soest et al., (2010), G. J. Ughi et al., (2013) –Cardiovascular OCT
– D. Sheet et al., (2013, 2014) –Cutaneous wounds, oral
• Challenges– Heuristic features
• Texture
• Intensity statistics
– Heuristic computational models• Transfer learning of speckle
occurrence models
– Incomplete representation dictionary
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 4
Multi-scale
modeling of
OCT speckles
Training
image
set Ground
truth
Random forest
learning
Multi-scale
modeling of
OCT speckles
Test image
Labeled
tissue
![Page 5: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/5.jpg)
Heuristics in State of Art
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 5
![Page 6: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/6.jpg)
The Solution
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 6
Den
oisi
ngA
uto
Enc
oder
Den
oisi
ngA
uto
Enc
oder
Logi
stic
Reg
.
![Page 7: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/7.jpg)
Unfurling the Deep Network
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 7
![Page 8: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/8.jpg)
Learning of Representations
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 8
Representation of speckle
appearance models learned by DAE1
![Page 9: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/9.jpg)
Learning of Representations
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 9
Sparsity of representations learned by
DAE2
![Page 10: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/10.jpg)
Experiment Design• Data Collection
– School of Medical Science and Technology, Indian Institute of Technology Kharagpur
– 1300 nm (HPBW 100 nm) Swept Source OCT System • OCS 1300 SS, ThorLabs, NJ,
USA
• 8 bit bitmap images
– Histology for ground truth• HE stained
• Samples– Mus musculus (small mice)
– 16 healthy skin
– 2 wounds on skin
• DNN architecture– Patch size – 36 × 36 px
– DAE1 – 400 nodes
– DAE2 – 100 nodes
– Target – Logistic Reg. • 5 outputs
– Sparsity – 20%
– Mini-batch training
• In situ Histology Performance– Epithelium – 96%
– Papillary dermis – 93%
– Dermis – 99%
– Adipose tissue – 98%
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 10
![Page 11: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/11.jpg)
Results in Wounds
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 11
(a) OCT image of wound (b) Ground truth (c) In situ histology
Epithelium, Papillary
dermis, Dermis, Adipose
Epithelium, Papillary
dermis, Dermis, Adipose
![Page 12: Deep Learning of Tissue Specific Speckle Representations in Optical Coherence Tomography and Deeper Exploration for In situ Histology](https://reader030.fdocuments.us/reader030/viewer/2022032616/55a8d83b1a28abb93e8b45d8/html5/thumbnails/12.jpg)
Take Home Message• Photons interact characteristically with different tissues.
– Stochastic similarity exists in speckle appearance.
– Such representations are hard to heuristically encode.
• Deep learning and auto-encoders for computational imaging– Speckle imaging application viz. OCT tissue characterization
– Hierarchical learning• Locally embedded representations.
• Sparsity is in learned (auto-encoded) representations.
ISBI 2015 / FrDT3.5 - Deep Learning of Tissue Specific Speckle... - Debdoot Sheet 12
Queries: Debdoot Sheet ([email protected])