RADIOGENOMICS learning GENOMIC CLASSES based on IMAGE … · gm_wm_bg_16-eps-converted-to.pdf...
1
RADIOGENOMICS learning GENOMIC CLASSES based on IMAGE FEATURES Esther Alberts, Benedikt Wiestler, Bjoern H. Menze Images Disease or class Classification study Survival time Prediction study Genetical data Correlation study Classification Regression Classification Regression Three common radiogenomics approaches Images with labels (training set) images + labels Unseen images (testing set) images Feature extraction Learning (Training) Classification (Testing) Classifier Learned classifier labels Images MR brain tumor images T1, T1c, T2, FLAIR tumor segmentations edema, active core, necrotic core Labels genetic subclasses CIMP codel, CIMP non-codel, CIMP neg CIMP codel CIMP non-codel Fig. 1: Data available for each patient: MR images, tumor segmentations and genetic subclass. Computer Aided Diagnosis – our application Radiogenomics – What are we learning? Multi-modal medical images Multi-modal segmentations Set of inputs I. Supervised learning by 3d lbp II. Unsupervised learning by auto-encoder III. Hand-coded shape features Feature extraction Feature fusion Classifiers svm,rf,knn Classification Feature extraction – Quantifying texture 1. Spherical sampling: get neighbours 2. Create a neighbourhood function by: • 0: neighbour intensity < center intensity • 1: neighbour intensity > center intensity 3. Get frequency components of spherical harmon- ics for the neighbourhood function Fig. 2: Spherical harmonics (order 1-4). Feature generation Using the training set: 1. Calculate LBPs for all tumor voxels 2. Learn common patterns ⇓ In the test set: cluster LBPs towards the learned patterns generated feature = frequency of the patterns in the new image Fig. 3: Learnt texture patterns ( = 25). Fig. 4: LBP clustering of tumor voxels. I. Supervised learning by 3D Linear Binary Patterns An auto-encoder is defined by encoder φ and decoder ψ transformations: φ : x ∈ R → z ∈ R (1) ψ : z ∈ R → x ∈ R (2) Here, << , in order to get a compressed feature representation. The transformations φ and ψ are learnt through a minimisation problem: argmin φψ ||x - (ψ◦φ)x|| 2 (3) Encoder φ 1 Decoder ψ 2 II. Unsupervised learning by auto-encoder Difficult classification task – human raters cannot differentiate CIMP codel CIMP non-codel CIMP neg Fig. 5: Prediction results using LBP, HOG and auto-encoder features from T1, T1 gad, T2 and FLAIR. We are looking for relations. In other words, we don’t know how well the best possible classifier would perform! • Only 117 patients (= 117 labels) • Every patient has 4 images (T1, T1 gad, T2 and FLAIR) • Over 500 features per image • ⇒ over 2000 features per label! Main problem: Curse of dimensionality • Smart feature selection to reduce dimension • Patch-based deep learning approach to increase samples • Testing on other datasets and other labels to look for more obvious relations • Investigate specialized classifiers • Add genetical metadata • Look into semisupervised learning techniques Next steps Classifiers – when p<<n . . . [1]Menze, B. H., Jakab, A., Bauer, S., et al.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans. Med. Imag (2014) [2] Menze, B.H., Van Leemput, K., Lashkari, D., et al.: A generative model for brain tumor segmentation in multi-modal images. Proc MICCAI 13(2), 151–159 (2010) [3] Wiestler, B., Capper, D., Sill, M., et al.: Integrated DNA methylation and copynumber profiling identify three clinically and biologically relevant groups of anaplastic glioma. Acta Neuropathol. (2014) [4] Banerjee, J., Moelker, A., Niessen, W. J., et al.: 3D LBP-Based Rotationally Invariant Region Description. ACCV Workshops (2012) Literature
Transcript of RADIOGENOMICS learning GENOMIC CLASSES based on IMAGE … · gm_wm_bg_16-eps-converted-to.pdf...
gm_wm_bg_16-eps-converted-to.pdf
RADIOGENOMICSlearning GENOMIC CLASSES based on IMAGE FEATURES
Esther Alberts, Benedikt Wiestler, Bjoern H. Menze
Images
Disease or class
Classification study
Survival time
Prediction study
Genetical data
Correlation study
ClassificationRegressionClassification Regression
Three common radiogenomics approaches
Images with labels(training set)
images + labels
Unseen images(testing set)
images
Feature extraction
Learning (Training)
Classification (Testing)
Classifier
Learned classifier
labels
Images MR brain tumor images T1, T1c, T2, FLAIR
tumor segmentations edema, active core, necrotic core
Labels genetic subclasses CIMP codel, CIMP non-codel, CIMP neg
CIMP codel CIMP non-codelFig. 1: Data available for each patient: MR images, tumor segmentations and genetic subclass.
Computer Aided Diagnosis – our application
Radiogenomics – What are we learning?
Multi-modalmedical images
Multi-modalsegmentations
Set of inputs
I. Supervised learningby 3d lbp
II. Unsupervised learningby auto-encoder
III. Hand-codedshape features
Feature extraction Feature fusion
Classifierssvm,rf,knn
Classification
Feature extraction – Quantifying texture
1. Spherical sampling: get neighbours2. Create a neighbourhood function by:• 0: neighbour intensity < center intensity• 1: neighbour intensity > center intensity
3. Get frequency components of spherical harmon-ics for the neighbourhood function
Fig. 2: Spherical harmonics (order 1-4).
Feature generationUsing the training set:
1. Calculate LBPs for all tumor voxels2. Learn n common patterns
⇓In the test set:
cluster LBPs towards the n learned patterns
generated feature= frequency of the n patterns
in the new image
Fig. 3: Learnt texture patterns (n = 25).
Fig. 4: LBP clustering of tumor voxels.
I. Supervised learning by 3D Linear Binary Patterns
An auto-encoder is defined by encoderφ and decoder ψ transformations:
φ : x ∈ Rd → z ∈ Rp (1)
ψ : z ∈ Rp→ x ∈ Rd (2)Here, p << d, in order to get acompressed feature representation.The transformations φ and ψ are learntthrough a minimisation problem:
argminφ,ψ
||x− (ψ ◦ φ)x||2 (3)Encoder φ 1 Decoder ψ 2
II. Unsupervised learning by auto-encoder
Difficult classification task – human raters cannot differentiate
CIMP codel CIMP non-codel CIMP neg
Fig. 5: Prediction results using LBP, HOG and auto-encoder features from T1, T1 gad, T2 and FLAIR.
We are looking for relations. In other words, we don’t know how well the best possible classifierwould perform!
•Only 117 patients (= 117 labels)• Every patient has 4 images (T1, T1 gad, T2 and FLAIR)•Over 500 features per image•⇒ over 2000 features per label!
Main problem: Curse of dimensionality
• Smart feature selection to reduce dimension•Patch-based deep learning approach to increase samples• Testing on other datasets and other labels to look for more obvious relations• Investigate specialized classifiers• Add genetical metadata• Look into semisupervised learning techniques
Next steps
Classifiers – when p<<n . . .
[1] Menze, B. H., Jakab, A., Bauer, S., et al.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS).IEEE Trans. Med. Imag (2014)
[2] Menze, B.H., Van Leemput, K., Lashkari, D., et al.: A generative model for brain tumor segmentation in multi-modalimages. Proc MICCAI 13(2), 151–159 (2010)
[3] Wiestler, B., Capper, D., Sill, M., et al.: Integrated DNA methylation and copynumber profiling identify three clinicallyand biologically relevant groups of anaplastic glioma. Acta Neuropathol. (2014)
[4] Banerjee, J., Moelker, A., Niessen, W. J., et al.: 3D LBP-Based Rotationally Invariant Region Description. ACCVWorkshops (2012)
Literature
