How a toolkit of diverse MS lesions segmentation methods can support the translation to the clinic

F. Galassi, O. Commowick, C. Barillot

2

FLAIR

•  Multiple Sclerosis (MS):

§  Chronic inflammatory-demyelinating CNS disease

§  Lead to acute handicap in young adults (high prevalence in Brittany)

§  Most frequent CNS disease in young adults

•  MRI for MS lesion evaluation:

§  clinical diagnosis

§  disease progression

§  treatment monitoring

•  Segmentation Methods:

§  guiding clinicians in the medical decision process

§  manual segmentation:

§  time consuming task

§  intra- inter- expert variability

Introduction

Wattjes, M. P. et al. (2015)

3

Introduction

•  Automatic Segmentation methods:

•  Multimodal intensity information

•  Spatial information:

•  local neighborhood level (e.g. MRF, Graph Cut)

•  anatomical level (probability templates, topological atlases)

FLAIR Gd-enhancing lesions Hyperintense T2w Hypointense T1w

FLAIR T1w Gd PDw T2w

From: Garcia-Lorenzo et al. Review of automatic segmentation methods of WM lesions on conventional MRI. Medical Image Analysis, 2013.

PiMS : Priors of MS lesions locations based on 72 subjects

4

Participants Approach Sequences

CMIC Multimodal patch matching T1w,T2w,PDw,FLAIR

VISAGES EM initialized Graph Cut T1w,T2w,FLAIR

IMI & PVG Random Forest T1w,T2w,PDw,FLAIR

ITT MADRAS n3 Convolutional Neuronal Network T1w,T2w,PDw,FLAIR

MSmetrix Hierarchical EM T1w, FLAIR

VISAGES Dictionary Learning T1w,T2w,FLAIR

U

SS

S

U

Automatic Segmentation Methods in Visages

Supervised Unsupervised

S

U

5/13

Participants Approach Sequences

Roura et al. Rules & Level Sets T1w,FLAIR

VISAGES EM initialized Graph Cut T1w,T2w,FLAIR

VISAGES Atlas based comparison T1w,T2w,PDw,FLAIR

McKinley et al. Deep Dag-like Convolutional Network

FLAIR

Valverde et al. Convolutional Network of 3D patches T1w,T2w,PDw,FLAIR

U

S

S

U

U

Commowick O et al. MSSEG Challenge Proceedings MS Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure, 2016.

Carass A et al. Longitudinal multiple sclerosis lesion segmentation: Resource and challenge, Neuroimage, 2017

7/10 S

Method #1 :MS Lesions as outliers from normal appearing brain tissues

5 … … tn

t1

t2 t2

tn

t1 Estimation of “normal” appearances

Identification of “irregular” appearances (e.g. lesions,

artifacts)

Neuroradiological Expertise

(keep WM lesions only)

plesion = N µWMT 2 ,σWM

T 2( )dyyth

∞

∫

di =minj (yi −µ j )T

j−1

Σ (yi −µ j ){ }> p tℵn2( )

θ̂MLE = argmaxθ

f (yi θ )i=1

m

∏"

#$

%

&'; f (yi θ ) = α j ⋅N µ j ,Σ j( )*

+,-

j=1

n

∑

θ̂TL = argmaxθ

f (ysort (i) θ )i=1

k

∏"

#$

%

&'; m

2 < k ≤m( )

0

1

22

3

22

n1

1yn1

2yn1

iyn1

my

n2

1yn2

2yn2

iyn2

my

D. Garcia. et al. IEEE-TMI 2011

6

EM-initialized Graph Cut:

•  Estimate NABT 3-class Gaussian Mixture model (multivariate):

𝑓 𝒚↓𝒊 �𝜃  = ∑𝑗=1↑3▒𝛼↓𝑗  𝑁(𝜇↓𝑗 , Σ↓𝑗  ) •  Maximum Likelihood principle with Expectation Maximization

•  MS lesions as outliers, Trimmed EM segmentation:

TL(𝜃)=𝑎𝑟𝑔𝑚𝑎𝑥∏𝑖=1↑𝑘▒𝑓 𝒚↓𝒗(𝒊) �𝜃    •  Reject ℎ=(𝑛− 𝑘/𝑛 ) voxels with largest residuals •  Compute EM segmentation on remaining ones

•  Output:

•  Mean 𝜇 and covariance Σ of each class j •  Mahalanobis distances of each voxel to each distribution:

𝑑↓𝑖 =√� ( 𝒚↓𝒊 −𝜇↓𝑗 )↑𝑇 Σ↓𝑗↑−1  (𝒚↓𝒊 −𝜇↓𝑗 ) 

CSF

GM WM

NABT probability model, FGM model

Mahalanobis distance, 3 classes.

D. Garcia. et al. IEEE-TMI 2011

7

•  Image as a Graph: •  Weight of a n-link represents voxels similarity

•  Compute the optimal cut between MS lesions and background

•  t-links weights:

•  χ↑2  p-value𝑀𝑎ℎ𝑎𝑙𝑎𝑛𝑜𝑏𝑖𝑠 : probability not to fit into NABT : probability not to fit into NABT

•  MS lesions have high p-value𝑀𝑎ℎ𝑎𝑙𝑎𝑛𝑜𝑏𝑖𝑠

𝑊↓𝑠𝑖𝑛𝑘 =β(1 − min�(p−value𝑀𝑎ℎ𝑎𝑙𝑎𝑛𝑜𝑏𝑖𝑠) )+ (1- β)PiMS

•  Distinguish from other outliers using hyperintensity

•  𝑊↓𝑠𝑜𝑢𝑟𝑐𝑒 =𝑚𝑖𝑛{p−value𝑀𝑎ℎ𝑎𝑙𝑎𝑛𝑜𝑏𝑖𝑠, 𝑊↓𝑇2𝑊 , 𝑊↓𝐹𝐿𝐴𝐼𝑅 }

Sink: NABT

Source: MS lesions

voxels

t-link, Wsource

t-link, Wsink

n-link

Mahalanobis distance distribution, p-value

MS hyperintensity, FLAIR mage

EM-initialized Graph Cut:

Intensities prior Spatial prior

8

•  Limitations: tuning of parameters for different lesion loads.

•  Accuracy increases with TLL

•  Priors can improve accuracy

EM-initialized Graph Cut:

Without priors

With location priors

•  Dice x  PPV

9

•  Approach: MS lesions as outliers to normal control subjects

•  Intensity normalization: •  Estimate NABT 3-class GM model (multivariate):

𝑓 𝒚↓𝒊 �𝜃  = ∑𝑗=1↑3▒𝛼↓𝑗  𝑁(𝜇↓𝑗 , Σ↓𝑗  ) •  using Notsu et al. 𝛾-loss EM robust to outliers -loss EM robust to outliers

•  Output: Mean 𝜇 and covariance Σ of each class •  Linear regression on means for normalization

•  Output: intensity-normalized image patients

Method #2 : Atlas-based comparison segmentation

Healthy Control Subjects

Statistical atlas

Registration

&Comparison

Intensity Normalized Patient

10

•  Three steps segmentation:

•  Registration on healthy controls •  Non-linear registration

Voxel-wise computation (Mahalanobis distance) •  Patient vs Controls mean and variance

Derivation of p-value of abnormality presence •  Corrected for multiple comparisons

Atlas-based comparison segmentation

•  Post-processing : §  remove lesions too small or touching the brain border §  keep lesions inside a probable lesions mask

•  Towards a locally multivariate approach •  A contrario approach [Maumet et al. Neuroimage 2016]

Healthy Control Subjects

Statistical atlas

Registration

&Comparison

Intensity Normalized Patient

Machine learning: Probabilistic One Class SVM for Automatic Detection of MS Lesions

Goal: Propose an automatic framework for MSL Detection based on multichannel MRI patch based information

State-of-the-art machine learning algorithms: •  SVM [Vapnik et al.1995], Logistic Regression [Zhang et al.2002], Neural Network… •  Works well in practice when training examples in classes are balanced

If not ? •  Class Imbalance ⇒ under-/over-fitting of the Classifier [Chawala 2005] •  Class imbalance between Normal Brain Tissues and MS lesions •  Solution : A higher misclassification penalty on the minority class (MS lesion)

Toy example of SVM for balanced and unbalanced classes, Courtesy : www.scikit-learn.org.

[Karpate et al, 2015]: Probabilistic One Class Learning for Automatic Detection of MS Lesions. Proceedings of ISBI 2015

Training Phase Testing Phase

12

Method #3 : Probabilistic One Class SVM for Automatic Detection of MS Lesions

[Karpate et al, 2015]: Probabilistic One Class Learning for Automatic Detection of MS Lesions. Proceedings of ISBI 2015

Training Phase Testing Phase

13

Method #3 : Probabilistic One Class SVM for Automatic Detection of MS Lesions

Axial FLAIR Ground Truth

OCSVM Probabilistic OCLSVM

•  Goal: Robust detection of lesions as deviation from normal appearing tissues

•  Challenge: overcome learning approaches problems with MS lesions

•  Two-class imbalance problem (much more normal samples than lesions)

•  Contributions:

§  Robust spatio-temporal multi-modal intensity normalization for T1-Gd and longitudinal MS lesion detection

§  One class learning for lesion detection from multidimensional MRI §  Dimensionality reduction of the feature space

§  Lesions modeled as the complementary of the normal class

§  Testing by comparing patient patches characteristics to the pdf of the normal class

§  Able to robustly detect lesion locations in patients

[Karpate et al, 2015]: Probabilistic One Class Learning for Automatic Detection of MS Lesions. Proceedings of ISBI 2015

Axial FLAIR Ground Truth OCSVM Probabilistic OCLSVM

14

Method #3 : Probabilistic One Class SVM for Automatic Detection of MS Lesions

•  Goal: New sparse representation and dictionary learning method for classification

•  Challenge: competitive dictionary learning

§  One dictionary per class, classification decision based on reconstruction error

§  Representative Dictionary Learning : good for denoising, inpainting, … How to optimize DL for classification

•  Sparse Representation: SR represents signals using linear combination of few basis elements in a set of

redundant basis functions:

§  SR is an optimization problem (ɛ is an approximation error):

•  Related Dictionary learning (DL) : Finds D such that each signal can be represented by sparse linear

combination of its atoms:

§  Classification using DL: find k classes such as :

[Deshpande et al, 2015]: Classification of Multiple Sclerosis Lesions using Adaptive Dictionary Learning. Computerized Medical Imaging and Graphics, (Dec.), 2015

Method #4 : Detection of MS lesions via competitive Dictionary Learning

Method #4 : Detection of MS lesions via competitive Dictionary Learning

•  Goal: New sparse representation and dictionary learning method for classification

•  Contribution:

§  Adaptation of dictionary size to a class complexity: improved over standard DL or discriminative methods

§  Detection of MS Lesions by classification on multimodal MRI images

[Deshpande et al, 2015]: Classification of Multiple Sclerosis Lesions using Adaptive Dictionary Learning. Computerized Medical Imaging and Graphics, (Dec.), 2015

Axial FLAIR Axial T1-w Axial T2-w Axial PD

16

Σmethodi : Merging MS lesions Annotations

17

Automatic Annotations Consensus (LOP STAPLE*)

Akhondi-Asl A, Hoyte L, Lockhart ME, Warfield SK. A Logarithmic Opinion Pool Based STAPLE Algorithm For The Fusion of Segmentations With Associated Reliability Weights, IEEE Trans Med Imaging. 2014

18

Conclusions

FLAIR •  Challenges:

§  Multicenter datasets, Availability (MICCAI challenges)

§  Several manual segmentations (STAPLE,…)

§  Accuracy: Multiple metrics (DSC, F1,…) §  Robustness: Large multicenter database (MUSIC) –

combination of methods to compensate for limitations of each methods

§  ???

MUSIC étape 4 : analyse des images

19

MUSIC étape 4 : analyse des images

20

Contrôle qualité des images

Segmentation des lésions

Détection de l’évolution des lésions

Basées sur algorithmes développés au laboratoire Visages

60

Chapter 5. Longitudinal Intensity Normalization in Multiple

Sclerosis Patients

5.3.5 Active Gd-Enhanced Lesions Detection

Table 5.6 reports values of precision and recall for various thresholds for Gd-dataset 1. For this dataset, a high recall is achieved at 0.92 at ' = 0.2 and0.79 even at ' = 0.4. Figure 5.11 shows the active lesion in T1-w Gd imageand its detection. The 1�Precision vs Recall plot is constructed as describedin Section 5.3.4. We report the AUC for proposed method as 0.73. It is evidentfrom the plot that our proposed method outperforms other methods even forGd dataset.

Figure 5.10: Lesion detection examples. For top and bottom, from left to right:Slice of FLAIR for t0, t3, |t0 � t3(Normalized)|, ground truth and lesions detectedby our algorithm.

60

Chapter 5. Longitudinal Intensity Normalization in Multiple

Sclerosis Patients

5.3.5 Active Gd-Enhanced Lesions Detection

Table 5.6 reports values of precision and recall for various thresholds for Gd-dataset 1. For this dataset, a high recall is achieved at 0.92 at ' = 0.2 and0.79 even at ' = 0.4. Figure 5.11 shows the active lesion in T1-w Gd imageand its detection. The 1�Precision vs Recall plot is constructed as describedin Section 5.3.4. We report the AUC for proposed method as 0.73. It is evidentfrom the plot that our proposed method outperforms other methods even forGd dataset.

Figure 5.10: Lesion detection examples. For top and bottom, from left to right:Slice of FLAIR for t0, t3, |t0 � t3(Normalized)|, ground truth and lesions detectedby our algorithm.

Flair ti Flair ti+1 Difference Changes Detection Labeled Evolution

Etape 4 : une étape délicate…

Validationetaméliorationdel’outildesegmentation-  Financementd’unposted’ingénieurderecherchependant2ans-  PHRCinter-régionalobtenuen2016

ETUDE POCADIMS

Performance d’un outil d’aide au diagnostic des lésions visualisées en IRM dans le diagnostic et le suivi de la SEP (CADIMS) en pratique clinique

Objectif principalEvaluer, en aveugle, la performance diagnostique de l’outil CADIMS du projet MUSIC pour la détection de lésions de SEP sur des IRM cérébrales réalisées en pratique courante en comparaison à un consensus d’expert.