Dilated Residual U-Net for Multi-organ Segmentation in

Thoracic CT

Is 2D better than 3D segmentation?

Sulaiman Vesal1, Nishant Ravikumar1, Andreas Maier1

1Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

ISBI2019 SegTHOR Challenge

2April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Radiotherapy & Organs at Risk (OAR)

Figure: Risks of side effects in the surrounding organs increases during breast cancer radiotherapy.

Radiation

Radiation

Cancer

Heart exposed to

radiation during

radiotherapy of left

sided breast cancer

3April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Motivation

Axial View Sagittal View Coronal View

4April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Motivation

Axial View Sagittal View Coronal View

Esophagus

Trachea

Heart

Aorta

5April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

3D U-Net[1]

[1] Çiçek et. al. MICCAI 2016

[2] Yee et. al. 2017

6April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

3D U-Net[1]

[1] Çiçek et. al. MICCAI 2016

[2] Yee et. al. 2017

3D U-Net Output LGE-MRI Volume Ground Truth

7April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Pre-ProcessingPre-Processing

Figure: Pre-processing pipeline for SegTHOR Data

Cropping

(Nx288x288)(Nx512x512)

CLAHE

112 px

112 px

112 px

112 px

8April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

2D UNet + DR Architecture

9April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

2D UNet + DR Architecture

10April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Loss Functions

● Multi-class Dice Loss[1]:

No of classes Predicted voxel of class k Ground truth

[1] Milletari et. al. MICCAI 2016

[2] Salehi et. al. MICCAI MLMI 2017

● Tversky Loss[2]:● Tversky Loss[2]:

11April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

SegTHOR Dataset[1]

● 60 CTs from patients with non small cell lung cancer

● Resolution: 0.98×0.98×2.50mm3

● Dimensions:

● N×512×512 voxels

● N → 150 to 284 slices

[1] https://competitions.codalab.org/competitions/21012#learn_the_details-overview

● Data Split:

40 Patients 20 Patients

Training data Testing data

5 Fold-Cross Validation

12April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

ISBI SegTHOR

DSC: 0.921(0.016)

HD: 0.016(0.174)DSC: 0.942(0.028)HD: 0.092(0.054)

DSC: 0.909(0.062)

HD: 0.304(0.161)

Methods Training DSC Validation DSC Testing DSC Testing HD

2D DR U-Net 0.978 0.936 0.887 0.446

2D DR U-Net (Augmented) 0.974 0.945 0.914 0.254

2D DR U-Net (Augmented) + Tversky Loss 0.975 0.946 0.916 0.250

Patient_41 Patient_44Patient_55

13April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Quantitative Analysis

Figure: Violin box plot of Dice score for individual organs for 2D UNet + DR method

14April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Quantitative Analysis

Figure: Violin box plot of Hausdorf distance for individual organs for 2D UNet + DR method

15April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

ISBI SegTHOR Leaderboard

16April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Conclusion

● 2D U-Net+DR outperformed 3D U-Net with less trainable parameters

● Incorporating global contextual information improve the performance

● Fast (≈ 5.08 seconds) and robust method

● MR images, may be beneficial for some OARs with poorly-visible boundaries such as the esophagus

17April 4th - 2019Sulaiman Vesal | ISBI2019 SegTHOR | Dilated Residual U-Net for Multi-organ Segmentation in Thoracic CT

Thank you for your attention!

https://www5.cs.fau.de/~vesal

Questions?