Title CycleGANを用いたCT画像における金属アーチファクト低減法

Author(s) 中尾, 恵; 今西, 勁峰; 上田, 順宏; 今井, 裕一郎; 桐田, 忠昭;松田, 哲也

Citation 電子情報通信学会技術報告 (MI) (2019), 119(193): 63-68

Issue Date 2019-09

URL http://hdl.handle.net/2433/244334

Right © 2019 by IEICE 技術研究報告に掲載された論文の著作権は電子情報通信学会に帰属します．

Type Journal Article

Textversion publisher

Kyoto University

THE INSTITUTE OF ELECTRONICS,INFORMATION AND COMMUNICATION ENGINEERS

TECHNICAL REPORT OF IEICE.

CycleGAN CT† †† ††† †††† †††

†

† 606–8501†† 607–8062 403

††† 634–8521 840†††† 607–8062 2

E-mail: †megumi@i.kyoto-u.ac.jp

CT

CT CycleGANCT

3 3 96 3CT

, , , CT

Metal artifact reduction using CycleGAN for CT imagesMegumi NAKAO†, Keiho IMANISHI††, Nobuhiro UEDA†††, Yuichiro IMAI††††, Tadaaki

KIRITA†††, and Tetsuya MATSUDA†

† Graduate School of informatics, Kyoto UniversityYoshida Honmachi, Sakyo-ku, Kyoto, 606–8501, Japan

†† e-Growth Co., Ltd., 403, Shimo-Maruya-cho, Nakagyo-ku, Kyoto, 604–8006, Japan††† Department of Oral and Maxillofacial Surgery, Nara Medical University

Shijocho 840, Kashihara-shi, Nara, 634–8521, Japan†††† Department of Oral and Maxillofacial Surgery, Rakuwakai Otowa Hospital

Otowachinjicho 2, Yamashina-ku, Kyoto, 607–8062, JapanE-mail: †megumi@i.kyoto-u.ac.jp

Abstract In this study, we propose a method for metal artifact reduction in real CT images based on unsupervisedimage transfer. We do not suppose the existence of training data with reduced artifacts or synthesized images ofmetal artifacts, and focus on a problem to obtain domain transfer between clinical CT images with and withoutdental metals. Based on the concept of CycleGAN, a novel loss function that reduces metal artifacts while preserv-ing their CT values was designed. A adversarial training framework considering three-dimensional (3D) anatomicalstructures and 3D distribution of metal artifact was developed. This presentation reports the preliminary results ofmetal artifact reduction effectively performed by domain transfer learnt from 96 3D-CT images.Key words Unsupervised learning, metal artifact reduction, adversarial generative network, CT images

1.

Computed Tomography(CT) X

CT

— 1 —

Copyright ©201 by IEICE This article is a technical report without peer review, and its polished and/or extended version may be published elsewhere.

- 63 -

CT3 CT

[1] [2]CT

[3] [4][5]

CTCT [6]

CT

[7] [8][9] [10]

3CT

(Generative Adversarial Network, GAN) [11][12]

CT CTCT

[13]

CT

CT

CT

CycleGAN [12]

1 CT , X:, Y :

CT3

396 CT

2.

CTX = {xi}(i = 1, 2, ..., N), CT

Y = {yi}(i = 1, 2, ..., M)

X Y

1) CT2)

CycleGAN

2. 1 CycleGANGAN [11] X Y

(GX , DX), (GY , DY )

• Y → X GX

• X → Y GY

• GX(Y ) X DX

• GY (X) Y DY

GX , GY X, Y

DX , DY

50% DX , DY

GX , GY X, Y

CycleGAN [7] GX GY

X → Y → X Y → X → Y

(1)

Lcgan(GX , GY , DX , DY ) = Ladv(GY , DY , X, Y ) (1)

— 2 —- 64 -

2 3

+ Ladv(GX , DX , Y, X)

+ λLcyc(GX , GY ),

Ladv adversarial loss (2)GY (X) Y DY

Ladv(GY , DY , X, Y ) = Ey[log DY (y)] (2)

+ Ex[log(1 − DY (GY (x)))].

Lcyc cycle consistency loss (3)

Lcyc(GX , GY ) = Ex[||GX(GY (x)) x||1] (3)

+ Ey[||GY (GX(y)) y||1],

X → Y → X

GX(GY (x)) x

λ adversarial loss cycle consistency lossGX , GY (1)

DX , DY

(1) [7] adver-sarial loss D

cycle consistency lossG(x)

D

2. 2

CT(1)

feature loss,CT intensity loss

zero-centered gradient penalty [15]

feature loss (4) CTCT 3

L1 CTVGG16 [14]

Lfea = Ex[||f(GY (x)) f(x)||1 (4)

+||f(GX(GY (x)) f(GY (x))||1+||f(GX(GY (x)) f(x)||1]

+Ey[||f(GX(y)) f(y)||1+||f(GY (GX(y)) f(GX(y))||1+||f(GY (GX(y)) f(y)||1],

intensity loss (5) CT

CTlog

Lint = Ex[log(|GY (x) x| + 1.0)]

+ Ey[log(|GX(y) y| + 1.0)], (5)

zero-centered gradient penalty [15] (6)GAN

D

Lgp = Ex̃[||∇Dx̃||2 (6)

LMA (7) λfea, λint, λgp

G∗Y (8)

LMA = Lcgan + λfeaLfea + λintLint − λgpLgp (7)

G∗X , G∗

Y = arg minGX ,GY

maxDX ,DY

LMA(GX , GY , DX , DY ) (8)

2. 3 3CycleGAN x, y

21

3

— 3 —- 65 -

3 (a) , (b) CycleGAN (c)feature loss , (d) intensity loss , (e) gradient penalty

3

2 3 CTz

z s

T

T

T 3 1

Graphic Processing Unit (GPU)

3X Y

z

X

Y

3 x y

Y

3.

3 CT96 512 512 pixel, 157-515 slices

76 ,20

X Y 2

18601662 CT

[-1000HU, 1000HU][-1, 1]

T = 51epoch 5

CycleGAN (GX , DX) (GY , DY )

U-net [16], VGG16 [14] CPU:Intel Core i7-7900X, Memory: 32.0 GB, GPU: GeForce RTX2080, OS: Windows 10

3. 1

Cycle GAN 3

Model A CycleGAN Lcgan

Model B LMA λfea = 1.0, λint = 0.0, λgp = 0.0Model C LMA λfea = 1.0, λint = 50.0, λgp = 0.0Model D LMA λfea = 1.0, λint = 50.0, λgp = 0.0001

Model B feature loss Model C featureloss intensity loss , Model DModel C gradient penalty

— 4 —- 66 -

3CT

(b)(c)(d)

(b)(c)(d)

(e)

3. 23

Model D 45

3(a)(b)

(c)

2 1

GAN

3

4.

CT96

CT

(B)( 19H04484) ( ) (18K19918)

[1] J. I. Efanov, A. Roy, K. N. Huang, D. E. Borsuk, VirtualSurgical Planning: The Pearls and Pitfalls, Plast ReconstrSurg Glob Open., 6(1):e1443, 2018.

[2] B. Rigaud, A. Simon et al., Statistical Shape Model to Gen-erate a Planning Library for Cervical Adaptive Radiother-apy. IEEE Trans Med Imaging, 38(2), pp.406-416, 2019.

[3] M. Nakao, M. Hosokawa, Y. Imai, N. Ueda, T. Hatanaka, T.Kirita and T. Matsuda, Volumetric fibular transfer planning

with shape-based indicators in mandibular reconstruction,IEEE Journal of Biomedical and Health Informatics, 19(2),pp.581-589, 2015.

[4] M. Nakao, S. Aso, Y. Imai, N. Ueda, T. Hatanaka, M. Shiba,T. Kirita and T. Matsuda, Automated planning with mul-tivariate shape descriptors for fibular transfer in mandibu-lar reconstruction, IEEE Trans. on Biomedical Engineering,64(8), pp.1772-1785, 2017.

[5] M. Nakamura, M. Nakao, H. Hirashima, H. Iramina, T.Mizowaki, Performance evaluation of a newly developedthree-dimensional model-based global-to-local registrationin prostate cancer, Journal of Radiation Research, rrz031,2019.

[6] E. Meyer, R. Raupach, M. Lell, B. Schmidt, M. Kachelriess,Normalized metal artifact reduction (NMAR) in computedtomography. Med Phys, 37(10), pp. 5482-93, 2010.

[7] L. Gjesteby, Q. Yang, Y. Xi, Y. Zhou, J. Zhang, G. Wang,Deep learning methods to guide CT image reconstructionand reduce metal artifacts, SPIE Medical Imaging. Vol.10132. 2017.

[8] , , , , ,, , , , , ,

,AIM , 6(7), SIG-AIMED-006-07,

2018.[9] Y. Zhang and H. Yu, Convolutional neural network based

metal artifact reduction in X-ray computed tomography.IEEE Transactions on Medical Imaging, 37(6), pp.1370-1381, 2018.

[10] M. Sakamoto, Y. Hiasa, Y. Otake, M. Takao, Y. Suzuki,N. Sugano, Y. Sato, Automated Segmentation of Hip andThigh Muscles in Metal Artifact-Contaminated CT usingConvolutional Neural Network-Enhanced Normalized MetalArtifact Reduction, arXiv:1906.11484, 2019.

[11] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Gen-erative adversarial nets, In NIPS, 2014.

[12] J. Zhu, T. Park, P. Isola, A. A. Efros. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Net-works, in IEEE International Conference on Computer Vi-sion (ICCV), 2017.

[13] Q. Yang, P. Yan, Y. Zhang, H. Yu, Y. Shi, X. Mou, M. K.Kalra, Y. Zhang, L. Sun, G. Wang, Low-Dose CT ImageDenoising Using a Generative Adversarial Network WithWasserstein Distance and Perceptual Loss. IEEE Transac-tions on Medical Imaging, 37(6), pp.1348-1357, 2018.

[14] K. Simonyan, A. Zisserman, Very Deep Convolutional Net-works for Large-Scale Image Recognition, ICLR, 2014.

[15] H. Thanh-Tung, T. Tran, S. Venkatesh, Improving Gener-alization and Stability of Generative Adversarial Networks,ICLR, 2019.

[16] O. Ronneberger, P. Fischer, T. Brox, U-Net: ConvolutionalNetworks for Biomedical Image Segmentation, Medical Im-age Computing and Computer-Assisted Intervention (MIC-CAI), pp. 234-241, 2015.

— 5 —- 67 -

4 CT (a)(b)(c) , (d)(e)(f)

5 3 CT (a)(b)(c) , (d)(e)(f)

— 6 —- 68 -