Deep Learning in MR Image Processing |
Lee, Doohee
(Laboratory for Imaging Science and Technology, Department of Electrical and Computer Engineering, Institute of Engineering Research, Seoul National University)
Lee, Jingu (Laboratory for Imaging Science and Technology, Department of Electrical and Computer Engineering, Institute of Engineering Research, Seoul National University) Ko, Jingyu (Laboratory for Imaging Science and Technology, Department of Electrical and Computer Engineering, Institute of Engineering Research, Seoul National University) Yoon, Jaeyeon (Laboratory for Imaging Science and Technology, Department of Electrical and Computer Engineering, Institute of Engineering Research, Seoul National University) Ryu, Kanghyun (Department of Electrical and Electronic Engineering, Yonsei University) Nam, Yoonho (Department of Radiology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea) |
1 | LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998;86:2278-2324 DOI |
2 | Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning. Cambridge: MIT Press, 2016 |
3 | Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015;61:85-117 DOI |
4 | Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015:1-9 |
5 | Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems. 2014;2:2672-2680 |
6 | Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:1125-1134 |
7 | Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-toimage translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2017:2223-2232 |
8 | Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature 2018;555:487-492 DOI |
9 | Lauterbur PC. Image formation by induced local interactions. Examples employing nuclear magnetic resonance. 1973. Clin Orthop Relat Res 1989:3-6 |
10 | Shmueli K, de Zwart JA, van Gelderen P, Li TQ, Dodd SJ, Duyn JH. Magnetic susceptibility mapping of brain tissue in vivo using MRI phase data. Magn Reson Med 2009;62:1510-1522 DOI |
11 | Liu T, Spincemaille P, de Rochefort L, Kressler B, Wang Y. Calculation of susceptibility through multiple orientation sampling (COSMOS): a method for conditioning the inverse problem from measured magnetic field map to susceptibility source image in MRI. Magn Reson Med 2009;61:196-204 DOI |
12 | Liu T, Liu J, de Rochefort L, et al. Morphology enabled dipole inversion (MEDI) from a single-angle acquisition: comparison with COSMOS in human brain imaging. Magn Reson Med 2011;66:777-783 DOI |
13 | Wharton S, Schafer A, Bowtell R. Susceptibility mapping in the human brain using threshold-based k-space division. Magn Reson Med 2010;63:1292-1304 DOI |
14 | de Rochefort L, Liu T, Kressler B, et al. Quantitative susceptibility map reconstruction from MR phase data using bayesian regularization: validation and application to brain imaging. Magn Reson Med 2010;63:194-206 DOI |
15 | Papyan V, Romano Y, Elad M. Convolutional Neural Networks Analyzed via Convolutional Sparse Coding. J Mach Learn Res 2017;83:1-52 |
16 | Sodickson DK, Manning WJ. Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays. Magn Reson Med 1997;38:591-603 DOI |
17 | Mansfield P, Maudsley AA. Medical imaging by NMR. Br J Radiol 1977;50:188-194 DOI |
18 | Griswold MA, Jakob PM, Heidemann RM, et al. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 2002;47:1202-1210 DOI |
19 | Wiatowski T, Bolcskei H. A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction. IEEE Trans Inf Theory 2018;64:1845-1866 DOI |
20 | Ye JC, Han Y, Cha E. Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems. SIAM J Imaging Sci 2018;11:991-1048 DOI |
21 | Lustig M, Donoho D, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn Reson Med 2007;58:1182-1195 DOI |
22 | Wang G, Ye JC, Mueller K, Fessler JA. Image reconstruction is a new frontier of machine learning. IEEE Trans Med Imaging 2018;37:1289-1296 DOI |
23 | Boyd S, Vandenberghe L. Convex optimization. New York: Cambridge University Press, 2004:127-214 |
24 | Han Y, Yoo J, Kim HH, Shin HJ, Sung K, Ye JC. Deep learning with domain adaptation for accelerated projection-reconstruction MR. Magn Reson Med 2018;80:1189-1205 DOI |
25 | Ye JC, Han Y, Cha E. Deep convolutional framelets: a general deep learning framework for inverse problems. SIAM J Imaging Sci 2018;11:991-1048 DOI |
26 | Lee D, Yoo J, Ye JC. Deep residual learning for compressed sensing MRI. In IEEE 14th International Symposium on Biomedical Imaging (ISBI). 2017:15-18 |
27 | Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018;79:3055-3071 DOI |
28 | Knoll F, Hammernik K, Kobler E, Pock T, Recht MP, Sodickson DK. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn Reson Med 2019;81:116-128 DOI |
29 | Razavian AS, Azizpour H, Sullivan J, Carlsson S. CNN features off-the-shelf: an astounding baseline for recognition. arXiv preprint arXiv 2014:1403.6382 |
30 | Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? In Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds.). Advances in neural information processing systems 27. Curran Associates, Inc. 2014:3320-3328 |
31 | Fong RC, Vedaldi A. Interpretable explanations of black boxes by meaningful perturbation. IEEE International Conference on Computer Vision (ICCV). 2017:3449-3457 |
32 | Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Volume 8689 LNCS, No. PART 1. Springer Verlag. 2014:818-833 |
33 | Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology 2018;286:676-684 DOI |
34 | Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Med Phys 2017;44:1408-1419 DOI |
35 | Xiang L, Wang Q, Nie D, et al. Deep embedding convolutional neural network for synthesizing CT image from T1-Weighted MR image. Med Image Anal 2018;47:31-44 DOI |
36 | Yang G, Yu S, Dong H, et al. DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging 2018;37:1310-1321 DOI |
37 | Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. SENSE: sensitivity encoding for fast MRI. Magn Reson Med 1999;42:952-962 DOI |
38 | Knoll F, Hammernik K, Kobler E, Pock T, Recht MP, Sodickson DK. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn Reson Med 2019;81:116-128 DOI |
39 | Akcakaya M, Moeller S, Weingartner S, Ugurbil K. Scanspecific robust artificial-neural-networks for k-space interpolation (RAKI) reconstruction: Database-free deep learning for fast imaging. Magn Reson Med 2019;81:439-453 DOI |
40 | Wolterink JM, Dinkla AM, Savenije MHF, Seevinck PR, van den Berg CAT, Isgum I. Deep MR to CT synthesis using unpaired data. International Workshop on Simulation and Synthesis in Medical Imaging. Springer, Cham, 2017:14-23 |
41 | Quan TM, Nguyen-Duc T, Jeong WK. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 2018;37:1488-1497 DOI |
42 | Mardani M, Gong E, Cheng JY, et al. Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans Med Imaging 2019;38:167-179 DOI |
43 | Hyun CM, Kim HP, Lee SM, Lee S, Seo JK. Deep learning for undersampled MRI reconstruction. Phys Med Biol 2018;63:135007 DOI |
44 | Eo T, Jun Y, Kim T, Jang J, Lee HJ, Hwang D. KIKInet: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn Reson Med 2018;80:2188-2201 DOI |
45 | Aggarwal HK, Mani MP, Jacob M. MoDL: Model-based deep learning architecture for inverse problems. IEEE Trans Med Imaging 2019;38:394-405 DOI |
46 | Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-assisted Invervention. 2015:234-241 |
47 | Jun Y, Eo T, Kim T, et al. Deep-learned 3D black-blood imaging using automatic labelling technique and 3D convolutional neural networks for detecting metastatic brain tumors. Sci Rep 2018;8:9450 DOI |
48 | Gong E, Pauly JM, Wintermark M, Zaharchuk G. Deep learning enables reduced gadolinium dose for contrast-enhanced brain MRI. J Magn Reson Imaging 2018;48:330-340 DOI |
49 | Ryu K, Shin NY, Kim DH, Nam Y. Synthesizing T1 weighted MPRAGE image from multi echo GRE images via deep neural network. Magn Reson Imaging 2019 [Epub ahead of print] |
50 | Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Springer, Cham, 2015:3431-3440 |
51 | Yang Q, Yan P, Zhang Y, et al. Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss. IEEE Trans Med Imaging 2018;37:1348-1357 DOI |
52 | Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative adversarial networks. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2017-20th International Conference, Proceedings. Springer Verlag, 2017:417-425 |
53 | Fischl B. FreeSurfer. Neuroimage 2012;62:774-781 DOI |
54 | Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM. Fsl. Neuroimage 2012;62:782-790 DOI |
55 | Zoran D, Weiss Y. From learning models of natural image patches to whole image restoration. In IEEE International Conference on Computer Vision (ICCV). 2011:479-486 |
56 | Benou A, Veksler R, Friedman A, Riklin Raviv T. Ensemble of expert deep neural networks for spatio-temporal denoising of contrast-enhanced MRI sequences. Med Image Anal 2017;42:145-159 DOI |
57 | Dabov K, Foi A, Katkovnik V, Egiazarian K. Image restoration by sparse 3D transform-domain collaborative filtering. Image Processing: Algorithms and Systems VI. International Society for Optics and Photonics 2008:681207 |
58 | Elad M, Aharon M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 2006;15:3736-3745 DOI |
59 | Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process 2017;26:4509-4522 DOI |
60 | Gu S, Zhang L, Zuo W, Feng X. Weighted nuclear norm minimization with application to image denoising. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014:2862-2869 |
61 | Pereira S, Pinto A, Alves V, Silva CA. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 2016;35:1240-1251 DOI |
62 | Kamnitsas K, Chen L, Ledig C, Rueckert D, Glocker B. Multi-scale 3D convolutional neural networks for lesion segmentation in brain MRI. Ischemic Stroke Lesion Segment 2015;13:46 |
63 | Rajchl M, Pawlowski N, Rueckert D, Matthews PM, Glocker B. NeuroNet: Fast and robust reproduction of multiple brain image segmentation pipelines. arXiv preprint arXiv 2018:1806.04224 |
64 | Havaei M, Davy A, Warde-Farley D, et al. Brain tumor segmentation with deep neural networks. Med Image Anal 2017;35:18-31 DOI |
65 | Akkus Z, Galimzianova A, Hoogi A, Rubin DL, Erickson BJ. Deep learning for brain MRI segmentation: state of the art and future directions. J Digit Imaging 2017;30:449-459 DOI |
66 | Liu S, Zheng H, Feng Y, Li W. Prostate cancer diagnosis using deep learning with 3D multiparametric MRI. SPIE Med Imaging 2017;10134:1-4 |
67 | Song Y, Zhang YD, Yan X, et al. Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric MRI. J Magn Reson Imaging 2018;48:1570-1577 DOI |
68 | Milletari F, Navab N, Ahmadi S-A. V-net: fully convolutional neural networks for volumetric medical image segmentation. In 2016 Fourth International Conference on 3D Vision (3DV). IEEE 2016:565-571 |
69 | Guerrero R, Qin C, Oktay O, et al. White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. Neuroimage Clin 2018;17:918-934 DOI |
70 | Lee D, Yoo J, Tak S, Ye J. Deep residual learning for accelerated mri using magnitude and phase networks. IEEE Trans Biomed Eng 2018;65:1985-1995 DOI |
71 | Kyathanahally SP, Doring A, Kreis R. Deep learning approaches for detection and removal of ghosting artifacts in MR spectroscopy. Magn Reson Med 2018;80:851-863 DOI |
72 | Atkinson D, Hill DL, Stoyle PN, Summers PE, Keevil SF. Automatic correction of motion artifacts in magnetic resonance images using an entropy focus criterion. IEEE Trans Med Imaging 1997;16:903-910 DOI |
73 | Loktyushin A, Nickisch H, Pohmann R, Scholkopf B. Blind retrospective motion correction of MR images. Magn Reson Med 2013;70:1608-1618 DOI |
74 | Ooi MB, Krueger S, Thomas WJ, Swaminathan SV, Brown TR. Prospective real-time correction for arbitrary head motion using active markers. Magn Reson Med 2009;62:943-954 DOI |
75 | Maclaren J, Armstrong BS, Barrows RT, et al. Measurement and correction of microscopic head motion during magnetic resonance imaging of the brain. PLoS One 2012;7:e48088 DOI |
76 | Haskell MW, Cauley SF, Wald LL. TArgeted Motion Estimation and Reduction (TAMER): Data Consistency Based Motion Mitigation for MRI Using a Reduced Model Joint Optimization. IEEE Trans Med Imaging 2018;37:1253-1265 DOI |
77 | Kober T, Marques JP, Gruetter R, Krueger G. Head motion detection using FID navigators. Magn Reson Med 2011;66:135-143 DOI |
78 | Zhou Z, Zhao G, Kijowski R, Liu F. Deep convolutional neural network for segmentation of knee joint anatomy. Magn Reson Med 2018;80:2759-2770 DOI |
79 | Norman B, Pedoia V, Majumdar S. Use of 2D U-net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology 2018;288:177-185 DOI |
80 | Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. Med image Comput Comput Assist Interv 2013;16:246-253 |
81 | Tamada D, Kromrey M-L, Onishi H, Motosugi U. Method for motion artifact reduction using a convolutional neural network for dynamic contrast enhanced MRI of the liver. arXiv preprint arXiv 2018:1807.06956 |
82 | Johnson PM, Drangova M. Motion correction in MRI using deep learning. In Proceeding ISMRM Scientific Meeting & Exhibition. 2018:4098 |
83 | Pawar K, Chen Z, Shah J, Egan GF. Motion correction in MRI using deep convolutional neural network. In Proceeding ISMRM Scientific Meeting & Exhibition. 2018:1174 |
84 | Lee H, Ryu K, Nam Y, Lee J, Kim DH. Reduction of respiratory motion artifact in c-spine imaging using deep learning: Is substitution of navigator possible? In Proceeding ISMRM Scientific Meeting & Exhibition. 2018:2660 |
85 | Meding K, Loktyushin A, Hirsch M. Automatic detection of motion artifacts in MR images using CNNS. 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2017:811-815 |
86 | Pham CH, Ducournau A, Fablet R, Rousseau F. Brain MRI super-resolution using deep 3D convolutional networks. IEEE 14th International Symposium on Biomedical Imaging (ISBI). 2017:197-200 |
87 | Shi J, Liu Q, Wang C, Zhang Q, Ying S, Xu H. Superresolution reconstruction of MR image with a novel residual learning network algorithm. Phys Med Biol 2018;63:085011 DOI |
88 | Chen Y, Xie Y, Zhou Z, Shi F, Christodoulou AG, Li D. Brain MRI super resolution using 3D deep densely connected neural networks. 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). 2018:739-742 |
89 | Sommer K, Brosch T, Rafael W, et al. Correction of motion artifacts using a multi-resolution fully convolutional neural network. In Proceeding ISMRM Scientific Meeting & Exhibition. 2018:1175 |
90 | Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. NIPS 2012: Neural Information Processing Systems, Lake Tahoe, Nevada, 2012:1097-1105 |
91 | Hinton G, Deng I, Yu D, et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Processing Magazine 2012;29:82-97 DOI |
92 | Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Vol 2. 2014:3104-3112 |
93 | Ryu K, Nam Y, Gho SM, et al. Data-driven synthetic MRI FLAIR artifact correction via deep neural network. J Magn Reson Imaging 2019 [Epub ahead of print] |
94 | Jifara W, Jiang F, Rho S, Cheng M, Liu S. Medical image denoising using convolutional neural network: a residual learning approach. J Supercomput 2017:1-15 |
95 | Kim KH, Do WJ, Park SH. Improving resolution of MR images with an adversarial network incorporating images with different contrast. Med Phys 2018;45:3120-3131 DOI |
96 | Chaudhari AS, Fang Z, Kogan F, et al. Super-resolution musculoskeletal MRI using deep learning. Magn Reson Med 2018;80:2139-2154 DOI |
97 | Dong C, Loy CC, He K, Tang X. Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans Pattern Anal Mach Intell 2016;38:295-307 DOI |
98 | Tanenbaum LN, Tsiouris AJ, Johnson AN, et al. Synthetic MRI for Clinical Neuroimaging: Results of the Magnetic Resonance Image Compilation (MAGiC) Prospective, Multicenter, Multireader Trial. AJNR Am J Neuroradiol 2017;38:1103-1110 DOI |
99 | Lee J, Han Y, Ye JC. k-Space Deep Learning for Reference-free EPI Ghost Correction. arXiv preprint arXiv 2018:1806.00153v2 |
100 | Kim KH, Choi SH, Park SH. Improving Arterial Spin Labeling by Using Deep Learning. Radiology 2018;287:658-666 DOI |
101 | Golkov V, Dosovitskiy A, Sperl JI, et al. q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans. IEEE Trans Med Imaging 2016;35:1344-1351 DOI |
102 | Bertleff M, Domsch S, Weingartner S, et al. Diffusion parameter mapping with the combined intravoxel incoherent motion and kurtosis model using artificial neural networks at 3 T. NMR Biomed 2017;30 [Epub ahead of print] |
103 | Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014 |
104 | Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 2010;11:3371-3408 |
105 | Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 2017;26:3142-3155 DOI |
106 | He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:770-778 |
107 | Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 2018;40:834-848 DOI |
108 | Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2015:1520-1528 |
109 | LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436-444 DOI |
110 | Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), 2011. Volume 15 of JMLR: W&CP 15. 2011:315-323 |
111 | LeCun Y, Boser B, Denker JS, et al. Handwritten digit recognition with a back-propagation network. In Advances in neural information processing system 2. 1990:396-404 |
112 | Yoon J, Gong E, Chatnuntawech I, et al. Quantitative susceptibility mapping using deep neural network: QSMnet. Neuroimage 2018;179:199-206 DOI |
113 | Domsch S, Murle B, Weingartner S, Zapp J, Wenz F, Schad LR. Oxygen extraction fraction mapping at 3 Tesla using an artificial neural network: a feasibility study. Magn Reson Med 2018;79:890-899 DOI |
114 | Lee D, Jung W, Lee J, et al. SafeNet: Artificial neural network for real-time T2 mapping with quality assurance. Joint Annual Meeting ISMRM-ESMRMB. ISMRM 2018:2277 |
115 | Ma D, Gulani V, Seiberlich N, et al. Magnetic resonance fingerprinting. Nature 2013;495:187-192 DOI |
116 | Cohen O, Zhu B, Rosen MS. MR fingerprinting Deep RecOnstruction NEtwork (DRONE). Magn Reson Med 2018;80:885-894 DOI |