Browse > Article
http://dx.doi.org/10.13104/imri.2020.24.4.179

A Comparative Study of Unsupervised Deep Learning Methods for MRI Reconstruction  

He, Zhuonan (Department of Electronic Information Engineering, Nanchang University)
Quan, Cong (Department of Electronic Information Engineering, Nanchang University)
Wang, Siyuan (Department of Electronic Information Engineering, Nanchang University)
Zhu, Yuanzheng (Department of Electronic Information Engineering, Nanchang University)
Zhang, Minghui (Department of Electronic Information Engineering, Nanchang University)
Zhu, Yanjie (Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences)
Liu, Qiegen (Department of Electronic Information Engineering, Nanchang University)
Publication Information
Investigative Magnetic Resonance Imaging / v.24, no.4, 2020 , pp. 179-195 More about this Journal
Abstract
Recently, unsupervised deep learning methods have shown great potential in image processing. Compared with a large-amount demand for paired training data of supervised methods with a specific task, unsupervised methods can learn a universal and explicit prior information on data distribution and integrate it into the reconstruction process. Therefore, it can be used in various image reconstruction environments without showing degraded performance. The importance of unsupervised learning in MRI reconstruction appears to be growing. Nevertheless, the establishment of prior formulation in unsupervised deep learning varies a lot depending on mathematical approximation and network architectures. In this work, we summarized basic concepts of unsupervised deep learning comprehensively and compared performances of several state-of-the-art unsupervised learning methods for MRI reconstruction.
Keywords
MRI reconstruction; Unsupervised deep learning; Prior information; Data distribution;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Tezcan KC, Baumgartner CF, Luechinger R, Pruessmann KP, Konukoglu E. MR image reconstruction using deep density Priors. IEEE Trans Med Imaging 2019;38:1633-1642   DOI
2 Liu Q, Leung H. Synthesis-analysis deconvolutional network for compressed sensing. 2017 IEEE International Conference on Image Processing (ICIP), 2017:1940-1944
3 Mardani M, Monajemi H, Papyan V, Vasanawala S, Donoho D, Pauly J. Recurrent generative adversarial networks for proximal learning and automated compressive image recovery. arXiv preprint arXiv 2017:1711.10046
4 Hussein S, Kandel P, Bolan CW, Wallace MB, Bagci U. Lung and Pancreatic Tumor Characterization in the deep learning era: novel supervised and unsupervised learning approaches. IEEE Trans Med Imaging 2019;38:1777-1787   DOI
5 Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, 2009
6 Rifai S, Vincent P, Muller X, Glorot X, Bengio Y. Contractive auto-encoders: explicit invariance during feature extraction. In Proceeding International Conference on Machine Learning (ICML), 2011:833-840
7 Bengio Y. Deep learning of representations: looking forward. International Conference on Statistical Language and Speech Processing, 2013:1-37
8 Yuan Y, Liu S, Zhang J, et al. Unsupervised image superresolution using cycle-in-cycle generative adversarial networks. Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018:701-710
9 Erhan D, Courville A, Bengio Y, Vincent P. Why does unsupervised pre-training help deep learning- JMLR Workshop and Conference Proceedings, 2010:201-208
10 Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 2010;11:3371-3408
11 Lugmayr A, Danelljan M, Timofte R. Unsupervised learning for real-world super-resolution. IEEE/CVF International Conference on Computer Vision Workshop (ICCV Workshops), 2019:3408-3416
12 Quan TM, Nguyen-Duc T, Jeong WK. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging 2018;37:1488-1497   DOI
13 Dilokthanakul N, Mediano PAM, Garnelo M, et al. Deep unsupervised clustering with gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648, 2016
14 Sutskever I, Jozefowicz R, Gregor K, Rezende D, Lillicrap T, Vinyals O. Towards principled unsupervised learning. arXiv preprint arXiv:1511.06440, 2015
15 Kingma DP, Dhariwal P. Glow: Generative flow with invertible 1x1 convolution. Advances in Neural Information Processing Systems 31 (NeurIPS 2018), 2018:10215-10224
16 Heidemann RM, Ozsarlak O, Parizel PM, et al. A brief review of parallel magnetic resonance imaging. Eur Radiol 2003;13:2323-2337   DOI
17 Wang S, Su Z, Ying L, et al. Accelerating magnetic resonance imaging via deep learning. Proc IEEE Int Symp Biomed Imaging, 2016:514-517
18 Huang J, Zhang S, Metaxas D. Efficient MR image reconstruction for compressed MR imaging. Med Image Anal 2011;15:670-679   DOI
19 Dong W, Shi G, Li X, Ma Y, Huang F. Compressive sensing via nonlocal low-rank regularization. IEEE Trans Image Process 2014;23:3618-3632   DOI
20 Ravishankar S, Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging 2011;30:1028-1041   DOI
21 Lee D, Yoo J, Ye JC. Deep residual learning for compressed sensing MRI. Proc IEEE Int Symp Biomed Imaging, 2017:15-18
22 Aggarwal HK, Mani MP, Jacob M. MoDL: model-based deep learning architecture for inverse problems. IEEE Trans Med Imaging 2019;38:394-405   DOI
23 Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv: 1511.06434, 2015
24 Lin FH, Wang FN, Ahlfors SP, Hamalainen MS, Belliveau JW. Parallel MRI reconstruction using variance partitioning regularization. Magn Reson Med 2007;58:735-744   DOI
25 Qu X, Hou Y, Lam F, Guo D, Zhong J, Chen Z. Magnetic resonance image reconstruction from undersampled measurements using a patch-based nonlocal operator. Med Image Anal 2014;18:843-856   DOI
26 He J, Liu Q, Christodoulou AG, Ma C, Lam F, Liang ZP. Accelerated High-Dimensional MR Imaging With Sparse Sampling Using Low-Rank Tensors. IEEE Trans Med Imaging 2016;35:2119-2129   DOI
27 Larochelle H, Murray I. The neural autoregressive distribution estimator. Proceeding International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, 2011:29-37
28 Mardani M, Gong E, Cheng JY, et al. Deep generative adversarial networks for compressed sensing automates MRI. arXiv preprint arXiv:1706.00051, 2017
29 Kingma DP, Welling M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013
30 Bishop CM. Pattern recognition and machine learning. New York, NY: Springer, 2006:738
31 Dinh L, Krueger D, Bengio Y. Nice: non-linear independent components estimation. arXiv preprint arXiv:1410.8516, 2014
32 Ma X, Kong X, Zhang S, Hovy E. MaCow: masked convolutional generative flow. Advances in Neural Information Processing Systems 32 (NeurIPS 2019), 2019:5893-5902
33 Alain G, Bengio Y. What regularized auto-encoders learn from the data-generating distribution. J Mach Learn Res 2014;15:3743-3773
34 Bigdeli SA, Zwicker M. Image restoration using autoencoding priors. arXiv preprint arXiv:1703.09964, 2017
35 Zhang M, Li M, Zhou J, et al. High-dimensional embedding network derived prior for compressive sensing MRI reconstruction. Med Image Anal 2020;64:101717   DOI
36 Mardani M, Gong E, Cheng JY, et al. Deep generative adversarial neural networks for compressive sensing MRI. IEEE Trans Med Imaging 2019;38:167-179   DOI
37 Luo G, Zhao N, Jiang W, Hui ES, Cao P. MRI reconstruction using deep Bayesian estimation. Magn Reson Med 2020;84:2246-2261   DOI
38 Lee H, Grosse R, Ranganath R, Ng AY. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceeding International Conference on Machine Learning (ICML), 2009:609-616
39 Antun V, Renna F, Poon C, Adcock B, Hansen AC. On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc Natl Acad Sci U S A & 2020;117:30088-30095   DOI
40 Vincent P, Larochelle H, Bengio Y, Manzagol P. Extracting and composing robust features with denoising autoencoders. In Proceedings of International Conference on Machine Learning, 2008:1096-1103
41 Bengio Y, Courville AC, Vincent P. Unsupervised feature learning and deep learning: a review and new perspectives. CoRR 2012;abs/1206.5538
42 Bengio Y, LeCun Y. Scaling learning algorithms towards AI. Large-scale Kernel Machines 2007;34:1-41
43 Rolfe JT. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200, 2016
44 Kipf TN, Welling M. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016
45 Yi Z, Zhang H, Tan P, Gong M. Dualgan: Unsupervised dual learning for image-to-image translation. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017:2849-2857
46 Uecker M, Hohage T, Block KT, Frahm J. Image reconstruction by regularized nonlinear inversion-joint estimation of coil sensitivities and image content. Magn Reson Med 2008;60:674-682   DOI
47 Akcakaya M, Nam S, Hu P, et al. Compressed sensing with wavelet domain dependencies for coronary MRI: a retrospective study. IEEE Trans Med Imaging 2011;30:1090-1099   DOI
48 Song Y, Ermon S. Generative modeling by estimating gradients of the data distribution. 33rd Conference on Neural Information Processing Systems, 2019:11918-11930
49 Oord A, Kalchbrenner N, Kavukcuoglu K. Pixel recurrent neural networks. arXiv preprint arXiv:1601.06759, 2016
50 Lustig M, Donoho DL, Santos JM, Pauly JM. Compressed sensing MRI. IEEE Signal Proc Mag 2008;25:72-82   DOI
51 Liu Q, Wang S, Yang K, Luo J, Zhu Y, Liang D. Highly undersampled magnetic resonance image reconstruction using two-level Bregman method with dictionary updating. IEEE Trans Med Imaging 2013;32:1290-1301   DOI
52 Ye JC, Han Y, Cha E. Deep convolutional framelets: a general deep learning framework for inverse problems. SIAM J Imaging Sci 2018;11:991-1048   DOI
53 Nguyen A, Clune J, Bengio Y, Dosovitskiy A, Yosinski J. Plug & play generative networks: conditional iterative generation of images in latent space. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017:4467-4477
54 Xiong J, Liu Q, Wang Y, Xu X. A two-stage convolutional sparse prior model for image restoration. J Vis Commun Image R 2017;48:268-280   DOI
55 Hammernik K, Knoll F, Sodickson D, Pock T. Learning a variational model for compressed sensing MRI reconstruction. Proceedings of the International Society of Magnetic Resonance in Medicine (ISMRM), 2016:1088
56 Bigdeli SA, Zwicker M, Favaro P, Jin M. Deep meanshift priors for image restoration. Advances in Neural Information Processing Systems 30 (NIPS 2017), 2017:763-772
57 Rezende DJ, Mohamed S, Wierstra D. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082, 2014
58 Salimans T, Karpathy A, Chen X, Kingma DP. Pixelcnn++: improving the PixelCNN with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, 2017
59 Lustig M, Donoho DL, Santos JM, Pauly JM. Compressed sensing MRI. IEEE Signal Proc Mag 2008;25:72-82   DOI
60 Donoho DL. Compressed sensing. IEEE Trans Inf Theory 2006;52:1289-1306   DOI
61 Dinh L, Sohl-Dickstein J, Bengio S. Density estimation using real NVP. arXiv preprint arXiv:1605.08803, 2016
62 Hoogeboom E, Berg R, Welling M. Emerging convolutions for generative normalizing flows. arXiv preprint arXiv:1901.11137, 2019
63 Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Advances in neural information processing systems 27 (NIPS 2014), 2014:2672-2680
64 Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention, 2015:234-241
65 Grover A, Dhar M, Ermon S. Flow-GAN: combining maximum likelihood and adversarial learning in generative models. arXiv preprint arXiv:1705.08868, 2017
66 Wang C, Macnaught G, Papanastasiou G, ManGillicray T, Newby D. Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks. International Workshop on Simulation and Synthesis in Medical Imaging, 2018:52-60
67 Lin FH, Kwong KK, Belliveau JW, Wald LL. Parallel imaging reconstruction using automatic regularization. Magn Reson Med 2004;51:559-567   DOI
68 Eksioglu EM. Decoupled algorithm for MRI reconstruction using nonlocal block matching model: BM3D-MRI. J Math Imaging Vis 2016;56:430-440   DOI
69 Huang Y, Paisley J, Lin Q, Ding X, Fu X, Zhang XP. Bayesian nonparametric dictionary learning for compressed sensing MRI. IEEE Trans Image Process 2014;23:5007-5019   DOI
70 Liang D, Liu B, Wang J, Ying L. Accelerating SENSE using compressed sensing. Magn Reson Med 2009;62:1574-1584   DOI
71 Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D. A deep cascade of convolutional neural networks for MR image reconstruction. IMPI 2017: Information Processing in Medical Imaging, 2017:647-658
72 Liu Q, Yang Q, Cheng H, Wang S, Zhang M, Liang D. Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors. Magn Reson Med 2020;83:322-336   DOI