Browse > Article
http://dx.doi.org/10.14316/pmp.2019.30.2.39

Deep-Learning-Based Molecular Imaging Biomarkers: Toward Data-Driven Theranostics  

Choi, Hongyoon (Department of Nuclear Medicine, Seoul National University Hospital)
Publication Information
Progress in Medical Physics / v.30, no.2, 2019 , pp. 39-48 More about this Journal
Abstract
Deep learning has been applied to various medical data. In particular, current deep learning models exhibit remarkable performance at specific tasks, sometimes offering higher accuracy than that of experts for discriminating specific diseases from medical images. The current status of deep learning applications to molecular imaging can be divided into a few subtypes in terms of their purposes: differential diagnostic classification, enhancement of image acquisition, and image-based quantification. As functional and pathophysiologic information is key to molecular imaging, this review will emphasize the need for accurate biomarker acquisition by deep learning in molecular imaging. Furthermore, this review addresses practical issues that include clinical validation, data distribution, labeling issues, and harmonization to achieve clinically feasible deep learning models. Eventually, deep learning will enhance the role of theranostics, which aims at precision targeting of pathophysiology by maximizing molecular imaging functional information.
Keywords
Deep learning; Molecular imaging; Theranostics; Medical imaging; Imaging biomarker;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Suk H-I, Lee S-W, Shen D, Initiative AsDN. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage. 2014;101:569-582.   DOI
2 Li F, Tran L, Thung K-H, Ji S, Shen D, Li J. A robust deep model for improved classification of AD/MCI patients. IEEE journal of biomedical and health informatics. 2015;19(5):1610-1616.   DOI
3 Choi H, Ha S, Im HJ, Paek SH, Lee DS. Refining diagnosis of Parkinson's disease with deep learning-based interpretation of dopamine transporter imaging. NeuroImage: Clinical. 2017.
4 Martinez-Murcia FJ, Gorriz JM, Ramirez J, Ortiz A. Convolutional Neural Networks for Neuroimaging in Parkinson's Disease: Is Preprocessing Needed? International journal of neural systems. 2018:1850035-1850035.   DOI
5 Kim DH, Wit H, Thurston M. Artificial intelligence in the diagnosis of Parkinson's disease from ioflupane-123 single-photon emission computed tomography dopamine transporter scans using transfer learning. Nuclear medicine communications. 2018;39(10):887-893.   DOI
6 Betancur JA, Hu L-H, Commandeur F, et al. Deep Learning Analysis of Upright-Supine High-Efficiency SPECT Myocardial Perfusion Imaging for Prediction of Obstructive Coronary Artery Disease: A Multicenter Study. Journal of Nuclear Medicine. 2018:jnumed.118.213538.
7 Xu C, Xu L, Gao Z, et al. Direct detection of pixel-level myocardial infarction areas via a deep-learning algorithm. Paper presented at: International Conference on Medical Image Computing and Computer-Assisted Intervention2017.
8 Betancur J, Commandeur F, Motlagh M, et al. Deep learning for prediction of obstructive disease from fast myocardial perfusion SPECT: a multicenter study. JACC: Cardiovascular Imaging. 2018;11(11):1654-1663.   DOI
9 Kim K, Wu D, Gong K, et al. Penalized PET reconstruction using deep learning prior and local linear fitting. IEEE transactions on medical imaging. 2018;37(6):1478-1487.   DOI
10 Gong K, Catana C, Qi J, Li Q. PET Image Reconstruction Using Deep Image Prior. IEEE transactions on medical imaging. 2018.
11 Gong K, Guan J, Kim K, et al. Iterative PET image reconstruction using convolutional neural network representation. IEEE transactions on medical imaging. 2019;38(3):675-685.   DOI
12 Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature. 2018;555(7697):487.   DOI
13 Pfaehler E, De Jong JR, Dierckx RA, van Velden FH, Boellaard R. SMART (SiMulAtion and ReconsTruction) PET: an efficient PET simulation-reconstruction tool. EJNMMI physics. 2018;5(1):16.   DOI
14 Hwang D, Kang SK, Kim KY, et al. Generation of PET attenuation map for whole-body time-of-f light 18F-FDG PET/MRI using a deep neural network trained with simultaneously reconstructed activity and attenuation maps. Journal of Nuclear Medicine. 2019:jnumed. 118.219493.
15 Han X. MR-based synthetic CT generation using a deep convolutional neural network method. Medical physics. 2017;44(4):1408-1419.   DOI
16 Liu F, Jang H, Kijowski R, Bradshaw T, McMillan AB. Deep learning MR imaging-based attenuation correction for PET/MR imaging. Radiology. 2017;286(2):676-684.   DOI
17 Leynes AP, Yang J, Wiesinger F, et al. Direct pseudoCT generation for pelvis PET/MRI attenuation correction using deep convolutional neural networks with multi-parametric MRI: zero echo-time and dixon deep pseudoCT (ZeDD-CT). Journal of Nuclear Medicine. 2017:jnumed.117.198051.
18 Chen KT, Gong E, de Carvalho Macruz FB, et al. Ultra-Low-Dose 18F-Florbetaben Amyloid PET Imaging Using Deep Learning with Multi-Contrast MRI Inputs. Radiology. 2018:180940.
19 Hwang D, Kim KY, Kang SK, et al. Improving the accuracy of simultaneously reconstructed activity and attenuation maps using deep learning. Journal of Nuclear Medicine. 2018;59(10):1624-1629.   DOI
20 Xiang L, Qiao Y, Nie D, et al. Deep auto-context convolutional neural networks for standard-dose PET image estimation from low-dose PET/MRI. Neurocomputing. 2017;267:406-416.   DOI
21 Wang Y, Yu B, Wang L, et al. 3D conditional generative adversarial networks for high-quality PET image estimation at low dose. Neuroimage. 2018;174:550-562.   DOI
22 Wang T, Lei Y, Tang H, et al. A learning-based automatic segmentation and quantification method on left ventricle in gated myocardial perfusion SPECT imaging: A feasibility study. Journal of Nuclear Cardiology. 2019:1-12.
23 Belal SL, Sadik M, Kaboteh R, et al. Deep Learning for Segmentation of 49 Selected Bones in CT Scans: First Step in Automated PET/CT-based 3D Quantification of Skeletal Metastases. European Journal of Radiology. 2019.
24 Chen L, Shen C, Zhou Z, et al. Automatic PET cervical tumor segmentation by combining deep learning and anatomic prior. Physics in medicine and biology. 2019.
25 Zhong Z, Kim Y, Plichta K, et al. Simultaneous cosegmentation of tumors in PET-CT images using deep fully convolutional networks. Medical physics. 2019;46(2):619-633.   DOI
26 Huang B, Chen Z, Wu P-M, et al. Fully Automated Delineation of Gross Tumor Volume for Head and Neck Cancer on PET-CT Using Deep Learning: A Dual-Center Study. Contrast media & molecular imaging. 2018;2018.
27 Samarin A, Burger C, Wollenweber SD, et al. PET/MR imaging of bone lesions-implications for PET quantification from imperfect attenuation correction. European journal of nuclear medicine and molecular imaging. 2012;39(7):1154-1160.   DOI
28 Choi H, Lee DS. Generation of structural MR images from amyloid PET: Application to MR-less quantification. Journal of Nuclear Medicine. 2018;59(7):1111-1117.   DOI
29 Kang SK, Seo S, Shin SA, et al. Adaptive template generation for amyloid PET using a deep learning approach. Human brain mapping. 2018;39(9):3769-3778.   DOI
30 Ding Y, Sohn JH, Kawczynski MG, et al. A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology. 2018:180958.
31 Choi H, Cheon GJ, Kim H-J, et al. Segmentation-based MR attenuation correction including bones also affects quantitation in brain studies: an initial result of 18F-FPCIT PET/MR for patients with parkinsonism. Journal of Nuclear Medicine. 2014;55(10):1617-1622.   DOI
32 Park J, Bae S, Seo S, et al. Measurement of Glomerular Filtration Rate using Quantitative SPECT/CT and Deeplearning-based Kidney Segmentation. Scientific reports. 2019;9(1):4223.   DOI
33 Beam AL, Kohane IS. Translating artificial intelligence into clinical care. Jama. 2016;316(22):2368-2369.   DOI
34 He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nature medicine. 2019;25(1):30.   DOI
35 Saria S, Butte A, Sheikh A. Better medicine through machine learning: What's real, and what's artificial? : Public Library of Science; 2018.
36 Wei Q, Ren Y, Hou R, Shi B, Lo JY, Carin L. Anomaly detection for medical images based on a one-class classification. Paper presented at: Medical Imaging 2018: Computer-Aided Diagnosis2018.
37 Park SH, Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology. 2018;286(3):800-809.   DOI
38 Redelmeier DA, Shafir E. Medical decision making in situations that offer multiple alternatives. Jama. 1995;273(4):302-305.   DOI
39 Gal Y, Ghahramani Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. Paper presented at: international conference on machine learning2016.
40 Schlegl T, Seebock P, Waldstein SM, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. Paper presented at: International Conference on Information Processing in Medical Imaging2017.
41 Choi H, Kang H, Lee DS, Initiative AsDN. Predicting aging of brain metabolic topography using variational autoencoder. Frontiers in aging neuroscience. 2018;10.
42 Le QV, Ranzato MA, Monga R, et al. Building high-level features using large scale unsupervised learning. arXiv preprint arXiv:11126209. 2011.
43 Bengio Y. Deep learning of representations for unsupervised and transfer learning. Paper presented at: Proceedings of ICML Workshop on Unsupervised and Transfer Learning2012.
44 Rasmus A, Berglund M, Honkala M, Valpola H, Raiko T. Semi-supervised learning with ladder networks. Paper presented at: Advances in neural information processing systems2015.
45 Odena A. Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:160601583. 2016.
46 Pan L, Cheng C, Haberkorn U, Dimitrakopoulou-Strauss A. Machine learning-based kinetic modeling: a robust and reproducible solution for quantitative analysis of dynamic PET data. Physics in Medicine & Biology. 2017;62(9):3566.   DOI
47 Choi H, Na KJ. Integrative analysis of imaging and transcriptomic data of the immune landscape associated with tumor metabolism in lung adenocarcinoma: Clinical and prognostic implications. Theranostics. 2018;8(7):1956.   DOI
48 Shin H-C, Lu L, Kim L, Seff A, Yao J, Summers RM. Interleaved text/image deep mining on a very large-scale radiology database. Paper presented at: Proceedings of the IEEE conference on computer vision and pattern recognition2015.
49 Gomez L, Patel Y, Rusinol M, Karatzas D, Jawahar C. Selfsupervised learning of visual features through embedding images into text topic spaces. Paper presented at: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition2017.
50 Landau SM, Breault C, Joshi AD, et al. Amyloid-${\beta}$ imaging with Pittsburgh compound B and florbetapir: comparing radiotracers and quantification methods. Journal of Nuclear Medicine. 2013;54(1):70-77.   DOI
51 Klunk WE, Koeppe RA, Price JC, et al. The Centiloid Project: standardizing quantitative amyloid plaque estimation by PET. Alzheimer's & dementia. 2015;11(1):1-15. e14.   DOI
52 Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. Jama. 2014;311(24):2479-2480.   DOI
53 Ravi D, Wong C, Deligianni F, et al. Deep learning for health informatics. IEEE journal of biomedical and health informatics. 2017;21(1):4-21.   DOI
54 Russakovsky O, Deng J, Su H, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision. 2015;115(3):211-252.   DOI
55 Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Paper presented at: Advances in neural information processing systems2012.
56 Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118.   DOI
57 Rajpurkar P, Irvin J, Ball RL, et al. Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS medicine. 2018;15(11):e1002686.   DOI
58 Dhungel N, Carneiro G, Bradley AP. Automated mass detection in mammograms using cascaded deep learning and random forests. Paper presented at: Digital Image Computing: Techniques and Applications (DICTA), 2015 International Conference on2015.
59 Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Medical image analysis. 2017;42:60-88.   DOI
60 Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284(2):574-582.   DOI
61 Choi H. Deep learning in nuclear medicine and molecular imaging: current perspectives and future directions. Nuclear medicine and molecular imaging. 2018:1-10.
62 Wang H, Zhou Z, Li Y, et al. Comparison of machine learning methods for classifying mediastinal lymph node metastasis of non-small cell lung cancer from 18 F-FDG PET/CT images. EJNMMI research. 2017;7(1):11.   DOI
63 Kirienko M, Sollini M, Silvestri G, et al. Convolutional Neural Networks Promising in Lung Cancer T-Parameter Assessment on Baseline FDG-PET/CT. Contrast Media & Molecular Imaging. 2018;2018.
64 Choi H, Jin KH, Initiative AsDN. Predicting cognitive decline with deep learning of brain metabolism and amyloid imaging. Behavioural brain research. 2018;344:103-109.   DOI
65 Ding Y, Sohn JH, Kawczynski MG, et al. A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain. Radiology. 2018;290(2):456-464.   DOI
66 Liu M, Cheng D, Yan W. Classification of Alzheimer's Disease by Combination of Convolutional and Recurrent Neural Networks Using FDG-PET images. Frontiers in neuroinformatics. 2018;12:35.   DOI
67 Liu S, Liu S, Cai W, et al. Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer's disease. IEEE Transactions on Biomedical Engineering. 2015;62(4):1132-1140.   DOI