Browse > Article
http://dx.doi.org/10.9717/kmms.2020.23.3.468

Three-Dimensional Visualization of Medical Image using Image Segmentation Algorithm based on Deep Learning  

Lim, SangHeon (Dept. of Biomedical Engineering, College of Medicine, Gachon University)
Kim, YoungJae (Dept. of Biomedical Engineering, College of Medicine, Gachon University)
Kim, Kwang Gi (Dept. of Biomedical Engineering, College of Medicine, Gachon University)
Publication Information
Journal of Korea Multimedia Society / v.23, no.3, 2020 , pp. 468-475 More about this Journal
Abstract
In this paper, we proposed a three-dimensional visualization system for medical images in augmented reality based on deep learning. In the proposed system, the artificial neural network model performed fully automatic segmentation of the region of lung and pulmonary nodule from chest CT images. After applying the three-dimensional volume rendering method to the segmented images, it was visualized in augmented reality devices. As a result of the experiment, when nodules were present in the region of lung, it could be easily distinguished with the naked eye. Also, the location and shape of the lesions were intuitively confirmed. The evaluation was accomplished by comparing automated segmentation results of the test dataset to the manual segmented image. Through the evaluation of the segmentation model, we obtained the region of lung DSC (Dice Similarity Coefficient) of 98.77%, precision of 98.45%, recall of 99.10%. And the region of pulmonary nodule DSC of 91.88%, precision of 93.05%, recall of 90.94%. If this proposed system will be applied in medical fields such as medical practice and medical education, it is expected that it can contribute to custom organ modeling, lesion analysis, and surgical education and training of patients.
Keywords
Augmented Reality; Deep Neural Network; Medical Training; Virtual Surgery Simulation;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Y. Balagurunathan, V. Kumar, Y. Gu, J. Kim, H. Wang, Y. Liu, et al., "Test-reset Reproducibility Analysis of Lung CT Image Features," Journal of Digital Imaging, Vol. 27, No. 6, pp. 805-823, 2014.   DOI
2 A.J. Buckler, P.D. Mozley, L. Schwartz, N. Petrick, M. Mcnitt-Gray, C. Fenimore, et al., "Volumetric CT in Lung Cancer: An Example for the Qualification of Imaging as a Biomarker," Academic Radiology, Vol. 17, No. 1, pp. 107-115, 2010.   DOI
3 O. Ronneberger, P. Fischer, and T. Brox, "UNet: Convolutional Networks for Biomedical Image Segmentation," Proceeding of International Conference on Medical Image Computer-assisted Intervention, Vol. 9351, pp. 234-241, 2015.
4 S.H. Lim, H.S. Choi, H.J. Bae, S.K. Jung, J.K. Jung, M.S. Lee, et al., "Multi-class Whole Heart Segmentation Using Residual Multi-dilated Convolution U-Net," Proceeding of The Spring Conference of the Korea Information Processing Society, pp. 508-510, 2019.
5 F. Yu, V. Koltun, and T. Funkhouser, "Dilated Residual Networks," Proceeding of The IEEE Conference on Computer Vision and Pattern Recognition, pp. 472-480, 2017.
6 M. Liang and X. Hu, "Recurrent Convolutional Neural Network for Object Recognition," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3367-3375, 2015.
7 C.H. Sudre, W. Li, T. Vercauteren, S. Ourselin, and M.J. Cardoso, "Generaliseed Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations," Proceeding of Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pp. 240-248, 2017.
8 D.P. Kingma and L.J. Ba, "Adam: A Method for Stochastic Optimization," Proceeding of International Conference on Learning Representations, pp. 1-15, 2015.
9 R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, B. Maclntyre, et al., "Recent Advances in Augmented Reality," IEEE Computer Graphics and Application, Vol. 21, No. 6, pp. 34-47, 2001.   DOI
10 E.Z. Barsom, M. Graafland, and M.P. Schijven, "Systematic Review on the Effectiveness of Augmented Reality Applications in Medical Training," Surgical Endoscopy, Vol. 30, No. 10, pp. 4174-4183, 2016.   DOI
11 S. Avinash, K. Manjunath, and S.S. Kumar, "An Improved Image Processing Analysis for the Detection of Lung Caner Using Gabor Filters and Watershed Segmentation Technique," Proceeding of International Conference on Inventive Computation Technologies, pp. 1-6, 2016.
12 C. Hansen, J. Wieferich, F. Ritter, and H. Peitgen, "Illustrative Visualization of 3D Planning Models for Augmented Reality in Liver Surgery," International Journal of Computer Assisted Radiology and Surgery, Vol. 5, No. 2, pp. 133-141, 2009.   DOI
13 J. Jang, C.M. Tschabrunn, M. Barkagan, E. Anter, B. Menze, R. Nezafat, et al., "Three-dimensional Holographic Visualization of High-Resolution Myocardial Scar on Hololens," Plos One, 2018.
14 S.Y. Kwon, Y.J. Kim, and K.G. Kim, "An Automatic Breast Mass Segmentation Based on Deep Learning on Mammogram," Journal of Korea Multimedia Society, Vol. 21, No. 12, pp. 1535-1536, 2018.
15 S.J. Park, Y.J. Kim, D.Y. Park, and J.W. Jung, "Evaluation of Transfer Learning in Gastroscopy Image Classfication Using Convolutional Neural Network," Journal of Korea Society of Medical and Biological Engineering, Vol. 39, No. 5, pp. 213-219, 2018.
16 D.J. Withey and Z.J. Koles, "Medical Image Segmentation: Methods and Software," Proceeding of Joint Meeting of the 6th International Symposium on Noninvasive Functional Source Imaging of the Brain and Heart and the International Conference on Functional Biomedical Imaging, pp. 140-143, 2007.
17 A.K. Singh, V. Saini, and L.M. Saini, "ROI based Detection of Abnormalities in Lungs Using Medical Image Processing," Proceeding of International Conference on Inventive Computation Technologies, International Conference on Intelligent Computing and Control System, pp. 876-880, 2017.
18 S. Sivakumar and C. Chandrasekar, "Lung Nodule Detection Using Fuzzy Clustering and Support Vector Machines," International Journal of Engineering and Technology, Vol. 5, No. 1, pp. 179-185, 2013.
19 D. Shen, G. Wu, and H.I. Suk, "Deep Learning in Medical Image Analysis," Annual Review of Biomedical Engineering, Vol. 19, No. 9, pp. 221-248, 2017.   DOI