[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.3837/tiis.2021.02.007

Artificial Intelligence-based Echocardiogram Video Classification by Aggregating Dynamic Information

Ye, Zi (Department of Information Technology, Wenzhou Polytechnic)
Kumar, Yogan J. (Centre for Advanced Computing Technology, Faculty of Information and Communication Technology Universiti Teknikal Malaysia Melaka)
Sing, Goh O. (Centre for Advanced Computing Technology, Faculty of Information and Communication Technology Universiti Teknikal Malaysia Melaka)
Song, Fengyan (Shanghai Gen Cong Information Technology Co. Ltd)
Ni, Xianda (Department of Ultrasonography, the First Affiliated Hospital of Wenzhou Medical University)
Wang, Jin (School of Computer & Communication Engineering, Changsha University of Science & Technology)

Publication Information

KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.2, 2021 , pp. 500-521 More about this Journal

Abstract

Echocardiography, an ultrasound scan of the heart, is regarded as the primary physiological test for heart disease diagnoses. How an echocardiogram is interpreted also relies intensively on the determination of the view. Some of such views are identified as standard views because of the presentation and ease of the evaluations of the major cardiac structures of them. However, finding valid cardiac views has traditionally been time-consuming, and a laborious process because medical imaging is interpreted manually by the specialist. Therefore, this study aims to speed up the diagnosis process and reduce diagnostic error by providing an automated identification of standard cardiac views based on deep learning technology. More importantly, based on a brand-new echocardiogram dataset of the Asian race, our research considers and assesses some new neural network architectures driven by action recognition in video. Finally, the research concludes and verifies that these methods aggregating dynamic information will receive a stronger classification effect.

Keywords

Classification; Deep Learning; Echocardiogram View; LSTM; Two-Stream Network;

Citations & Related Records

Reference

1	M. Long and Y. Zeng, "Detecting Iris Liveness with Batch Normalized ConVolutional Neural Network," Computers, Materials and Continua, vol. 58, no. 2, pp. 493-504, 2019.
2	W. Lu, L. Quan, and L. Ping, "Image Classification using Optimized MKL for SSPM," Intelligent Automation and Soft Computing, vol. 25, no. 2, pp. 249-257, 2019.
3	A. Madani, R. Arnaout, M. Mofrad, and R. Arnaout, "Fast and accurate view classification of echocardiograms using deep learning," NPJ Digital Medicine, vol. 1, no. 1, pp. 1-8, Mar. 2018. DOI
4	J. Zhang, S. Gajjala, P. Agrawal, G. H. Tison, L. A. Hallock, L. Beussink-Nelson, M. H. Lassen, E. Fan, M. A. Aras, C. Jordan, K. E. Fleischmann, M. Melisko, A. Qasim, S. J. Shah, R. Bajcsy, and R. C. Deo, "Fully automated echocardiogram interpretation in clinical practice: feasibility and diagnostic accuracy," Circulation, vol. 138, no. 16, pp. 1623-1635, Sep. 2018. DOI
5	Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015. DOI
6	X. Gao, W. Li, M. Loomes, and L. Wang, "A fused deep learning architecture for viewpoint classification of echocardiography," Inform Fusion, vol. 36, pp. 103-113, July 2017. DOI
7	Z. Xu, Q. Zhou, and Z. Yan, "Special Section on Recent Advances in Artificial Intelligence for Smart Manufacturing - Part I," Intellignet Automaiton and Soft Computing, vol. 25, no. 4, pp. 693-694, 2019.
8	A. A. M. Jamel and B. Akay, "A Survey and systematic categorization of parallel K-means and Fuzzy-c-Means algorithms," Computer Systems Science and Engineering, vol. 34, no. 5, pp. 259-281, Sep. 2019.
9	L. Aguilar, S. W. Nava-Diaz, and G. Chavira, "Implementation of decision trees as an alternative for the support in the decision-making within an intelligent system in order to automatize the regulation of the VOCs in non-industrial inside environments," Computer Systems Science Engineering, vol. 34, no. 5, pp. 297-303, 2019.
10	F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 1251-1258, 2017.
11	H. Chen, D. Ni, J. Qin, S. Li, X. Yang, T. Wang, and P. A. Heng, "Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks," IEEE Journal Biomedical Health Informatics, vol. 19, no. 5, pp. 1627-1636, Sep. 2015. DOI
12	A. M. El Missiri, K. A. L. El Meniawy, S. A. S. Sakr, and A. S. E. D. Mohamed, "Normal reference values of echocardiographic measurements in young Egyptian adults," Egypt Heart Journal, vol. 68, no. 4, pp. 209-215, Dec. 2016. DOI
13	C. Zhang and Y. Tian, "Automatic video description generation via lstm with joint two-stream encoding," in Proc. of the 23rd International Conference on Pattern Recognition(ICPR), pp. 2924-2929, 2016.
14	X. Hong, X. Zheng, J. Xia, L. Wei, and W. Xue, "Cross-Lingual Non-Ferrous Metals Related News Recognition Method Based on CNN with A Limited Bi-Lingual Dictionary," Computers, Materials, and Contiuna, vol. 58, no. 2, pp. 379-389, 2019.
15	S. Y. Tan, "Automated Interpretation of Echocardiograms Technical Milestone Report," to be published.
16	A. Ostvik, E. Smistad, S. A. Aase, B. O. Haugen, and L. Lovstakken, "Real-time standard view classification in transthoracic echocardiography using convolutional neural networks," Ultrasound in Medicine and Biology, vol. 45, no. 2, pp. 374-384, Feb. 2019. DOI
17	J. P. Howard, J. Tan, M. J. Shun-Shin, D. Mahdi, A. N. Nowbar, A. D. Arnold, Y. Ahmad, P. McCartney, M. Zolgharni, N. W. F. Linton, N. Sutaria, B. Rana, J. Mayet, D. Rueckert, G. D. Cole, and D. P. Francis, "Improving ultrasound video classification: an evaluation of novel deep learning methods in echocardiography," Journal of Medical Artificial Intelligence, vol. 3, no. 4, Mar. 2020.
18	S. Ebadollahi, S. F. Chang, and H. Wu, "Automatic view recognition in echocardiogram videos using parts-based representation," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition(CVPR), pp. 2-9, 2004.
19	G. N. Balaji, T. S. Subashini, and N. Chidambaram, "Automatic classification of cardiac views in echocardiogram using histogram and statistical features," Procedia Computer Science, vol. 46, pp.1569-1576, 2015. DOI
20	H. Khamis, G. Zurakhov, V. Azar, A. Raz, Z. Friedman, and D. Adam, "Automatic apical view classification of echocardiograms using a discriminative learning dictionary," Medical Image Analysis, vol. 36, pp. 15-21, Feb. 2017. DOI
21	R. G. Dantas, E. T. Costa, and S. Leeman, "Ultrasound speckle and equivalent scatterers," Ultrasonics, vol. 43, no. 6, pp. 405-420, May 2005. DOI
22	H. Wu, Q. Liu, and X. Liu, "A review on deep learning approaches to Image classification And object segmentation," Computers, Maerial and Continua, vol. 60, no. 2, pp. 575-597, 2019.
23	Z. Liu, B. Xiang, Y. Song, H. Lu, and Q. Liu, "An Improved Unsupervised Image Segmentation Method Based on Multi-Objective Particle, Swarm Optimization Clustering Algorithm," Computers, Materials and Continua, vol. 58, no. 2, pp. 451-461, 2019.
24	K. Simonyan and A. Zisserman, "Two-stream convolutional networks for action recognition in videos," Advances in Neural Information Processing Systems, pp. 568-576, 2014.
25	T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High accuracy optical flow estimation based on a theory for warping," in Proc. of Eupopean Conference on Computer Vision, pp. 25-36, 2004.
26	A. Sengupta, Y. Ye, R. Wang, C. Liu, and K. Roy, "Going deeper in spiking neural networks: Vgg and residual architectures," Fronters in Neuroscienc, vol. 13, Mar. 2019.
27	C. Szegedy, S. Ioffe, V. Vanhoucke, and A. Alemi, "Inception-v4, inception-resnet and the impact of residual connections on learning," arXiv preprint arXiv:1602.07261, Aug. 2016.
28	S. Zhou, L. Chen, and V. Sugumaran, "Hidden two-stream collaborative learning network for action recognition," Computers, Maerial and Continua, vol. 63, no. 3, pp. 1545-1561, 2020.
29	A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, May 2017. DOI
30	D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in Proc. of the 3rd International Conference for Learning Representations, 2015.
31	D. Ouyang, B. He, A. Ghorbani, N. Yuan, J. Ebinger, C. P. Langlotz, P. A. Heidenreich, R. A. Harrington, D. H. Liang, E. A. Ashley, and J. Y. Zou, "Video-based AI for beat-to-beat assessment of cardiac function," Nature, vol. 580, pp. 252-256, Mar. 2020. DOI
32	F. Duran and M. Teke, "Design and Implementation of an Intelligent Ultrasonic Cleaning Device," Intellignet Automaiton and Soft Computing, vol. 25, no. 3, pp. 441-449, 2019.