Browse > Article
http://dx.doi.org/10.3837/tiis.2022.01.002

Boundary-Aware Dual Attention Guided Liver Segment Segmentation Model  

Jia, Xibin (Faculty of Information Technology, Beijing University of Technology)
Qian, Chen (Faculty of Information Technology, Beijing University of Technology)
Yang, Zhenghan (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Xu, Hui (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Han, Xianjun (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Ren, Hao (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Wu, Xinru (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Ma, Boyang (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Yang, Dawei (Department of Radiology, Beijing Friendship Hospital, Capital Medical University)
Min, Hong (Department of Computer Software Engineering, Soonchunhyang University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.1, 2022 , pp. 16-37 More about this Journal
Abstract
Accurate liver segment segmentation based on radiological images is indispensable for the preoperative analysis of liver tumor resection surgery. However, most of the existing segmentation methods are not feasible to be used directly for this task due to the challenge of exact edge prediction with some tiny and slender vessels as its clinical segmentation criterion. To address this problem, we propose a novel deep learning based segmentation model, called Boundary-Aware Dual Attention Liver Segment Segmentation Model (BADA). This model can improve the segmentation accuracy of liver segments with enhancing the edges including the vessels serving as segment boundaries. In our model, the dual gated attention is proposed, which composes of a spatial attention module and a semantic attention module. The spatial attention module enhances the weights of key edge regions by concerning about the salient intensity changes, while the semantic attention amplifies the contribution of filters that can extract more discriminative feature information by weighting the significant convolution channels. Simultaneously, we build a dataset of liver segments including 59 clinic cases with dynamically contrast enhanced MRI(Magnetic Resonance Imaging) of portal vein stage, which annotated by several professional radiologists. Comparing with several state-of-the-art methods and baseline segmentation methods, we achieve the best results on this clinic liver segment segmentation dataset, where Mean Dice, Mean Sensitivity and Mean Positive Predicted Value reach 89.01%, 87.71% and 90.67%, respectively.
Keywords
Segmentation model; liver segment; attention mechanism; boundary-aware;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, X. Tang, "Residual attention network for image classication," in Proc. of the IEEE conference on computer vision and pattern recognition, Honolulu, Hawaii, United States, pp. 3156-3164, 2017.
2 X. Wang, R. Girshick, A. Gupta, K. He, "Non-local neural networks," in Proc. of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, United States, pp. 7794-7803, 2018.
3 J. Hu, L. Shen, G. Sun, "Squeeze-and-excitation networks," in Proc. of the IEEE conference on computer vision and pattern recognition, Salt Lake City, Utah, United States, pp. 7132-7141, 2018.
4 D. Jha, P. H. Smedsrud, M. A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, H. D. Johansen, "Resunet++: An advanced architecture for medical image segmentation," in Proc. of 2019 IEEE International Symposium on Multimedia (ISM), Taichung, China, pp. 225-2255, 2019.
5 K.-L. Tseng, Y.-L. Lin, W. Hsu, C.-Y. Huang, "Joint sequence learning and cross-modality convolution for 3d biomedical segmentation," in Proc. of the IEEE conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, United States, pp. 6393-6400, 2017.
6 X. Jia, Y. Liu, Z. Yang, D. Yang, "Multi-modality self-attention aware deep network for 3d biomedical segmentation," BMC Medical Informatics and Decision Making, vol. 20, no. 3, pp. 1-7, 2020.   DOI
7 O. Ronneberger, P. Fischer, T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Proc. of International Conference on Medical image computing and computer-assisted intervention, Munich, Germany, pp. 234-241, 2015.
8 O. Cicek, A. Abdulkadir, S. S. Lienkamp, T. Brox, O. Ronneberger, "3d u-net: learning dense volumetric segmentation from sparse annotation," in Proc. of International conference on medical image computing and computerassisted intervention, Athens, Greece, pp. 424-432, 2016.
9 N. Ibtehaz, M. S. Rahman, "Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation," Neural Networks, vol. 121, pp. 74-87, 2020.   DOI
10 V. Iglovikov, A. Shvets, "Ternausnet: U-net with vgg11 encoder pre-trained on imagenet for image segmentation," arXiv preprint arXiv:1801.05746, 2018.
11 Z. Zhang, Q. Liu, Y. Wang, "Road extraction by deep residual u-net," IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 749-753, 2018.   DOI
12 Li S, Zhang C, He X, "Shape-aware semi-supervised 3d semantic segmentation for medical images," in Proc. of International Conference on Medical Image Computing and Computer Assisted Intervention, Lima, Peru, pp. 552-561, 2020.
13 O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz, et al., "Attention u-net: Learning where to look for the pancreas," arXiv preprint arXiv:1804.03999, 2018.
14 X. Wang, S. Han, Y. Chen, D. Gao, N. Vasconcelos, "Volumetric attention for 3d medical image segmentation and detection," in Proc. of International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Shenzhen, China, pp. 175-184, 2019.
15 J. Zhang, Z. Jiang, J. Dong, Y. Hou, B. Liu, "Attention gate resu-net for automatic mri brain tumor segmentation," IEEE Access, vol. 8, pp. 58533-58545, 2020.   DOI
16 Z. Zhang, H. Fu, H. Dai, J. Shen, Y. Pang, L. Shao, "Et-net: A generic edge-attention guidance network for medical image segmentation," in Proc. of International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, pp. 442-450, 2019.
17 J. Chen, Y. Lu, Q. Yu, X. Luo, E. Adeli, Y. Wang, L. Lu, A. L. Yuille, Y. Zhou, "Transunet: Transformers make strong encoders for medical image segmentation," arXiv preprint arXiv:2102.04306, 2021.
18 T. He, Z. Zhang, H. Zhang, Z. Zhang, J. Xie, M. Li, "Bag of tricks for image classification with convolutional neural networks," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, California, United States, pp. 558-567, 2019.
19 Y. Liu, X. Jia, Z. Yang, D. Yang, "Style consistency constrained fusion feature learning for liver tumor segmentation," in Proc. of Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Xi'an, China, pp. 390-396, 2019.
20 X. Li, H. Chen, X. Qi, Q. Dou, C.-W. Fu, P.-A. Heng, "H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes," IEEE transactions on medical imaging, vol. 37, no. 12, pp. 2663-2674, 2018.   DOI
21 D. P. Kingma, J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
22 Selvaraju R R, Cogswell M, Das A, et al., "Grad-cam: Visual explanations from deep networks via gradient-based localization," International Journal of Computer Vision, vol. 128, pp. 336-359, 2020.   DOI
23 M. Lafortune, F. Madore, H. Patriquin, G. Breton, "Segmental anatomy of the liver: a sonographic approach to the couinaud nomenclature," Radiology, vol. 181, no. 2, pp. 443-448, 1991.   DOI
24 Zhang Y, Peng C, Peng L, et al., "Multi-phase Liver Tumor Segmentation with Spatial Aggregation and Uncertain Region Inpainting," in Proc. of International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, France, pp. 68-77, 2021.
25 X. Yang, J. Do Yang, H. P. Hwang, H. C. Yu, S. Ahn, B.-W. Kim, H. You, "Segmentation of liver and vessels from CT images and classification of liver segments for preoperative liver surgical planning in living donor liver transplantation," Computer methods and programs in biomedicine, vol. 158, pp. 41-52, 2018.   DOI
26 F. Sutherland, J. Harris, "Claude couinaud: a passion for the liver," Archives of surgery, vol. 137, no. 11, pp. 1305-1310, 2002.   DOI
27 J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, H. Lu, "Dual attention network for scene segmentation," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, California, United States, pp. 3146-3154, 2019.
28 K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition," in Proc. of the IEEE conference on computer vision and pattern recognition, Las Vegas, United States, pp. 770- 778, 2016.
29 D. Selle, B. Preim, A. Schenk, H.-O. Peitgen, "Analysis of vasculature for liver surgical planning," IEEE transactions on medical imaging, vol. 21, no. 11, pp. 1344-1357, 2002.   DOI
30 A. Chartsias, G. Papanastasiou, C. Wang, S. Semple, D. Newby, R. Dharmakumar, S. Tsaftaris, "Disentangle, align and fuse for multimodal and semi-supervised image segmentation," IEEE Transactions on Medical Imaging, vol. 40, no. 3, pp. 781-792, 2021.   DOI
31 Liu Q, Chen C, Qin J, et al., "FedDG: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1013-1023, 2021.
32 R. Beichel, T. Pock, C. Janko, R. B. Zotter, B. Reitinger, A. Bornik, K. Palagyi, E. Sorantin, G. Werkgartner, H. Bischof, et al., "Liver segment approximation in ct data for surgical resection planning," in Medical Imaging 2004: Image Processing, San Diego, California, United States, 2004, pp. 1435-1446.
33 A. Sinha, J. Dolz, "Multi-scale self-guided attention for medical image segmentation," IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 1, pp. 121-130, 2021.   DOI
34 Cai J, Xia Y, Yang D, et al., "End-to-end adversarial shape learning for abdomen organ deep segmentation," in Proc. of International Workshop on Machine Learning in Medical Imaging, Shenzhen, China, pp. 124-132, 2019.
35 G. Noyel, C. Vartin, P. Boyle, L. Kodjikian, "Retinal vessel segmentation by probing adaptive to lighting variations," in Proc. of 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, United States, pp. 1246-1249, 2020.
36 Y. Chen, P. Sun, "The research and practice of medical image enhancement and 3d reconstruction system," in Proc. of 2017 International Conference on Robots & Intelligent System (ICRIS), Huai'an, China, pp. 350-353, 2017.
37 M. Moradi, P. Abolmaesumi, D. R. Siemens, E. E. Sauerbrei, A. H. Boag, P. Mousavi, "Augmenting detection of prostate cancer in transrectal ultrasound images using svm and rf time series," IEEE Transactions on Biomedical Engineering, vol. 56, no. 9, pp. 2214-2224, 2009.   DOI