Browse > Article
http://dx.doi.org/10.3837/tiis.2020.03.011

X-ray Image Segmentation using Multi-task Learning  

Park, Sejin (Hanyang University - Ansan Campus, Department of Computer Science and Engineering)
Jeong, Woojin (Hanyang University - Ansan Campus, Department of Computer Science and Engineering)
Moon, Young Shik (Hanyang University - Ansan Campus, Department of Computer Science and Engineering)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.14, no.3, 2020 , pp. 1104-1120 More about this Journal
Abstract
The chest X-rays are a common way to diagnose lung cancer or pneumonia. In particular, the finding of a lung nodule is the most important problem in the early detection of lung cancer. Recently, a lot of automatic diagnosis algorithms have been studied to find the lung nodules missed by doctors. The algorithms are typically based on segmentation network like U-Net. However, the occurrence of false positives that similar to lung nodules present outside the lungs can severely degrade performance. In this study, we propose a multi-task learning method that simultaneously learns the lung region and nodule-labeled data based on the prior knowledge that lung nodules exist only in the lung. The proposed method significantly reduces false positives outside the lung and improves the recognition rate of lung nodules to 83.8 F1 score compared to 66.6 F1 score of single task learning with U-net model. The experimental results on the JSRT public dataset demonstrate the effectiveness of the proposed method compared with other baseline methods.
Keywords
image segmentation; convolutional neural network; lung nodule segmentation;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 S. Schalekamp, B. van Ginneken, E. Koedam, "Computer-aided detection improves detection of pulmonary nodules in chest radiographs beyond the support by bone-suppressed images," Radiology, vol. 272, no. 1, pp. 252-261, 2014.   DOI
2 J. Redmon, S. Divvala, R. Girshick, A. Farhadi, "You only look once: Unified, real-time object detection," in Proc. of the IEEE conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
3 B. de Hoop, D. W. De Boo, H. A. Gietema, "Computer-aided detection of lung cancer on chest radiographs: effect on observer performance," Radiology, vol. 257 no. 2, pp. 532-540, 2010.   DOI
4 J. Nam, S. Park, E. Hwang, J. Lee, K. Jin, K. Lim, T. H. Vu, J. Sohn, S. Hwang, J. Goo, "Development and Validation of Deep Learning-based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs," Radiology, vol. 290, no. 1, pp. 218-228, 2018.   DOI
5 A. M. R. Schilham, B. van Ginneken, M. Loog, "A computer-aided diagnosis system for detection of lung nodules in chest radiographs with an evaluation on a public database," Medical Image Analysis, vol. 10, no. 2, pp. 247-258, 2006.   DOI
6 R. C. Hardie, S. K. Rogers, T. Wilson, A. Rogers, "Performance analysis of a new computer aided detection system for identifying lung nodules on chest radiographs," Medical Image Analysis, vol. 12, no. 3, pp. 240-258, 2008.   DOI
7 B. Gupta, M. Tiwari, S. Lamba, "Visibility improvement and mass segmentation of mammogram images using quantile separated histogram equalisation with local contrast enhancement," CAAI Transactions on Intelligence Technology, vol. 4, pp. 73-79, 2019.   DOI
8 A. Knokher, R. Talwar, "Content-based Image Retrieval: Feature Extraction Techniques and Applications," International Journal of Science, Engineering and Technology Research, vol. 3, no. 5, 2014.
9 J. Song, Y. Guo, L. Gao, X. Li, A. Hanjalic, "From Deterministic to Generative: Multimodal Stochastic RNNs for Video Captioning," IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 10, pp. 3047-3058, 2019.   DOI
10 Y. Zhu, Z. Chen, S. Zhao, H. Xie, "ACE-Net: Biomedical Image Segmentation with Augmented Contracting and Expansive Paths," in Proc. of the 22th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 712-720, 2019.
11 S. Lia, Y. Gao, A. Oto, D. Shen, "Representation learning: a unified deep learning framework for automatic prostate MR segmentation," in Proc. of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 254-261, 2013.
12 H. Xie, D. Yang, N. Sun, Z. Chen, Y. Zhang, "Automated pulmonary nodule detection in CT images using deep convolutional neural networks," Pattern Recognition, vol. 85, pp. 109-119, 2019.   DOI
13 J. Song, X. Li, L. Gao, H. T. Shen, "Hierarchical LSTMs with Adaptive Attention for Visual Captioning," IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), pp. 1-1, 2019.
14 B.B. Ogul, P. Kosucu, A. Ozcam, S.D. Kanik, "Lung Nodule Detection in X-Ray Images: A New Feature Set," in Proc. of 6th European Conference of the International Federation for Medical and Biological Engineering, vol. 45, pp. 150-155, 2014.
15 S. Zhai, Z. Cheng, Y. Wei, Z. Liang, Y. Chen, "Compressive sensing ghost imaging object detection using generative adversarial networks," Optical Engineering, vol. 58, no. 1. p. 15, 2019.
16 B. Li, C. Tang, T. Zheng, Z. Lei, "Fully automated extraction of the fringe skeletons in dynamic electronic speckle pattern interferometry using a U-Net convolutional neural network," Optical Engineering, vol. 58, no. 1, pp. 105, 2019.
17 A. Devaraj and B. van Ginneken and A. Nair, D. Baldwin, "Use of Volumetry for Lung Nodule Management: Theory and Practice," Radiology, vol. 284, no. 3, pp. 630-644, 2017.   DOI
18 W. Liu, J. Hu, Z. Li, Z. Zhang, Z. Ma, D. Zhang, "Tongue image segmentation via thresholding and gray projection," KSII Transactions on Internet and Information Systems, vol. 13, no. 2, pp. 945-961, 2019.   DOI
19 O. Ronneberger, P. Fischer, T. Brox, "U-Net: Convolutional Networks for BiomedicalImage Segmentation," Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234-241, 2015.
20 O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," IJCV, vol. 115, no. 3, pp. 211-252, 2015.   DOI
21 R. Caruana, "Multitask Learning," Springer, vol. 28, pp. 95-133, 1998.   DOI
22 A. Kendall, Y. Gal, R. Cipolla, "Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7482-7491, 2018.
23 P. Agrawal, J. Carreira, J. Malik, "Learning to See by Moving," in Proc. of IEEE International Conference on Computer Vision (ICCV), 2015.
24 J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
25 L.C. Chen, G. Papandreou, F. Schroff, H. Adam, "Rethinking atrous convolution for semantic image segmentation," arXiv:1706.05587, 2017.
26 D. P. Kingma, J. Ba, "Adam: A Method for Stochastic Optimization," in Proc. of International Conference on Learning Representations (ICLR), 2015.
27 N. Wang, Y. Peng, Y. Wang, M. Wang, "Skin lesion image segmentation based on adversarial networks," KSII Transactions on Internet and Information Systems, vol. 12, no. 6, pp. 2826-2840, 2018.   DOI
28 I. Misra, A. Shrivastava, A. Gupta, M. Hebert, "Cross-Stitch Networks for Multi-task Learning," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
29 S. Ruder, "An Overview of Multi-Task Learning in Deep Neural Networks," arXiv preprint arXiv:1706.05098, 2017.
30 Y. Xu, Y. Li, Y. Wang, M. Liu, Y. Fan, M. Lai, I. Eric, C. Chang, "Gland instance segmentation using deep multichannel neural networks," IEEE Transactions on Biomedical Engineering, vol. 64, no. 12, pp. 2901-2912, 2017.   DOI
31 Y. W, K. He, "Group normalization," in Proc. of European Conference on Computer Vision (ECCV), 2018.
32 J. Shiraishi, S. Katsuragawa, J. Ikezoe, T. Matsumoto, T. Kobayashi, K. Komatsu, M. Matsui, H. Fujita, Y. Kodera, K. Doi, "Development of a digital image database for chest radiographs with and without a lung nodule: Receiver operating characteristic analysis of radiologists-detection of pulmonary nodules," Americal Journal of Radiology, vol. 174, pp. 71-74, 2000.
33 X. Wang, Y. Peng, L. Lu, "Chestx-ray8 : hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2017.
34 B. van Ginneken, M. B. Stegmann, M. Loog, "Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database," Medical Image Analysis, 2006.
35 L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adan, "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Proc. of the European conference on computer vision (ECCV), pp. 801-818, 2018.
36 H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, "Pyramid scene parsing network," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881-2890, 2017.
37 W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, A. C. Berg, "SSD: Single shot multibox detector," in Proc. of European Conference on Computer Vision, pp. 21-37, 2016.
38 S. Ren, K. He, R. Girshick, J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.   DOI