Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Mid-level Feature Extraction Method Based Transfer Learning to Small-Scale Dataset of Medical Images with Visualizing Analysis

  • Lee, Dong-Ho (Dept. of Computer Science and Engineering, Inha University) ;
  • Li, Yan (Dept. of Computer Science and Engineering, Inha University) ;
  • Shin, Byeong-Seok (Dept. of Computer Science and Engineering, Inha University)
  • Received : 2019.01.29
  • Accepted : 2019.09.22
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In fine-tuning-based transfer learning, the size of the dataset may affect learning accuracy. When a dataset scale is small, fine-tuning-based transfer-learning methods use high computing costs, similar to a large-scale dataset. We propose a mid-level feature extractor that retrains only the mid-level convolutional layers, resulting in increased efficiency and reduced computing costs. This mid-level feature extractor is likely to provide an effective alternative in training a small-scale medical image dataset. The performance of the mid-level feature extractor is compared with the performance of low- and high-level feature extractors, as well as the fine-tuning method. First, the mid-level feature extractor takes a shorter time to converge than other methods do. Second, it shows good accuracy in validation loss evaluation. Third, it obtains an area under the ROC curve (AUC) of 0.87 in an untrained test dataset that is very different from the training dataset. Fourth, it extracts more clear feature maps about shape and part of the chest in the X-ray than fine-tuning method.

Keywords

Acknowledgement

This work was supported by the Institute for Information & Communications Technology Promotion (IITP) grant funded by the Korean government (MSIT) (No. 2017-0-018715, Development of AR-based Surgery Toolkit and Applications).

References

  1. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  2. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Advances in Neural Information Processing Systems, vol. 25, pp. 1097-1105, 2012.
  3. C. Li, M. Liang, W. Song, and K. Xiao, "A multi-scale parallel convolutional neural network based intelligent human identification using face information," Journal of Information Processing Systems, vol. 14, no. 6, pp. 1494-1507, 2018. https://doi.org/10.3745/JIPS.02.0103
  4. S. Zhou and S. Xiao, "3D face recognition: a survey," Human-centric Computing and Information Sciences, vol. 8, article no. 35, 2018.
  5. K. M Koo and E. Y. Cha, "Image recognition performance enhancements using image normalization," Human-centric Computing and Information Sciences, vol. 7, article no. 33, 2017.
  6. A. Sun, Y. Li, Y. M. Huang, Q. Li, and G. Lu, "Facial expression recognition using optimized active regions," Human-centric Computing and Information Sciences, vol. 8, article no. 33, 2018.
  7. J. Zhang, X. Jin, Y. Liu, A. K. Sangaiah, and J. Wang, "Small sample face recognition algorithm based on novel Siamese network," Journal of Information Processing Systems, vol. 14, no. 6, pp. 1464-1479, 2018. https://doi.org/10.3745/JIPS.02.0101
  8. A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, "Dermatologist-level classification of skin cancer with deep neural networks," Nature, vol. 542, no. 7639, pp. 115-118, 2017. https://doi.org/10.1038/nature21056
  9. S. Sarraf, G. Tofighi, and Alzheimer's Disease Neuroimaging Initiative, "DeepAD: Alzheimer's disease classification via deep convolutional neural networks using MRI and fMRI," 2016 [Online]. Available: https://doi.org/10.1101/070441.
  10. R. Chitra and V. Seenivasagam, "Heart disease prediction system using supervised learning classifier," International Journal of Software Engineering and Soft Computing, vol. 3, no. 1, pp. 1-7, 2013. https://doi.org/10.9756/BIJSESC.4336
  11. Y. Bar, I. Diamant, L. Wolf, S. Lieberman, E. Konen, and H. Greenspan, "Chest pathology detection using deep learning with non-medical training," in Proceedings of 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), New York, NY, 2015, pp. 294-297.
  12. H. C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and S. M. Summers, "Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning," IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1285-1298, 2016. https://doi.org/10.1109/TMI.2016.2528162
  13. I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep Learning. Cambridge, MA: MIT Press, 2016.
  14. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, et al., "ImageNet large scale visual recognition challenge," 2015 [Online]. Available: https://arxiv.org/abs/1409.0575
  15. A. Sharif Razavian, H. Azizpour, J. Sullivan, and S. Carlsson, "CNN features off-the-shelf: an astounding baseline for recognition," 2014 [Online]. Available: https://arxiv.org/abs/1403.6382.
  16. M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," in Computer Vision - ECCV 2014. Cham, Switzerland: Springer, 2014, pp. 818-833.
  17. P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan, et al., "CheXnet: radiologist-level pneumonia detection on chest x-rays with deep learning," 2017 [Online]. https://arxiv.org/abs/1711.05225.
  18. T. Tan, Z. Li, H. Liu, F. G. Zanjani, Q. Ouyang, Y. Tang, et al., "Optimize transfer learning for lung diseases in bronchoscopy using a new concept: sequential fine-tuning," 2018 [Online]. Available: https://arxiv.org/abs/1802.03617.
  19. J. M. Carrillo-de-Gea and G. Garcia-Mateos, "Detection of normality/pathology on chest radiographs using LBP," in Proceedings of the 1st International Conference on Bioinformatics, Valencia, Spain, 2010, pp. 167-172.
  20. U. Avni, H. Greenspan, E. Konen, M. Sharon, and J. Goldberger, "X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words," IEEE Transactions on Medical Imaging, vol. 30, no. 3, pp. 733-746, 2011. https://doi.org/10.1109/TMI.2010.2095026
  21. U. Avni, H. Greenspan, and J. Goldberger, "X-ray categorization and spatial localization of chest pathologies," in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2011. Heidelberg: Springer, 2011, pp. 199-206.
  22. J. Ramirez, J. M. Gorriz, D. Salas-Gonzalez, A. Romero, M. Lopez, I. Alvarez, and M. Gomez-Rio, "Computer-aided diagnosis of Alzheimer's type dementia combining support vector machines and discriminant set of features", Information Sciences, vol. 237, pp. 59-72, 2013 https://doi.org/10.1016/j.ins.2009.05.012
  23. M. Oquab, L. Bottou, I. Laptev, and J. Sovic, "Learning and transferring mid-level image representations using convolutional neural networks," in Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 1717-1724.
  24. J. Deng, W. Dong, R. Socher, L. Li, K. Li, and F. Li, "ImageNet: a large-scale hierarchical image database," in Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 2009, pp. 248-255.
  25. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, "The pascal visual object classes (VOC) challenge," International Journal of Computer Vision, vol. 88, no. 2, pp. 303-338, 2010. https://doi.org/10.1007/s11263-009-0275-4
  26. A. Oliva and A. Torralba, "Modeling the shape of the scene: a holistic representation of the spatial envelope," International Journal of Computer Vision, vol. 42, no. 3, pp. 145-175, 2001. https://doi.org/10.1023/A:1011139631724
  27. G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," in Proceedings of the 8th European Conference on Computer Vision: Workshop on Statistical Learning in Computer Vision, Prague, Czech Republic, 2004.
  28. G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," 2016 [Online]. Available: https://arxiv.org/abs/1608.06993.
  29. H. R. Roth, L. Lu, J. Liu, J. Yao, A. Seff, K. Cherry, L. Kim, and R. M. Summers, "Improving computeraided detection using convolutional neural networks and random view aggregation" IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1170-1181, 2016. https://doi.org/10.1109/TMI.2015.2482920
  30. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, 2015, pp. 1-9.
  31. Z. Qin, F. Yu, C. Liu, and X. Chen, "How convolutional neural network see the world: a survey of convolutional neural network visualization methods," Mathematical Foundations of Computing, vol. 1, no. 2, pp. 149-180, 2018. https://doi.org/10.3934/mfc.2018008
  32. W. Yu, K. Yang, Y. Bai, T. Xiao, H. Yao, and Y. Rui, "Visualizing and comparing AlexNet and VGG using deconvolutional layers," in Proceedings of the 33rd International Conference on Machine Learning, New York, NY, 2016.
  33. S. Jaeger, S. Candemir, S. Antani, Y. X. J. Wang, P. X. Lu, and G. Thoma, "Two public chest X-ray datasets for computer-aided screening of pulmonary diseases," Quantitative Imaging in Medicine and Surgery, vol. 4, no. 6, pp. 475-477, 2014.
  34. O. Ronneberger, P. Fischer, and T. Brox, "U-net: convolutional networks for biomedical image segmentation," in Medical Image Computing And Computer-Assisted Intervention - MICCAI 2015. Cham, Switzerland: Springer, 2015, pp. 234-241.
  35. V. Badrinarayanan, A. Handa, and R. Cipolla, "SegNet: a deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling," 2015 [Online]. Available: https://arxiv.org/abs/1505.07293.