Browse > Article
http://dx.doi.org/10.13088/jiis.2022.28.4.027

Self-supervised Meta-learning for the Application of Federated Learning on the Medical Domain  

Kong, Heesan (Department of Computer science and Engineering, Sunkyunkwan University)
Kim, Kwangsu (Department of Computer science and Engineering, Sunkyunkwan University)
Publication Information
Journal of Intelligence and Information Systems / v.28, no.4, 2022 , pp. 27-40 More about this Journal
Abstract
Medical AI, which has lately made significant advances, is playing a vital role, such as assisting clinicians with diagnosis and decision-making. The field of chest X-rays, in particular, is attracting a lot of attention since it is important for accessibility and identification of chest diseases, as well as the current COVID-19 pandemic. However, despite the vast amount of data, there remains a limit to developing an effective AI model due to a lack of labeled data. A research that used federated learning on chest X-ray data to lessen this difficulty has emerged, although it still has the following limitations. 1) It does not consider the problems that may occur in the Non-IID environment. 2) Even in the federated learning environment, there is still a shortage of labeled data of clients. We propose a method to solve the above problems by using the self-supervised learning model as a global model of federated learning. To that aim, we investigate a self-supervised learning methods suited for federated learning using chest X-ray data and demonstrate the benefits of adopting the self-supervised learning model for federated learning.
Keywords
Self-supervised Learning; Federated Learning; Chest X-ray; Medical AI;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, Ronald Summers, ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases, IEEE CVPR, pp. 3462-3471, 2017. NIH Chest X-ray14, https://www.kaggle.com/datasets/nih-chest-xrays/data
2 Zbontar, J., Jing, L., Misra, I., LeCun, Y., & Deny, S. (2021, July). Barlow twins: Self-supervised learning via redundancy reduction. In International Conference on Machine Learning (pp. 12310-12320). PMLR.
3 Doersch, C., Gupta, A., & Efros, A. A. (2015). Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision (pp. 1422-1430).
4 Banerjee, S., Misra, R., Prasad, M., Elmroth, E., & Bhuyan, M. H. (2020, November). Multi-diseases classification from chest-X-ray: A federated deep learning approach. In Australasian Joint Conference on Artificial Intelligence (pp. 3-15). Springer, Cham.
5 Yan, B., Wang, J., Cheng, J., Zhou, Y., Zhang, Y., Yang, Y., ... & Liu, B. (2021, July). Experiments of federated learning for COVID-19 chest X-ray images. In International Conference on Artificial Intelligence and Security (pp. 41-53). Springer, Cham.
6 Chen, X., Fan, H., Girshick, R., & He, K. (2020). Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297.
7 Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020). Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 33, 9912-9924.
8 Wang, J., Liu, Q., Liang, H., Joshi, G., & Poor, H. V. (2020). Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33, 7611-7623.
9 공희산, 박재훈, 김광수. (2021). 의료 데이터의 자기지도학습 적용을 위한 pretext task 분석. 2021 한국정보통신학회 춘계학술대회. 38-40.
10 Baltruschat, I. M., Nickisch, H., Grass, M., Knopp, T., & Saalbach, A. (2019). Comparison of deep learning approaches for multi-label chest X-ray classification. Scientific reports, 9(1), 1-10.   DOI
11 Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020, November). A simple framework for contrastive learning of visual representations. In International conference on machine learning (pp. 1597-1607). PMLR.
12 Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., ... & Ng, A. Y. (2017). Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225.
13 Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., & Ciurea-Ilcus, S. A large chest radiograph dataset with uncertainty labels and expert comparison. In Proc AAAI Conf Artif Intell (No. 33, p. 590). CheXPert. https://stanfordmlgroup.github.io/competitions/chexpert/
14 Komodakis, N., & Gidaris, S. (2018, April). Unsupervised representation learning by predicting image rotations. In International Conference on Learning Representations (ICLR).
15 McMahan, B., Moore, E., Ramage, D., Hampson, S., & y Arcas, B. A. (2017, April). Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics (pp. 1273-1282). PMLR.
16 Guan, Q., Huang, Y., Luo, Y., Liu, P., Xu, M., & Yang, Y. (2021). Discriminative Feature Learning for Thorax Disease Classification in Chest X-ray Images. IEEE Transactions on Image Processing, 30, 2476-2487.   DOI
17 Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4700-4708).
18 Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2020). Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2, 429-450.
19 Grill, J. B., Strub, F., Altche, F., Tallec, C., Richemond, P., Buchatskaya, E., ... & Valko, M. (2020). Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33, 21271-21284.
20 He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
21 Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., ... & Zhao, S. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1-2), 1-210.   DOI
22 Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S., Stich, S., & Suresh, A. T. (2020, November). Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning (pp. 5132-5143). PMLR.
23 Mangal, A., Kalia, S., Rajgopal, H., Rangarajan, K., Namboodiri, V., Banerjee, S., & Arora, C. (2020). CovidAID: COVID-19 detection using chest X-ray. arXiv preprint arXiv:2004.09803.
24 Misra, I., & Maaten, L. V. D. (2020). Self-supervised learning of pretext-invariant representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6707-6717).
25 Noroozi, M., & Favaro, P. (2016, October). Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision (pp. 69-84). Springer, Cham.