Proceedings of the Korea Information Processing Society Conference (한국정보처리학회:학술대회논문집)

  • 2024.05a
  • /
  • Pages.711-714
  • /
  • 2024
  • /
  • 2005-0011(pISSN)
  • /
  • 2671-7298(eISSN)
Korea Information Processing Society (한국정보처리학회)
권호 표지

A Survey on Privacy Vulnerabilities through Logit Inversion in Distillation-based Federated Learning

증류 기반 연합 학습에서 로짓 역전을 통한 개인 정보 취약성에 관한 연구

  • Subin Yun (Dept. of Electrical and Computer Engineering and Inter-University Semiconductor Research Center, Seoul National University) ;
  • Yungi Cho (Dept. of Electrical and Computer Engineering and Inter-University Semiconductor Research Center, Seoul National University) ;
  • Yunheung Paek (Dept. of Electrical and Computer Engineering and Inter-University Semiconductor Research Center, Seoul National University)
  • 윤수빈 (서울대학교 전기정보공학부, 반도체공동연구소) ;
  • 조윤기 (서울대학교 전기정보공학부, 반도체공동연구소) ;
  • 백윤흥 (서울대학교 전기정보공학부, 반도체공동연구소)
  • Published : 2024.05.23
Download PDF
⟨ Previous Next ⟩

Abstract

In the dynamic landscape of modern machine learning, Federated Learning (FL) has emerged as a compelling paradigm designed to enhance privacy by enabling participants to collaboratively train models without sharing their private data. Specifically, Distillation-based Federated Learning, like Federated Learning with Model Distillation (FedMD), Federated Gradient Encryption and Model Sharing (FedGEMS), and Differentially Secure Federated Learning (DS-FL), has arisen as a novel approach aimed at addressing Non-IID data challenges by leveraging Federated Learning. These methods refine the standard FL framework by distilling insights from public dataset predictions, securing data transmissions through gradient encryption, and applying differential privacy to mask individual contributions. Despite these innovations, our survey identifies persistent vulnerabilities, particularly concerning the susceptibility to logit inversion attacks where malicious actors could reconstruct private data from shared public predictions. This exploration reveals that even advanced Distillation-based Federated Learning systems harbor significant privacy risks, challenging the prevailing assumptions about their security and underscoring the need for continued advancements in secure Federated Learning methodologies.

Keywords

Acknowledgement

This work was supported by the BK21 FOUR program of the Education and Research Program for Future ICT Pioneers, Seoul National University in 2024. This work was supported by Inter-University Semiconductor Research Center (ISRC). This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2023-00277326). This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2022-0-00516, Derivation of a Differential Privacy Concept Applicable to National Statistics Data While Guaranteeing the Utility of Statistical Analysis). This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) under the artificial intelligence semiconductor support program to nurture the best talents (IITP-2023-RS-2023-00256081) grant funded by the Korea government (MSIT).

References

  1. Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581, 2019.
  2. Sijie Cheng, Jingwen Wu, Yanghua Xiao, and Yang Liu. Fedgems: Federated learning of larger server models via selective knowledge fusion. arXiv preprint arXiv:2110.11027, 2021.
  3. Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, and Koji Yamamoto. Distillation-based semisupervised federated learning for communication efficient collaborative training with non-iid private data. IEEE Transactions on Mobile Computing, pages 1-1, 2021.
  4. Takahashi, H.; Liu, J.; and Liu, Y. 2023. Breaching FedMD: Image Recovery via Paired-Logits Inversion Attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12198-12207.
  5. Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS '19, page 225-240, New York, NY, USA, 2019. Association for Computing Machinery.