Browse > Article
http://dx.doi.org/10.7472/jksii.2022.23.4.73

Efficient distributed consensus optimization based on patterns and groups for federated learning  

Kang, Seung Ju (Department of Information Security, Korea University)
Chun, Ji Young (Department of Bigdata & Information Security, Seoul Cyber University)
Noh, Geontae (Department of Bigdata & Information Security, Seoul Cyber University)
Jeong, Ik Rae (Department of Information Security, Korea University)
Publication Information
Journal of Internet Computing and Services / v.23, no.4, 2022 , pp. 73-85 More about this Journal
Abstract
In the era of the 4th industrial revolution, where automation and connectivity are maximized with artificial intelligence, the importance of data collection and utilization for model update is increasing. In order to create a model using artificial intelligence technology, it is usually necessary to gather data in one place so that it can be updated, but this can infringe users' privacy. In this paper, we introduce federated learning, a distributed machine learning method that can update models in cooperation without directly sharing distributed stored data, and introduce a study to optimize distributed consensus among participants without an existing server. In addition, we propose a pattern and group-based distributed consensus optimization algorithm that uses an algorithm for generating patterns and groups based on the Kirkman Triple System, and performs parallel updates and communication. This algorithm guarantees more privacy than the existing distributed consensus optimization algorithm and reduces the communication time until the model converges.
Keywords
Federated learning; Optimization; Weight model; Communication time; Privacy; ADMM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 N. Rieke, et al, "The future of digital health with federated learning", NPJ digital medicine, Vol 3.1, pp.1-7, 2020. https://www.nature.com/articles/s41746-020-00323-1   DOI
2 T. Li, et al, "Federated learning: Challenges, methods, and future directions", IEEE Signal Processing Magazine, Vol 37.3, pp.50-60, 2020. https://doi.org/10.1109/MSP.2020.2975749   DOI
3 V. Mothukuri, et al, "A survey on security and privacy of federated learning", Future Generation Computer Systems, Vol 115, pp.619-640, 2021. https://doi.org/10.1016/j.future.2020.10.007   DOI
4 S. Fu, C. Xie, B. Li, and Q. Chen, "Attack-resistant federated learning with residual-based reweighting", arXiv:1912.11464, 2019. https://doi.org/10.48550/arXiv.1912.11464   DOI
5 Y. Yu, et al, "Privacy-preserving incremental ADMM for decentralized consensus optimization", IEEE Transactions on Signal Processing, Vol 68, pp.5842-5854, 2020. https://doi.org/10.1109/TSP.2020.3027917   DOI
6 F. Archetti, and F. Schoen, "A survey on the global optimization problem: general theory and computational approaches", Annals of Operations Research, Vol 1.2, pp.87-110, 1984. https://link.springer.com/article/10.1007/BF01876141   DOI
7 G. Falcone, and M. Pavone, "Kirkman's Tetrahedron and the Fifteen Schoolgirl Problem", The American Mathematical Monthly Vol 118.10, pp.887-900, 2011. https://doi.org/10.4169/amer.math.monthly.118.10.887   DOI
8 X. Li, Z. Xu, and W. Chou, "A new method of constructing Kirkman triple system", Chinese Control and Decision Conference, IEEE, pp.4237-4242, 2011. https://doi.org/10.1109/CCDC.2011.5968970   DOI
9 I. Hegedus, G. Danner, M. Jelasity, "Gossip learning as a decentralized alternative to federated learning", IFIP International Conference on Distributed Applications and Interoperable Systems. Springer, Cham, pp.74-90, 2019. https://link.springer.com/chapter/10.1007/978-3-030-22496-7_5   DOI
10 B. Jeon, et al, "Privacy-preserving decentralized aggregation for federated learning", IEEE INFOCOM 2021-IEEE Conference on Computer Communications Workshops, pp.1-6, 2021. https://doi.org/10.1109/INFOCOMWKSHPS51825.2021 .9484437   DOI
11 H. B. McMahan, et al, "Communication-efficient learning of deep networks from decentralized data", Artificial intelligence and statistics. PMLR, pp. 1273-1282, 2017. http://proceedings.mlr.press/v54/mcmahan17a?ref=https://githubhelp.com
12 Noveck, B. Simone, "The single point of failure", Innovating government TMC Asser Press, pp.77-99, 2011.
13 N. Bouacida, and P. Mohapatra, "Vulnerabilities in Federated Learning", IEEE Access, Vol 9, pp.63229-63249, 2021. https://doi.org/10.1109/ACCESS.2021.3075203   DOI
14 H. S. Oluwatosin, "Client-Server Model", IOSR Journal of Computer Engineering, Vol 16.1, pp.67-71, 2014. https://www.researchgate.net/profile/Shakirat-Sulyman/p ublication/271295146_Client-Server_Model/links/5864e1 1308ae8fce490c1b01/Client-Server-Model.pdf   DOI
15 S. Boyd, et al, "Distributed optimization and statistical learning via the alternating direction method of multipliers", Foundations and Trends® in Machine learning, Vol 3.1, pp.1-122, 2011. https://doi.org/10.1561/2200000016   DOI