ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Deep reinforcement learning for base station switching scheme with federated LSTM-based traffic predictions

  • Hyebin Park (Telecommunications & Media Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Seung Hyun Yoon (Telecommunications & Media Research Laboratory, Electronics and Telecommunications Research Institute)
  • Received : 2023.02.21
  • Accepted : 2023.10.12
  • Published : 2024.06.20
Download PDF
⟨ Previous Next ⟩

Abstract

To meet increasing traffic requirements in mobile networks, small base stations (SBSs) are densely deployed, overlapping existing network architecture and increasing system capacity. However, densely deployed SBSs increase energy consumption and interference. Although these problems already exist because of densely deployed SBSs, even more SBSs are needed to meet increasing traffic demands. Hence, base station (BS) switching operations have been used to minimize energy consumption while guaranteeing quality-of-service (QoS) for users. In this study, to optimize energy efficiency, we propose the use of deep reinforcement learning (DRL) to create a BS switching operation strategy with a traffic prediction model. First, a federated long short-term memory (LSTM) model is introduced to predict user traffic demands from user trajectory information. Next, the DRL-based BS switching operation scheme determines the switching operations for the SBSs using the predicted traffic demand. Experimental results confirm that the proposed scheme outperforms existing approaches in terms of energy efficiency, signal-to-interference noise ratio, handover metrics, and prediction performance.

Keywords

Acknowledgement

The Institute of Information & Communications Technology Planning & Evaluation(IITP) grant funded by the Korea government (MSIT) (no. 2021-0-00851, On-demand data based network intelligence framework technology development).

References

  1. Ericsson, Ericsson mobility report, 2020. https://www.ericsson.com/4ac68e/assets/local/reports-papers/mobilityreport/documents/2020/june2020-ericsson-mobility-report.pdf
  2. H. Holtkamp, G. Auer, S. Bazzi, and H. Haas, Minimizing base station power consumption, IEEE J. Sel. Areas Commun. 32 (2014), no. 2, 297-306. https://doi.org/10.1109/JSAC.2014.141210
  3. C. Liu, B. Natarajan, and H. Xia, Small cell base station sleep strategies for energy efficiency, IEEE Trans. Veh. Technol. 65 (2015), no. 3, 1652-1661.
  4. E. Oh and B. Krishnamachari, Energy savings through dynamic base station switching in cellular wireless access networks, (IEEE Global Telecommunications Conference GLOBECOM 2010, Miami, FL, USA), 2010, pp. 1-5.
  5. E. Oh, K. Son, and B. Krishnamachari, Dynamic base station switching-on/off strategies for green cellular networks, IEEE Trans. Wirel. Commun. 12 (2013), no. 5, 2126-2136. https://doi.org/10.1109/TWC.2013.032013.120494
  6. S. Song, Y. Chang, X. Wang, and D. Yang, Coverage and energy modeling of HetNet under base station on-off model, ETRI J. 37 (2015), no. 3, 450-459. https://doi.org/10.4218/etrij.15.0114.0669
  7. A. Kumar and C. Rosenberg, Energy and throughput trade-offs in cellular networks using base station switching, IEEE Trans. Mob. Comput. 15 (2016), no. 2, 364-376. https://doi.org/10.1109/TMC.2015.2416181
  8. M. Feng, S. Mao, and T. Jiang, Base station on-off switching in 5G wireless networks: Approaches and challenges, IEEE Wireless Commun. 24 (2017), no. 4, 46-54.
  9. W.-T. Wong, Y.-J. Yu, and A.-C. Pang, Decentralized energy-efficient base station operation for green cellular networks, (IEEE Global Communications Conference (GLOBECOM), Anaheim, CA, USA), 2012, pp. 5194-5200.
  10. Y. Yang, Z. Liu, H. Zhu, X. Guan, and K. Y. Chan, Energy minimization by dynamic base station switching in heterogeneous cellular network, Wireless Netw. 2022 (2022), 1-16. https://doi.org/10.1186/s13638-021-02080-5
  11. R. Li, Z. Zhao, X. Chen, J. Palicot, and H. Zhang, Tact: a transfer actor-critic learning framework for energy saving in cellular radio access networks, IEEE Trans. Wireless Commun. 13 (2014), no. 4, 2000-2011. https://doi.org/10.1109/TWC.2014.022014.130840
  12. R. S. Sutton and A. G. Barto, Reinforcement learning: an introduction, MIT press, 2018.
  13. A. El-Amine, H. A. Haj Hassan, M. Iturralde, and L. Nuaymi, Location-aware sleep strategy for energy-delay tradeoffs in 5G with reinforcement learning, (IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications, Istanbul, Turkey), 2019, pp. 1-6.
  14. A. El-Amine, M. Iturralde, H. A. Haj Hassan, and L. Nuaymi, A distributed q-learning approach for adaptive sleep modes in 5G networks, (IEEE Wireless Communications and Networking Conference, Marrakesh, Morocco), 2019, pp. 1-6.
  15. A. El Amine, J.-P. Chaiban, H. A. H. Hassan, P. Dini, L. Nuaymi, and R. Achkar, Energy optimization with multisleeping control in 5g heterogeneous networks using reinforcement learning, IEEE Trans. Netw. Service Manag. 2022 (2022), 1-1.
  16. M. Masoudi, M. G. Khafagy, E. Soroush, D. Giacomelli, S. Morosi, and C. Cavdar, Reinforcement learning for traffic-adaptive sleep mode management in 5G networks, (IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, London, UK), 2020, pp. 1-6.
  17. Y. Li, Deep reinforcement learning: an overview, arXiv preprint, 2017, DOI: 10.48550/arXiv.1701.07274. arXiv preprint arXiv: 1701.07274.
  18. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, and G. Ostrovski, Human-level control through deep reinforcement learning, Nature 518 (2015), no. 7540, 529-533. https://doi.org/10.1038/nature14236
  19. J. Ye and Y.-J. A. Zhang, Drag: deep reinforcement learning based base station activation in heterogeneous networks, IEEE Trans. Mob. Comput. 19 (2019), no. 9, 2076-2087.
  20. H. Ju, S. Kim, Y. Kim, and B. Shim, Energy-efficient ultra-dense network with deep reinforcement learning, IEEE Trans. Wireless Commun. 21 (2022), no. 8, 6539-6552. https://doi.org/10.1109/TWC.2022.3150425
  21. Y. Zhu and S. Wang, Joint traffic prediction and base station sleeping for energy saving in cellular networks, (ICC 2021-IEEE International Conference on Communications, Montreal, Canada), 2021, pp. 1-6.
  22. G. Jang, N. Kim, T. Ha, C. Lee, and S. Cho, Base station switching and sleep mode optimization with lstm-based user prediction, IEEE Access 8 (2020), 222711-222723. https://doi.org/10.1109/ACCESS.2020.3044242
  23. Q. Wu, X. Chen, Z. Zhou, L. Chen, and J. Zhang, Deep reinforcement learning with spatio-temporal traffic forecasting for data-driven base station sleep control, IEEE/ACM Trans. Netw. 29 (2021), no. 2, 935-948. https://doi.org/10.1109/TNET.2021.3053771
  24. S. Kumari and B. Singh, Data-driven handover optimization in small cell networks, Wireless Netw. 25 (2019), no. 8, 5001-5009. https://doi.org/10.1007/s11276-019-02111-6
  25. K. Tan, D. Bremner, J. Le Kernec, Y. Sambo, L. Zhang, and M. A. Imran, Intelligent handover algorithm for vehicleto-network communications with double-deep q-learning, IEEE Trans. Veh. Technol. 71 (2022), no. 7, 7848-7862. https://doi.org/10.1109/TVT.2022.3169804
  26. K. Qi, T. Liu, and C. Yang, Federated learning based proactive handover in millimeter-wave vehicular networks, (15th IEEE International Conference on Signal Processing (ICSP), Beijing, China), 2020, pp. 401-406.
  27. J. Feng, C. Rong, F. Sun, D. Guo, and Y. Li, PMF: A privacy-preserving human mobility prediction framework via federated learning, Proc. ACM on Interact., Mobile, Wear. Ubiquitous Technol. 4 (2020), no. 1, 1-21.
  28. C. Koetsier, J. Fiosina, J. N. Gremmel, J. P. Muller, D. M. Woisetschlager, and M. Sester, Detection of anomalous vehicle trajectories using federated learning, ISPRS Open J. Photogr. Remote Sens. 4 (2022), 100013.
  29. K.-C. Chang, K.-C. Chu, H.-C. Wang, Y.-C. Lin, and J.-S. Pan, Energy saving technology of 5G base station based on internet of things collaborative control, IEEE Access 8 (2020), 32935-32946. https://doi.org/10.1109/ACCESS.2020.2973648
  30. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Y. Arcas, Communication-efficient learning of deep networks from decentralized data, (Artificial Intelligence and Statistics. PMLR), 2017, pp. 1273-1282.
  31. P. Geibel, Reinforcement learning for MDPS with constraints, (European Conference on Machine Learning, Berlin, Germany), 2006, pp. 646-653.
  32. H. Van Hasselt, A. Guez, and D. Silver, Deep reinforcement learning with double Q-learning, (Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA), 2016.
  33. 3GPP, Evolved universal terrestrial radio access (E-UTRA): Further advancements for E-UTRA physical layer aspects. 36.814. 3rd Generation Partnership Project (3GPP), 2017. Version 9.2.0.
  34. 3GPP, Technical specification group radio access network: Small cell enhancements for E-UTRA and EUTRAN- Physical layer aspects. 36.872. 3rd Generation Partnership Project (3GPP), 2013. Version 12.1.0.
  35. MatthiasGrossglauser Michal Piorkowski Natasa Sarafijanovic Djukic, Crawdad dataset epfl/mobility, 2009. (v. 2009 2024).
  36. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from https://www.tensorflow.org/
  37. D. J. Beutel, T. Topal, A. Mathur, X. Qiu, T. Parcollet, and N. D. Lane, Flower: A friendly federated learning research framework, arXive preprint, 2020, DOI: 10.48550/arXiv.2007.14390
  38. S. Ioffe and C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, (International Conference on Machine Learning. PMLR, Lille, France), 2015, pp. 448-456.