Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Resource Allocation Strategy of Internet of Vehicles Using Reinforcement Learning

  • Xi, Hongqi (College of Computer and Information Technology, Henan Finance University) ;
  • Sun, Huijuan (College of Computer and Information Technology, Henan Finance University)
  • Received : 2021.11.16
  • Accepted : 2022.03.15
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

An efficient and reasonable resource allocation strategy can greatly improve the service quality of Internet of Vehicles (IoV). However, most of the current allocation methods have overestimation problem, and it is difficult to provide high-performance IoV network services. To solve this problem, this paper proposes a network resource allocation strategy based on deep learning network model DDQN. Firstly, the method implements the refined modeling of IoV model, including communication model, user layer computing model, edge layer offloading model, mobile model, etc., similar to the actual complex IoV application scenario. Then, the DDQN network model is used to calculate and solve the mathematical model of resource allocation. By decoupling the selection of target Q value action and the calculation of target Q value, the phenomenon of overestimation is avoided. It can provide higher-quality network services and ensure superior computing and processing performance in actual complex scenarios. Finally, simulation results show that the proposed method can maintain the network delay within 65 ms and show excellent network performance in high concurrency and complex scenes with task data volume of 500 kbits.

Keywords

References

  1. S. K. Singh, P. K. Sharma, Y. Pan, and J. H. Park, "BIIoVT: blockchain-based secure storage architecture for intelligent Internet of Vehicular Things," IEEE Consumer Electronics Magazine, 2021. https://doi.org/10.1109/MCE.2021.3089992
  2. P. K. Sharma and J. H. Park, "Blockchain-based secure mist computing network architecture for intelligent transportation systems," IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 8, pp. 5168-5177, 2021. https://doi.org/10.1109/TITS.2020.3040989
  3. Y. Xiao, H. Liu, and X. Cheng, "Key technologies of Internet of vehicles and their development trends and challenges," Communications Technology, vol. 54, no. 1, pp. 1-8, 2021.
  4. L. Pu, X. Chen, G. Mao, Q. Xie, and J. Xu, "Chimera: an energy-efficient and deadline-aware hybrid edge computing framework for vehicular crowdsensing applications," IEEE Internet of Things Journal, vol. 6, no. 1, pp. 84-99, 2019. https://doi.org/10.1109/jiot.2018.2872436
  5. S. K. Singh, Y. S. Jeong, and J. H. Park, "A deep learning-based IoT-oriented infrastructure for secure smart city," Sustainable Cities and Society, vol. 60, article no. 102252, 2020. https://doi.org/10.1016/j.scs.2020.102252
  6. B. Fan, H. Tian, S. Zhu, Y. Chen, and X. Zhu, "Traffic-aware relay vehicle selection in millimeter-wave vehicle-to-vehicle communication," IEEE Wireless Communications Letters, vol. 8, no. 2, pp. 400-403, 2019. https://doi.org/10.1109/lwc.2018.2873585
  7. H. Zhang, Y. Cheng, K. Liu, and X. He, "The mobility management strategies by integrating mobile edge computing and CDN in vehicular networks," Journal of Electronics & Information Technology, vol. 42, no. 6, pp. 1444-1451, 2020.
  8. M. Sohail, L. Wang, R. Ali, S. Rahim, and J. Yao, "Efficient data handover and intelligent information assessment in software-defined vehicular social networks," IET Intelligent Transport Systems, vol. 13, no. 12, pp. 1814-1821, 2019. https://doi.org/10.1049/iet-its.2019.0323
  9. H. Yang, X. Xie, and M. Kadoch, "Intelligent resource management based on reinforcement learning for ultra-reliable and low-latency IoV communication networks," IEEE Transactions on Vehicular Technology, vol. 68, no. 5, pp. 4157-4169, 2019. https://doi.org/10.1109/tvt.2018.2890686
  10. X. Huang, K. Xu, Q. Chen, and J. Zhang, "Delay-aware caching in Internet-of-Vehicles networks," IEEE Internet of Things Journal, vol. 8, no. 13, pp. 10911-10921, 2021. https://doi.org/10.1109/JIOT.2021.3051290
  11. J. Cheng, J. Cheng, M. Zhou, F. Liu, S. Gao, and C. Liu, "Routing in internet of vehicles: a review," IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 5, pp. 2339-2352, 2015. https://doi.org/10.1109/TITS.2015.2423667
  12. X. Wang, Z. Ning, X. Hu, L. Wang, B. Hu, J. Cheng, and V. C. Leung, "Optimizing content dissemination for real-time traffic management in large-scale Internet of Vehicle systems," IEEE Transactions on Vehicular Technology, vol. 68, no. 2, pp. 1093-1105, 2019. https://doi.org/10.1109/tvt.2018.2886010
  13. C. Chen, L. Chen, L. Liu, S. He, X. Yuan, D. Lan, and Z. Chen, "Delay-optimized V2V-based computation offloading in urban vehicular edge computing and networks," IEEE Access, vol. 8, pp. 18863-18873, 2020. https://doi.org/10.1109/access.2020.2968465
  14. W. Zhuang, Q. Ye, F. Lyu, N. Cheng, and J. Ren, "SDN/NFV-empowered future IoV with enhanced communication, computing, and caching," Proceedings of the IEEE, vol. 108, no. 2, pp. 274-291, 2020. https://doi.org/10.1109/jproc.2019.2951169
  15. L. Zhang, Z. Zhao, Q. Wu, H. Zhao, H. Xu, and X. Wu, "Energy-aware dynamic resource allocation in UAV assisted mobile edge computing over social Internet of Vehicles," IEEE Access, vol. 6, pp. 56700-56715, 2018. https://doi.org/10.1109/access.2018.2872753
  16. J. Kim, J. Lee, S. Moon, and I. Hwang, "A position-based resource allocation scheme for V2V communication," Wireless Personal Communications, vol. 98, no. 1, pp. 1569-1586, 2018. https://doi.org/10.1007/s11277-017-4935-3
  17. F. Abbas, P. Fan, and Z. Khan, "A novel low-latency V2V resource allocation scheme based on cellular V2X communications," IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, pp. 2185-2197, 2019. https://doi.org/10.1109/tits.2018.2865173
  18. J. Wang, Q. Qi, S. Qing, and J. Liao, "Elastic vehicular resource providing based on service function-group resource mapping of smart identify network," IEEE Systems Journal, vol. 12, no. 2, pp. 1897-1908, 2018. https://doi.org/10.1109/jsyst.2017.2771443
  19. S. Wang, F. Huang, and C. Wang, "Adaptive proportional fairness resource allocation for OFDM-based cognitive radio networks," Wireless Networks, vol. 19, no. 3, pp. 273-284, 2013. https://doi.org/10.1007/s11276-012-0465-9
  20. Q. Zhang, L. Gui, F. Hou, J. Chen, S. Zhu, and F. Tian, "Dynamic task offloading and resource allocation for mobile-edge computing in dense cloud RAN," IEEE Internet of Things Journal, vol. 7, no. 4, pp. 3282-3299, 2020. https://doi.org/10.1109/jiot.2020.2967502
  21. J. Zhang, X. Hu, Z. Ning, E. C. H. Ngai, L. Zhou, J. Wei, J. Cheng, B. Hum and V. C. Leung, "Joint resource allocation for latency-sensitive services over mobile edge computing networks with caching," IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4283-4294, 2019. https://doi.org/10.1109/jiot.2018.2875917
  22. S. S. Lee and S. Lee, "Resource allocation for vehicular fog computing using reinforcement learning combined with heuristic information," IEEE Internet of Things Journal, vol. 7, no. 10, pp. 10450-10464, 2020. https://doi.org/10.1109/jiot.2020.2996213
  23. C. Cui, M. Zhao, and K. Wong, "An LSTM-method-based availability prediction for optimized offloading in mobile edges," Sensors, vol. 19, no. 20, article no. 4467, 2019. https://doi.org/10.3390/s19204467
  24. M. Chen, T. Wang, K. Ota, M. Dong, M. Zhao, and A. Liu, "Intelligent resource allocation management for vehicles network: an A3C learning approach," Computer Communications, vol. 151, pp. 485-494, 2020. https://doi.org/10.1016/j.comcom.2019.12.054
  25. J. Zhao, M. Kong, Q. Li, and X. Sun, "Contract-based computing resource management via deep reinforcement learning in vehicular fog computing," IEEE Access, vol. 8, pp. 3319-3329, 2019. https://doi.org/10.1109/access.2019.2963051
  26. X. Xu, Y. Xue, X. Li, L. Qi, and S. Wan, "A computation offloading method for edge computing with vehicleto-everything," IEEE Access, vol. 7, pp. 131068-131077, 2019. https://doi.org/10.1109/access.2019.2940295
  27. J. Kim, Y. Han, and I. Kim, "Efficient groupcast schemes for vehicle platooning in V2V network," IEEE Access, vol. 7, pp. 171333-171345, 2019. https://doi.org/10.1109/access.2019.2955791
  28. H. Peng, L. Liang, X. Shen, and G. Y. Li, "Vehicular communications: a network layer perspective," IEEE Transactions on Vehicular Technology, vol. 68, no. 2, pp. 1064-1078, 2019. https://doi.org/10.1109/tvt.2018.2833427
  29. Y. Liu, H. Yu, S. Xie, and Y. Zhang, "Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks," IEEE Transactions on Vehicular Technology, vol. 68, no. 11, pp. 11158-11168, 2019. https://doi.org/10.1109/tvt.2019.2935450
  30. M. S. Demir, H. B. Eldeeb, and M. Uysal, "Comp-based dynamic handover for vehicular VLC networks," IEEE Communications Letters, vol. 24, no. 9, pp. 2024-2028, 2020. https://doi.org/10.1109/lcomm.2020.2994416