Browse > Article
http://dx.doi.org/10.3837/tiis.2022.09.015

A Diversified Message Type Forwarding Strategy Based on Reinforcement Learning in VANET  

Xu, Guoai (School of Cyberspace Security, Beijing University of Posts and Telecommunications)
Liu, Boya (School of Cyberspace Security, Beijing University of Posts and Telecommunications)
Xu, Guosheng (School of Cyberspace Security, Beijing University of Posts and Telecommunications)
Zuo, Peiliang (Beijing Electronic Science and Technology Institute)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.9, 2022 , pp. 3104-3123 More about this Journal
Abstract
The development of Vehicular Ad hoc Network (VANET) has greatly improved the efficiency and safety of social transportation, and the routing strategy for VANET has also received high attention from both academia and industry. However, studies on dynamic matching of routing policies with the message types of VANET are in short supply, which affects the operational efficiency and security of VANET to a certain extent. This paper studies the message types in VANET and fully considers the urgency and reliability requirements of message forwarding under various types. Based on the diversified types of messages to be transmitted, and taking the diversified message forwarding strategies suitable for VANET scenarios as behavioral candidates, an adaptive routing method for the VANET message types based on reinforcement learning (RL) is proposed. The key parameters of the method, such as state, action and reward, are reasonably designed. Simulation and analysis show that the proposed method could converge quickly, and the comprehensive performance of the proposed method is obviously better than the comparison methods in terms of timeliness and reliability.
Keywords
Routing strategy; Message Forwarding Strategy; Reinforcement Learning; VANET; Black nodes;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 M. Syfullah, J. M. Lim and F. L. Siaw, "Mobility-Based Clustering Algorithm for Multimedia Broadcasting over IEEE 802.11p-LTE-enabled VANET," KSII Transactions on Internet and Information Systems, vol. 13, no. 3, pp. 1213-1237, 2019.   DOI
2 Jianyong Li, Research on optimization of learning algorithm for routing in wireless sensor networks, Southwest University, 2016.
3 Ming Yuan, Research on Vanet Routing Algorithm Based on Reinforcement Learnin, Xidian University, 2017.
4 Celimuge Wu, Satoshi Ohzahata, and Toshihiko Kato, "Flexible, Portable, and Practicable Solution for Routing in VANETs: A Fuzzy Constraint Q-Learning Approach," IEEE Transactions on Vehicular Technology, 62(9), 4251-4263, 2013.   DOI
5 Plate R, Wakayama C, "Utilizing kinematics and selective sweeping in reinforcement learningbased routing algorithms for underwater networks," Ad Hoc Networks, 34(NOV.), 105-120, 2015.   DOI
6 X. Yang, M. Li, Z. Qian and T. Di, "Improvement of GPSR Protocol in Vehicular Ad Hoc Network," IEEE Access, vol. 6, pp. 39515-39524, 2018.   DOI
7 H. Yuan, J. Geng, C. Liu, F. Bian, and T. Surapunt, "An improved GPSR routing algorithm based on vehicle trajectory mining," in Proc. of the 5th International Conference Geo-Spatial Knowledge and Intelligence, pp. 343-349, 2018.
8 Jinqiao Wu, Min Fang, Xiao Li, "Reinforcement Learning Based Mobility Adaptive Routing for Vehicular Ad-Hoc Networks," Wireless Personal Communications, 101(4), 2143-2171, 2018.   DOI
9 Yiming Lin, A Reinforcement Learning-based Routing Protocol in VANETs, Xiamen University, 2017.
10 G. Santhi, A. Nachiappan, M. Z. Ibrahime, R. Raghunadhane and M. K. Favas, "Q-learning based adaptive QoS routing protocol for MANETs," in Proc. of 2011 International Conference on Recent Trends in Information Technology (ICRTIT), pp. 1233-1238, 2011.
11 Shanshan Jiang, Zhitong Huang, Yuefeng Ji, "Adaptive UAV-Assisted Geographic Routing With Q-Learning in VANET," IEEE Communications Letters, vol. 25, no. 4, pp. 1358-1362, April 2021.   DOI
12 J. Aznar-Poveda, A. -J. Garcia-Sanchez, E. Egea-Lopez and J. Garcia-Haro, "MDPRP: A QLearning Approach for the Joint Control of Beaconing Rate and Transmission Power in VANETs," IEEE Access, vol. 9, pp. 10166-10178, 2021.   DOI
13 C. Lin, S. Yuan, S. Chiu and M. Tsai, "ProgressFace: An Algorithm to Improve Routing Efficiency of GPSR-Like Routing Protocols in Wireless Ad Hoc Networks," IEEE Transactions on Computers, vol. 59, no. 6, pp. 822-834, June 2010.   DOI
14 Alsaqour, R., Abdelhaq, M., Saeed, R., Uddin, M., Alsukour, O., Al-Hubaishi, M. and Alahdal, T., "Dynamic packet beaconing for GPSR mobile ad hoc position-based routing protocol using fuzzy logic," Journal of Network and Computer Applications, vol. 47, pp. 32-46, 2015.   DOI
15 Cyber Security Administration of the Ministry of Industry and Information Technology, White Paper on Network Security of Internet of Vehicles, 2020. [R/OL].
16 A. BENGAG, M. E. Boukhari, "Enhancing GPSR routing protocol based on Velocity and Density for real-time urban scenario," in Proc. of 2020 International Conference on Intelligent Systems and Computer Vision (ISCV), pp. 1-5, 2020.
17 R. A. Nazib, S. Moh, "Reinforcement Learning-Based Routing Protocols for Vehicular Ad Hoc Networks: A Comparative Survey," IEEE Access, vol. 9, pp. 27552-27587, 2021.   DOI
18 Network 5.0 Industry and Technology innovation Alliance, Network 5.0 Technology White Paper (2.0), 2021. [R/OL].
19 Wang, T., Anwar, S., Sun, H. and Zhou, Y., "Modified greedy perimeter stateless routing for vehicular ad hoc networking algorithm," International Journal of Sensor Networks, vol. 27(3), pp.163-171, 2018.   DOI
20 C. Wang, Q. Fan, X. Chen, and W. Xu, "Prediction based greedy perimeter stateless routing protocol for vehicular self-organizing network," in Proc. of IOP Conference Series: Materials Science and Engineering, vol. 322, p.052019, 20108. December 2017.
21 K. Gu, X. Dong, X. Li and W. Jia, "Cluster-Based Malicious Node Detection for False Downstream Data in Fog Computing-Based VANETs," IEEE Transactions on Network Science and Engineering, vol. 9, no. 3, pp. 1245-1263, 1 May-June 2022.   DOI
22 A. Benmir, A. Korichi, A. Bourouis, M. Alreshoodi and L. Al-Jobouri, "An Enhanced GPSR Protocol for Vehicular Ad hoc Networks," in Proc. of 2019 11th Computer Science and Electronic Engineering (CEEC), pp. 85-89, 2019.
23 S. Younes, M. Khelifi, A. Alioua and I. Souici, "EKF-GPSR: An Extended Kalman Filter for Efficient Routing in Vehicular Networks," in Proc. of 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), pp. 1-6, 2021.
24 M. Dabbaghjamanesh, A. Moeini and A. Kavousi-Fard, "Reinforcement Learning-Based Load Forecasting of Electric Vehicle Charging Station Using Q-Learning Technique," IEEE Transactions on Industrial Informatics, vol. 17, no. 6, pp. 4229-4237, June 2021.   DOI
25 Zhihua Zhou, Machine Learning, Beijing: Tsinghua University Press, 2016.
26 Yuanyuan Meng, Liancheng Min Xu, Ren, Yanfei Wang, "D-S Evidence Fusion Method Based on High Conflict Correction," ComputerEngineering, 44(1), 79-83, 90, 2018.
27 A. M. El-Semary and H. Diab, "BP-AODV: Blackhole Protected AODV Routing Protocol for MANETs Based on Chaotic Map," IEEE Access, vol. 7, pp. 95197-95211, 2019.   DOI
28 Celimuge Wu, Kazuya Kumekawa, and Toshihiko Kato, "Distributed Reinforcement Learning Approach for Vehicular Ad Hoc Networks," IEICE Transactions on Communications, E93.B(6), 1431-1442, 2010.   DOI