QLGR: A Q-learning-based Geographic FANET Routing Algorithm Based on Multi-agent Reinforcement Learning |
Qiu, Xiulin
(School of Computer Science and Engineering, Nanjing University of Science and Technology)
Xie, Yongsheng (School of Computer Science and Engineering, Nanjing University of Science and Technology) Wang, Yinyin (School of Computer Science and Engineering, Nanjing University of Science and Technology) Ye, Lei (School of Computer Science and Engineering, Nanjing University of Science and Technology) Yang, Yuwang (School of Computer Science and Engineering, Nanjing University of Science and Technology) |
1 | Otto, A., Agatz, N., Campbell, J., Golden, B., Pesch, E., "Optimization approaches for civil applications of unmanned aerial vehicles (UAVs) or aerial drones: A survey," Special Issue on Drone Delivery Systems, 72(4), 411-458, 2018. |
2 | Hayat, S., Yanmaz, E., Muzaffar, R., "Survey on unmanned aerial vehicle networks for civil applications, A communications viewpoint," IEEE Communications Surveys & Tutorials, 18(4), 2624-2661, 2016. DOI |
3 | Mowla, N.I., Tran, N.H., Doh, I., Chae, K., "AFRL: Adaptive federated reinforcement learning for intelligent jamming defense in FANET," Journal of Communications and Networks, 22(3), 244- 258, 2020. DOI |
4 | Gankhuyag, G., Shrestha, A.P., Yoo, S.J., "Robust and Reliable Predictive Routing Strategy for Flying Ad-Hoc Networks," IEEE Access, vol. 5, pp. 643-654, 2017. DOI |
5 | Arafat, M.Y., Moh, S., "Localization and Clustering Based on Swarm Intelligence in UAV Networks for Emergency Communications," IEEE Internet of Things Journal, 6(5), 8958-8976, 2019. DOI |
6 | Wang, S., Huang, C., Wang, D., "Delay-aware relay selection with heterogeneous communication range in VANETs," Wireless Networks, 26(2), 995-1004, 2020. DOI |
7 | Gunduz, D., De Kerret, P., Sidiropoulos, N.D., Gesbert, D., Murthy, C., Mihaela, V.D.S., "Machine Learning in the Air," IEEE Journal on Selected Areas in Communications, 37(10), 2184-2199, 2019. DOI |
8 | Valadarsky, A., Schapira, M., Shahaf, D., Tamar, A., "Learning to Route," in Proc. of the 16th ACM Workshop on Hot Topics in Networks, pp. 185-191, 2017. |
9 | Jung, W., Yim, J., Ko, Y., "QGeo: Q-Learning-Based Geographic Ad Hoc Routing Protocol for Unmanned Robotic Networks," IEEE Communications Letters, 21(10), 2258-2261, 2017. DOI |
10 | Defrawy, K.E., Tsudik, G., "ALARM: Anonymous Location-Aided Routing in Suspicious MANETs," IEEE Transactions on Mobile Computing, 10(9), 1345-1358, 2011. DOI |
11 | Narayanan, P.S., Joice, C.S., "Vehicle-to-Vehicle (V2V) Communication using Routing Protocols: A Review," in Proc. of 2019 International Conference on Smart Structures and Systems (ICSSS), pp. 1-10, 2019. |
12 | Karp, B., Kung, H.T., "GPSR: greedy perimeter stateless routing for wireless networks," in Proc. of the 6th annual international conference on Mobile computing and networking, Boston, Massachusetts, USA, pp. 243-254, 2000. |
13 | Boyan, J.A., Littman, M.L., "Packet Routing in Dynamically Changing Networks: A Reinforcement Learning Approach," J.A.i.n.i.p.s., 6, 671-678, 1993. |
14 | Pourpeighambar, B., Dehghan, M., Sabaei, M., "Multi-agent learning based routing for delay minimization in cognitive radio networks," Journal of Network and Computer Applications, 84, 82-92, 2017. DOI |
15 | Elwhishi, A., Ho, P.H., Naik, K., Shihada, B., "ARBR: Adaptive reinforcement-based routing for DTN," in Proc. of 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications, 2010. |
16 | Zeng, S., Xu, X., Chen, Y., "Multi-Agent Reinforcement Learning for Adaptive Routing: A Hybrid Method using Eligibility Traces," in Proc. of IEEE 16th International Conference on Control & Automation (ICCA), pp. 1332-1339, 2020. |
17 | Kaviani, S., Bo, R., Ahmed, E., Larson, K.A., Kim, J.H., "Robust and Scalable Routing with MultiAgent Deep Reinforcement Learning for MANETs," arXiv preprint arXiv:2101.03273, 2021. |
18 | Li, R., Li, F., Li, X., Wang, Y., "QGrid: Q-learning based routing protocol for vehicular ad hoc networks," in Proc. of 2014 IEEE 33rd International Performance Computing and Communications Conference (IPCCC), pp. 1-8, 2014. |
19 | Kasana, R., Kumar, S., "A geographic routing algorithm based on Cat Swarm Optimization for vehicular ad-hoc networks," in Proc. of 2017 4th International Conference on Signal Processing and Integrated Networks (SPIN), pp. 86-90, 2017. |
20 | Mammeri, Z., "Reinforcement Learning Based Routing in Networks: Review and Classification of Approaches," IEEE Access, 7, 55916-55950, 2019. DOI |
21 | Al-Rawi, H.A.A., Ng, M.A., Yau, K.-L.A., "Application of reinforcement learning to routing in distributed wireless networks: a review," Artificial Intelligence Review, 43(3), 381-416, 2015. DOI |
22 | Gawlowicz, P., Zubow, A., "ns3-gym: Extending openai gym for networking research," J.a.p.a., 2018. |
23 | Singh, N., Elamvazuthi, I., Nallagownden, P., Ramasamy, G., Jangra, A.J.S., "Routing Based Multi-Agent System for Network Reliability in the Smart Microgrid," Sensors, 20(10), 2020. |
24 | Liang, X., Balasingham, I., Byun, S.-S., "A multi-agent reinforcement learning based routing protocol for wireless sensor networks," in Proc. of 2008 IEEE International Symposium on Wireless Communication Systems, pp. 552-557, 2008. |
25 | Hong, J., Zhang, D., Niu, X., " Impact Analysis of Node Motion on the performance of FANET routing protocols," in Proc. of 14th International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM 2018), pp. 147-162, 2018. |
26 | Huang, H., Yin, H., Min, G., Zhang, J., Wu, Y., Zhang, X., "Energy-Aware Dual-Path Geographic Routing to Bypass Routing Holes in Wireless Sensor Networks," IEEE Transactions on Mobile Computing, 17(6), 1339-1352, 2018. DOI |
27 | Mukhutdinov, D., Filchenkov, A., Shalyto, A., Vyatkin, V., "Multi-agent deep learning for simultaneous optimization for time and energy in distributed routing system," Future Generation Computer Systems, 94, 587-600, 2019. DOI |
28 | Mao, H., Gong, Z., Zhang, Z., Xiao, Z., Ni, Y., "Learning multi-agent communication under limited-bandwidth restriction for internet packet routing," J.a.p.a., 2019. |
29 | Luong, N.C., Hoang, D.T., Gong, S., Niyato, D., Wang, P., Liang, Y., Kim, D.I., "Applications of Deep Reinforcement Learning in Communications and Networking: A Survey," IEEE Communications Surveys & Tutorials, 21(4), 3133-3174, 2019. DOI |
30 | Afzal, K., Tariq, R., Aadil, F., Iqbal, Z., Ali, N., Sajid, M., "An Optimized and Efficient Routing Protocol Application for IoV," Mathematical Problems in Engineering, 9977252, 2021. |
31 | Li, X., Hu, X., Zhang, R., Yang, L., "Routing Protocol Design for Underwater Optical Wireless Sensor Networks: A Multiagent Reinforcement Learning Approach," IEEE Internet of Things Journal, 7(10), 9805-9818, 2020. DOI |
32 | Chmaj, G., Selvaraj, H., "Distributed Processing Applications for UAV/drones: A Survey," Progress in Systems Engineering, pp 449-454, 2015. |
33 | Lakew, D.S., Sa'ad, U., Dao, N., Na, W., Cho, S., "Routing in Flying Ad Hoc Networks: A Comprehensive Survey," IEEE Communications Surveys & Tutorials, 22(2), 1071-1120, 2020. DOI |
34 | You, X., Li, X., Xu, Y., Feng, H., Zhao, J., Yan, H., "Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning," IEEE Transactions on Systems, Man, and Cybernetics: Systems, pp. 1-14, 2020. |
35 | Zheng, L.-M., Li, X.-D., Li, X.-Y., "RLAR: Adaptive routing algorithm based onreinforcement learning," J.C.E., (4), 13, 2011. |
36 | Li, X., Hu, X., Zhang, R., Yang, L., "Routing Protocol Design for Underwater Optical Wireless Sensor Networks: A Multi-Agent Reinforcement Learning Approach," IEEE Internet of Things Journal, 7(10), 9805-9818, 2020. DOI |
37 | Cruz, E.P.F.d., "A Comprehensive Survey in Towards to Future FANETs," IEEE Latin America Transactions, 16(3), 876-884, 2018. DOI |
38 | Sharma, M., Singh, M., Walia, K., Kaur, K., "A Comprehensive Study of Performance Parameters for MANET, VANET and FANET," in Proc. of 2019 IEEE 10th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), pp. 0643-0646, 2019. |