Browse > Article
http://dx.doi.org/10.6109/jkiice.2021.25.9.1267

Deep Reinforcement Learning-based Distributed Routing Algorithm for Minimizing End-to-end Delay in MANET  

Choi, Yeong-Jun (Department of Information and Communication Engineering, Pukyong National University)
Seo, Ju-Sung (Department of Information and Communication Engineering, Pukyong National University)
Hong, Jun-Pyo (Department of Information and Communication Engineering, Pukyong National University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.25, no.9, 2021 , pp. 1267-1270 More about this Journal
Abstract
In this paper, we propose a distributed routing algorithm for mobile ad hoc networks (MANET) where mobile devices can be utilized as relays for communication between remote source-destination nodes. The objective of the proposed algorithm is to minimize the end-to-end communication delay caused by transmission failure with deep channel fading. In each hop, the node needs to select the next relaying node by considering a tradeoff relationship between the link stability and forward link distance. Based on such feature, we formulate the problem with partially observable Markov decision process (MDP) and apply deep reinforcement learning to derive effective routing strategy for the formulated MDP. Simulation results show that the proposed algorithm outperforms other baseline schemes in terms of the average end-to-end delay.
Keywords
Routing algorithm; Reinforcement learning; MANET; Partially observable MDP;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Conti and S. Giordano, "Mobile ad hoc networking: Milestones, challenges, and new research directions," IEEE Commun. Mag., vol. 52, no. 1, pp. 85-96, Jan. 2014.   DOI
2 N. Singh and L. Shrivastava, "Impact of Antenna model with the variation of speed for Reactive and Hybrid routing protocols in Mobile Ad-Hoc Networks," Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, vol. 5, no. 5, pp. 17-27, Oct. 2015.   DOI
3 R. Li, F. Li, X. Li, and Y. Wang, "Qgrid : Q-learning based routing protocol for vehicular ad hoc networks," in Proc. IEEE IPCCC, pp. 1-8, Dec. 2014.
4 A. Ghaffari, "Real-time routing algorithm for mobile ad hoc networks using reinforcement learning and heuristic algorithms," in Wireless Networks, pp. 703-714, vol. 23, no. 3, 2017.   DOI
5 W. S. Jung, J. Yim, and Y. B. Ko, "QGeo: Q-learning based geograpthic ad hoc routing protocol for unmanned robotic networks," in IEEE Commun. Lett. vol. 21, no. 10, pp. 2258-2261, Oct. 2017.   DOI
6 K. Lee and J. P. Hong, "Device-to-device communication power control technique for ensuring communication security of cellular system," J. Korea Inst. Inf. Commun. Eng., vol. 21, no. 6, pp. 1100-1105, Jun. 2017.   DOI
7 R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. Cambridge, MA: MIT Press, 2018.
8 V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing Atari with deep reinforcement learning," in arXiv preprint arxiv:1312.5602, 2013.
9 J. Liu, Q. Wang, C. He, K. Jaffres-Runser, Y. Xu, Z. Li, and Y. Xu, "QMR: Q-learning based multi-objective optimization routing protocol for flying ad hoc networks," Comput. Commun., vol. 150, pp. 304-316, Jan. 2020.   DOI