Browse > Article
http://dx.doi.org/10.6109/jkiice.2021.25.1.50

Path selection algorithm for multi-path system based on deep Q learning  

Chung, Byung Chang (Electronics and Telecommunications Research Institute)
Park, Heasook (Electronics and Telecommunications Research Institute)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.25, no.1, 2021 , pp. 50-55 More about this Journal
Abstract
Multi-path system is a system in which utilizes various networks simultaneously. It is expected that multi-path system can enhance communication speed, reliability, security of network. In this paper, we focus on path selection in multi-path system. To select optimal path, we propose deep reinforcement learning algorithm which is rewarded by the round-trip-time (RTT) of each networks. Unlike multi-armed bandit model, deep Q learning is applied to consider rapidly changing situations. Due to the delay of RTT data, we also suggest compensation algorithm of the delayed reward. Moreover, we implement testbed learning server to evaluate the performance of proposed algorithm. The learning server contains distributed database and tensorflow module to efficiently operate deep learning algorithm. By means of simulation, we showed that the proposed algorithm has better performance than lowest RTT about 20%.
Keywords
Reinforcement learning; Delay compensation; Multi-path system; Multi-path TCP; Path selection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. S. Kim, J. Y. Lee, and B. C. Kim, "Design of MPTCP congestion control based on BW measurement for wireless networks," Journal of the Korea Institute of Information and Communication Engineering, vol. 21, no. 6, pp. 1127-1136, Jun. 2017.   DOI
2 K. Jang, "Reinforcement learning for node-disjoint path problem in wireless ad-hoc networks," Journal of the Korea Institute of Information and Communication Engineering, vol. 23, no. 8, pp. 1011-1017, Aug. 2019.   DOI
3 S. Park, B. Lim, and H. Jung, "CNN-Based Toxic Plant Identification System," Journal of the Korea Institute of Information and Communication Engineering, vol. 24, no. 8, pp. 993-998, Aug. 2020.   DOI
4 V. Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, "Human-level control through deep reinforcement learning," Nature, vol. 518, pp. 529-533, Feb. 2015.   DOI
5 B. C. Chung and D. H. Cho, "Semidynamic cell-clustering algorithm based on reinforcement learning in cooperative transmission system," IEEE Systems Journal, vol. 12, no. 4, pp. 3853-3856, Dec. 2018.   DOI
6 S. P. M. Choi and D. Y. Yeung, "Predictive Q-Routing: A Memory-based Reinforcement Learning Approach to Adaptive Traffic Control," in Proceedings of Advances in Neural Information Processing Systems, pp. 945-951, 1996.
7 S. Hoceini, A. Mellouk, and Y.Amirat, "K-Shortest Paths Q-Routing: A New QoS Routing Algorithm in Telecommunication Networks," in Proceedings of International Conference on Networking 2005, pp. 164-172, 2005.
8 Apache HBase, [Internet]. Available: https://hbase.apache.org/.
9 M. Series, "Guidelines for evaluation of radio interface technologies for IMT-Advanced," International Telecommunication Union (ITU), Geneva, Switzerland, Technical Report ITU-R M.2135-1, Dec. 2009.
10 C. Raiciu, M. Handley, and D. Wischik, "Coupled congestion control for multipath transport protocols," RFC 6356, IETF, Oct. 2011. [Internet]. Available: https://tools.ietf.org/html/rfc6356.