Browse > Article
http://dx.doi.org/10.4218/etrij.2021-0212

IRSML: An intelligent routing algorithm based on machine learning in software defined wireless networking  

Duong, Thuy-Van T. (Faculty of Information Technology, Ton Duc Thang University)
Binh, Le Huu (Faculty of Information Technology, University of Sciences, Hue University)
Publication Information
ETRI Journal / v.44, no.5, 2022 , pp. 733-745 More about this Journal
Abstract
In software-defined wireless networking (SDWN), the optimal routing technique is one of the effective solutions to improve its performance. This routing technique is done by many different methods, with the most common using integer linear programming problem (ILP), building optimal routing metrics. These methods often only focus on one routing objective, such as minimizing the packet blocking probability, minimizing end-to-end delay (EED), and maximizing network throughput. It is difficult to consider multiple objectives concurrently in a routing algorithm. In this paper, we investigate the application of machine learning to control routing in the SDWN. An intelligent routing algorithm is then proposed based on the machine learning to improve the network performance. The proposed algorithm can optimize multiple routing objectives. Our idea is to combine supervised learning (SL) and reinforcement learning (RL) methods to discover new routes. The SL is used to predict the performance metrics of the links, including EED quality of transmission (QoT), and packet blocking probability (PBP). The routing is done by the RL method. We use the Q-value in the fundamental equation of the RL to store the PBP, which is used for the aim of route selection. Concurrently, the learning rate coefficient is flexibly changed to determine the constraints of routing during learning. These constraints include QoT and EED. Our performance evaluations based on OMNeT++ have shown that the proposed algorithm has significantly improved the network performance in terms of the QoT, EED, packet delivery ratio, and network throughput compared with other well-known routing algorithms.
Keywords
intelligent routing; reinforcement learning; SDWN; supervised learning;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Abujoda, D. Dietrich, P. Papadimitriou, and A. Sathiaseelan, Software-defined wireless mesh networks for internet access sharing, Comput. Netw. 93 (2015), 359-372.   DOI
2 M. Bano, S. S. A. Gilani, and A. Qayyum, A comparative analysis of hybrid routing schemes for SDN based wireless mesh networks, (IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems, Exeter, UK), 2018, pp. 1189-1194.
3 A. Varga, Omnet++ discrete event simulation system, release 4.6, 2015. [Online]. Available: http://www.omnetpp.org
4 A. Virdis and M. Kirsche, Recent advances in network simulation-The OMNeT++ environmentand its ecosystem, Springer Nature Switzerland AG, 2019.
5 S. Khan, A.-S. K. Pathan, and N. A. Alrajeh, Wireless sensor networks - current status and future trends, CRC Press, 2012.
6 V. Ramamurthi, A. S. Reaz, D. G. S. Dixit, and B. Mukherjee, Channel, capacity, and flow assignment in wireless mesh networks, Comput. Netw. 55 (2011), 2241-2258.   DOI
7 A.-S. K. Pathan, M. M. Monowar, and S. Khan, Simulation technologies in networking and communications-Selecting the best tool for the test, CRC Press, Taylor & Francis Group, LLC, 2015.
8 C. J. Bernardos, A. de la Oliva, P. Serrano, A. Banchs, L. M. Contreras, H. Jin, and J. C. Zuniga, An architecture for software defined wireless networking, IEEE Wirel. Commun. 21 (2014), no. 3, 52-61.
9 T. M. Mitchell, Machine Learning, Science - Engineering - Math, McGraw-Hill, 1997.
10 Z. Mammeri, Reinforcement learning based routing in networks: Review and classification of approaches, IEEE Access 7 (2019), 55916-55950.   DOI
11 M. Boushaba, A. Hafid, A. Belbekkouche, and M. Gendreau, Reinforcement learning based routing in wireless mesh networks, Wirel. Netw. 19 (2013), no. 8, 2079-2091.   DOI
12 Y. Wang, M. Martonosi, and L.-S. Peh, Predicting link quality using supervised learning in wireless sensor networks, Mobile Comput. Commun. Rev. 11 (2007), no. 3, 71-83.
13 A. Woo, T. Tong, and D. Culler, Taming the underlying challenges of reliable multihop routing in sensor networks, (Proceedings of The 1st ACM International Conference on Embedded Networked Sensor Systems, Los Angeles, CA, USA), 2003, pp. 14-27.
14 K. Singh and J. Kaur, Machine learning based link cost estimation for routing optimization in wireless sensor networks, Adv. Wirel. Mobile Commun. 10 (2017), no. 1, 39-49.
15 T.-V. T. Duong, L. H. Binh, and V. M. Ngo, Reinforcement learning for QoS-guaranteed intelligent routing in Wireless Mesh Networks with heavy traffic load, ICT Express 8 (2022), no. 1, 18-24. https://doi.org/10.1016/j.icte.2022.01.017   DOI
16 DARPA, The network simulator NS2. [Online]. Available: http://www.isi.edu
17 A. R. Syed, K. A. Yau, J. Qadir, H. Mohamad, N. Ramli, and S. L. Keoh, Route selection for multi-hop cognitive radio networks using reinforcement learning: An experimental study, IEEE Access 4 (2016), 6304-6324.   DOI
18 C. Yu, J. Lan, Z. Guo, and Y. Hu, Drom: Optimizing the routing in software-defined networks with deep reinforcement learning, IEEE Access 6 (2018), 64533-64539.   DOI
19 M. Yin, J. Chen, X. Duan, B. Jiao, and Y. Lei, QEBR: Q-learning based routing protocol for energy balance in wireless mesh networks, (IEEE 4th International Conference on Computer and Communications, Chengdu, China), 2018, pp. 280-284.
20 M. Boushaba, A. Hafid, and A. Belbekkouche, Reinforcement learning-based best path to best gateway scheme for wireless mesh networks, (IEEE 7th International Conference on Wireless and Mobile Computing, Networking and Communications, Shanghai, China), 2011, pp. 373-379.
21 T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, Continuous control with deep reinforcement learning, arXiv preprint, 2015. https://doi.org/10.48550/arXiv.1509.02971   DOI
22 J. Rischke, P. Sossalla, H. Salah, F. H. P. Fitzek, and M. Reisslein, QR-SDN: Towards reinforcement learning states, actions, and rewards for direct flow routing in software-defined networks, IEEE Access 8 (2020), 174773-174791.   DOI
23 C. Cung Trong, V. Tu, and N. Hai, An innovative solution of DSR routing mechanism based on mobile agent in MANET networks, J. Comput. Sci. Cybern. 29 (2013), no. 1, 31-42.
24 T. Liu and W. Liao, Location-dependent throughput and delay in wireless mesh networks, IEEE Trans. Vehic. Technol. 57 (2008), no. 2, 1188-1198.   DOI
25 C. T. Cuong, V. T. Tu, and N. T. Hai, MAR-AODV: Innovative routing algorithm in MANET based on mobile agent, (27th International Conference on Advanced Information Networking and Applications Workshops, Barcelona, Spain), 2013, pp. 62-66.
26 L. H. Binh and T.-V. T. Duong, Load balancing routing under constraints of quality of transmission in mesh wireless network based on software defined networking, J. Commun. Netw. 23 (2021), no. 1, 12-22.   DOI
27 L. H. Binh and V. T. Tu, QTA-AODV: An improved routing algorithm to guarantee quality of transmission for mobile ad hoc networks using cross-layer model, J. Commun. 13 (2018), no. 7, 338-349.
28 A. Abdelaziz, A. Fong, A. Gani, S. Khan, F. Alotaibi, and M. Khan, On software-defined wireless network (SDWN) network virtualization: Challenges and open issues, Comput. J. 60 (2017), 1510-1519.   DOI