Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Reinforcement Learning for Node-disjoint Path Problem in Wireless Ad-hoc Networks

무선 애드혹 네트워크에서 노드분리 경로문제를 위한 강화학습

  • Jang, Kil-woong (Department of Data Informatics, Korea Maritime and Ocean University)
  • Received : 2019.05.09
  • Accepted : 2019.07.03
  • Published : 2019.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes reinforcement learning to solve the node-disjoint path problem which establishes multipath for reliable data transmission in wireless ad-hoc networks. The node-disjoint path problem is a problem of determining a plurality of paths so that the intermediate nodes do not overlap between the source and the destination. In this paper, we propose an optimization method considering transmission distance in a large-scale wireless ad-hoc network using Q-learning in reinforcement learning, one of machine learning. Especially, in order to solve the node-disjoint path problem in a large-scale wireless ad-hoc network, a large amount of computation is required, but the proposed reinforcement learning efficiently obtains appropriate results by learning the path. The performance of the proposed reinforcement learning is evaluated from the viewpoint of transmission distance to establish two node-disjoint paths. From the evaluation results, it showed better performance in the transmission distance compared with the conventional simulated annealing.

본 논문은 무선 애드혹 네트워크에서 신뢰성이 보장되는 데이터 전송을 위해 다중 경로를 설정하는 노드분리 경로문제를 해결하기 위한 강화학습을 제안한다. 노드분리 경로문제는 소스와 목적지사이에 중간 노드가 중복되지 않게 다수의 경로를 결정하는 문제이다. 본 논문에서는 기계학습 중 하나인 강화학습에서 Q-러닝을 사용하여 노드의 수가 많은 대규모의 무선 애드혹 네트워크에서 전송거리를 고려한 최적화 방법을 제안한다. 특히 대규모의 무선 애드혹 네트워크에서 노드분리 경로 문제를 해결하기 위해서는 많은 계산량이 요구되지만 제안된 강화학습은 효율적으로 경로를 학습함으로써 적절한 결과를 도출한다. 제안된 강화학습의 성능은 2개의 노드분리경로를 설정하기 위한 전송거리 관점에서 평가되었으며, 평가 결과에서 기존에 제안된 시뮬레이티드 어널링과 비교평가하여 전송거리면에서 더 좋은 성능을 보였다.

Keywords

References

  1. G. Lin, S. Soh, K. Chin, and M. Lazarescu, "Efficient heuristics for energy-aware routing in networks with bundled links," Computer Networks, vol. 57 no. 8, pp. 1774-1788, 2013. https://doi.org/10.1016/j.comnet.2013.03.006
  2. D. Zhang, Q. Liu, L. Chen, W. Xu, and K. Wang, "Multi-layer based multi-path routing algorithm for maximizing spectrum availability," Wireless Networks, vol. 24 no. 3, pp. 897-909, 2018. https://doi.org/10.1007/s11276-016-1377-x
  3. R. Hadid, M. H. Karaata, and V. Villain, "A Stabilizing Algorithm for Finding Two Node-Disjoint Paths in Arbitrary Networks," International Journal of Foundations of Computer Science, vol. 28, no. 4, pp. 411-435, 2017. https://doi.org/10.1142/S0129054117500253
  4. M. A. Alsheikh, S. Lin, D. Niyato, and H. P. Tan, "Machine learning in wireless sensor networks: algorithms, strategies, and applications", IEEE Communications Surveys and Tutorials, vol. 16. no. 4, pp. 1996-2018, April 2014. https://doi.org/10.1109/COMST.2014.2320099
  5. K. Xiong, Z. Qiu, Y. Guo, and H. Zhang, "Multi-constrained shortest disjoint paths for reliable QoS routing," ETRI Journal, vol. 31, no. 5, pp. 534-44, 2009. https://doi.org/10.4218/etrij.09.0108.0531
  6. S. Kim, "Adaptive MANET multipath routing algorithm based on the simulated annealing approach," Scientific World Journal, vol. 2014, pp. 1-8, 2014.
  7. G. Lin, S. Soh, K. W. Chin, and M. Lazarescu, "Energy aware two disjoint paths routing," Journal of Network and Computer Applications, vol. 43, pp. 27-41, 2014. https://doi.org/10.1016/j.jnca.2014.04.004
  8. C. Hsu and H. Cho, "A genetic algorithm for the maximum edge-disjoint paths problem," Neurocomputing, vol. 148, pp. 17-22, 2015. https://doi.org/10.1016/j.neucom.2012.10.046
  9. W. Fisher, M. Suchara, and J. Rexford, "Greening back bone networks: reducing energy consumption by shutting off cables in bundled links," in Proceeding of ACM SIGCOMM Workshops on Green Networking, pp. 29-34, 2010.
  10. M. Kurt, M. Berberler, and O. Ugurlu, "A new algorithm for finding vertex-disjoint paths," The International Arab Journal of Information Technology, vol. 12, no. 6, pp.550-555, 2015.
  11. A. Forster and A. L. Murphy, "FROMS: feedback routing for optimizing multiple sinks in WSN with reinforcement learning", in Proceeding of IEEE Intelligent Sensors, Sensor Networks and Information Processing Conference, pp. 371-376, 2007.
  12. S. Li, X. Kong, and D. Lowe, "Dynamic Path Determination of Mobile Beacons Employing Reinforcement Learning for Wireless Sensor Localization", in Proceeding of IEEE International Conference on Advanced Information Networking and Applications Workshops, pp. 760-765, 2012.
  13. Y. Yu, T. Wang, and S. C. Liew, "Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks", in Proceeding of IEEE International Conference on Communications, pp. 1-7, 2018.

Cited by

  1. Deep Q 학습 기반의 다중경로 시스템 경로 선택 알고리즘 vol.25, pp.1, 2021, https://doi.org/10.6109/jkiice.2021.25.1.50