KIPS Transactions on Computer and Communication Systems (정보처리학회논문지:컴퓨터 및 통신 시스템)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Equal Energy Consumption Routing Protocol Algorithm Based on Q-Learning for Extending the Lifespan of Ad-Hoc Sensor Network

애드혹 센서 네트워크 수명 연장을 위한 Q-러닝 기반 에너지 균등 소비 라우팅 프로토콜 기법

  • Received : 2021.01.18
  • Accepted : 2021.04.15
  • Published : 2021.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, smart sensors are used in various environments, and the implementation of ad-hoc sensor networks (ASNs) is a hot research topic. Unfortunately, traditional sensor network routing algorithms focus on specific control issues, and they can't be directly applied to the ASN operation. In this paper, we propose a new routing protocol by using the Q-learning technology, Main challenge of proposed approach is to extend the life of ASNs through efficient energy allocation while obtaining the balanced system performance. The proposed method enhances the Q-learning effect by considering various environmental factors. When a transmission fails, node penalty is accumulated to increase the successful communication probability. Especially, each node stores the Q value of the adjacent node in its own Q table. Every time a data transfer is executed, the Q values are updated and accumulated to learn to select the optimal routing route. Simulation results confirm that the proposed method can choose an energy-efficient routing path, and gets an excellent network performance compared with the existing ASN routing protocols.

최근 스마트 센서는 다양한 환경에서 사용되고 있으며, 애드혹 센서 네트워크 (ASN) 구현에 대한 연구가 활발하게 진행되고 있다. 그러나 기존 센서 네트워크 라우팅 알고리즘은 특정 제어 문제에 초점을 맞추며 ASN 작업에 직접 적용할 수 없는 문제점이 있다. 본 논문에서는 Q-learning 기술을 이용한 새로운 라우팅 프로토콜을 제안하는데, 제안된 접근 방식의 주요 과제는 균형 잡힌 시스템 성능을 확보하면서 효율적인 에너지 할당을 통해 ASN의 수명을 연장하는 것이다. 제안된 방법의 특징은 다양한 환경적 요인을 고려하여 Q-learning 효과를 높이며, 특히 각 노드는 인접 노드의 Q 값을 자체 Q 테이블에 저장하여 데이터 전송이 실행될 때마다 Q 값이 업데이트되고 누적되어 최적의 라우팅 경로를 선택하는 것이다. 시뮬레이션 결과 제안된 방법이 에너지 효율적인 라우팅 경로를 선택할 수 있으며 기존 ASN 라우팅 프로토콜에 비해 우수한 네트워크 성능을 얻을 수 있음을 확인하였다.

Keywords

Acknowledgement

본 연구는 과학기술정보통신부 및 정보통신기획평가원의 대학ICT연구센터 지원사업의 연구결과로 수행되었음(IITP-2021-2018-0-01799).

References

  1. Q. Sang, H. Wu, L. Xing, H. Ma, and P. Xie, "An energy-efficient opportunistic routing protocol based on trajectory prediction for FANETs," in IEEE Access, Vol.8, pp.192009-192020, 2020,. https://doi.org/10.1109/access.2020.3032956
  2. R. Priyadarshi, L. Singh, A. Singh, and A. Thakur, "SEEN: Stable energy efficient network for wireless sensor network," 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), Noida, India, pp.338-342, 2018.
  3. B. S. Rani and K. Shyamala, "Energy efficient load balancing approach for multipath routing protocol in Ad Hoc networks," 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP), Gangtok, India, pp.1-5, 2019.
  4. S. Din, A. Paul, A. Ahmad, and J. Kim, "Energy efficient topology management scheme based on clustering technique for software defined wireless sensor network," Peer-to-Peer Networking and Applications Volume, Vol.12, No.2, pp.348-356, 2019. https://doi.org/10.1007/s12083-017-0607-z
  5. R. S. Sutton and A. G. Barto, "Reinforcement learning: An introduction," MIT Press, 2018.
  6. K. Li, H. Huang, X. Gao, F. Wu, and G. Chen, "QLEC: A machine-learning-based energy-efficient clustering algorithm to prolong network lifespan for IoT in high-dimensional space," In Proceedings of the 48th International Conference on Parallel Processing (ICPP 2019), Association for Computing Machinery, pp.1-10, 2019.
  7. M. Zorzi and R. R. Rao, "Geographic random forwarding (GeRaF) for ad hoc and sensor networks: Multihop performance," in IEEE Transactions on Mobile Computing, Vol.2, No.4, pp.337-348, 2003. https://doi.org/10.1109/TMC.2003.1255648
  8. J. Bahi, W. Elghazel, C. Guyeux, M. Hakem, K. Medjaher and N. Zerhouni, "Reliable diagnostics using wireless sensor networks," Computers in Industry, Vol.104, pp.103-115, 2019. https://doi.org/10.1016/j.compind.2018.10.006
  9. Z. Mammeri, "Reinforcement learning based routing in networks: Review and classification of approaches," IEEE Access, Vol.7, pp.55916-55950, 2019. https://doi.org/10.1109/access.2019.2913776