Browse > Article
http://dx.doi.org/10.22156/CS4SMB.2020.10.06.001

Q-NAV: NAV Setting Method based on Reinforcement Learning in Underwater Wireless Networks  

Park, Seok-Hyeon (Department of Computer Science, Chungbuk National University)
Jo, Ohyun (Department of Computer Science, Chungbuk National University)
Publication Information
Journal of Convergence for Information Technology / v.10, no.6, 2020 , pp. 1-7 More about this Journal
Abstract
The demand on the underwater communications is extremely increasing in searching for underwater resources, marine expedition, or environmental researches, yet there are many problems with the wireless communications because of the characteristics of the underwater environments. Especially, with the underwater wireless networks, there happen inevitable delay time and spacial inequality due to the distances between the nodes. To solve these problems, this paper suggests a new solution based on ALOHA-Q. The suggested method use random NAV value. and Environments take reward through communications success or fail. After then, The environments setting NAV value from reward. This model minimizes usage of energy and computing resources under the underwater wireless networks, and learns and setting NAV values through intense learning. The results of the simulations show that NAV values can be environmentally adopted and select best value to the circumstances, so the problems which are unnecessary delay times and spacial inequality can be solved. Result of simulations, NAV time decreasing 17.5% compared with original NAV.
Keywords
Reinforcement learning; Q-learning; ALOHA-Q; Underwater Wireless Network; Underwater IoT Network; Sensor; RTS; CTS; NAV;
Citations & Related Records
연도 인용수 순위
  • Reference
1 X. Guo, M. R. Frater & M. J. Ryan. (2009). Design of a propagation-delay-tolerant MAC protocol for underwater acoustic sensor networks. IEEE Journal of Oceanic Engineering, 34(2), 170-180. DOI : 10.1109/JOE.2009.2015164   DOI
2 D. Shin & D. Kim. (2008, October). A dynamic NAV determination protocol in 802.11 based underwater networks. In 2008 IEEE International Symposium on Wireless Communication Systems (pp. 401-405). IEEE.
3 Y. D. Chen, C. C. Li, R. T. Dai & K. P. Shih. (2011, September). On enhancing four-way handshake with stair-like NAV setting for underwater acoustic networks. In OCEANS'11 MTS/IEEE KONA (pp. 1-6). IEEE.
4 Y. Chu, S. Kosunalp, P. D. Mitchell, D. Grace & T. Clarke. (2015). Application of reinforcement learning to medium access control for wireless sensor networks. Engineering Applications of Artificial Intelligence, 46, 23-32. DOI : 10.1016/j.engappai.2015.08.004   DOI
5 S. H. Park, P. D. Mitchell & D. Grace. (2019). Reinforcement Learning Based MAC Protocol (UW-ALOHA-Q) for Underwater Acoustic Sensor Networks. IEEE Access, 7, 165531-165542.   DOI
6 T. Lee & O. J. Shin. (2020). CoRL: Collaborative Reinforcement Learning-Based MAC Protocol for IoT Networks. Electronics, 9(1), 143. DOI : 10.3390/electronics9010143   DOI
7 IEEE Computer Society LAN MAN Standards Committee. (1999). Wireless LAN medium access control (MAC) and physical layer (PHY) specifications. ANSI/IEEE Std. 802. 11-1999.
8 I. F. Akyildiz, D. Pompili & T. Melodia. (2004). Challenges for efficient communication in underwater acoustic sensor networks. ACM Sigbed Review, 1(2), 3-8. DOI : 10.1145/1121776.1121779   DOI
9 https://water.usgs.gov/edu/earthhowmuch.html.
10 E. M. ozer, M. Stojanovic & J. G. Proakis. (2000). Underwater acoustic networks. IEEE journal of oceanic engineering, 25(1), 72-83. DOI : 10.1109/48.820738   DOI
11 F. Yunus, S. H. Ariffin & Y. Zahedi. (2010, May). A survey of existing medium access control (MAC) for underwater wireless sensor network (UWSN). In 2010 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation (pp. 544-549). IEEE.