Browse > Article
http://dx.doi.org/10.3837/tiis.2022.09.006

Reinforcement Learning-Based Intelligent Decision-Making for Communication Parameters  

Xie, Xia. (Dept. of Information and Communication Eng., Harbin Engineering University)
Dou, Zheng (Dept. of Information and Communication Eng., Harbin Engineering University)
Zhang, Yabin (Dept. of Information and Communication Eng., Harbin Engineering University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.9, 2022 , pp. 2942-2960 More about this Journal
Abstract
The core of cognitive radio is the problem concerning intelligent decision-making for communication parameters, the objective of which is to find the most appropriate parameter configuration to optimize transmission performance. The current algorithms have the disadvantages of high dependence on prior knowledge, large amount of calculation, and high complexity. We propose a new decision-making model by making full use of the interactivity of reinforcement learning (RL) and applying the Q-learning algorithm. By simplifying the decision-making process, we avoid large-scale RL, reduce complexity and improve timeliness. The proposed model is able to find the optimal waveform parameter configuration for the communication system in complex channels without prior knowledge. Moreover, this model is more flexible than previous decision-making models. The simulation results demonstrate the effectiveness of our model. The model not only exhibits better decision-making performance in the AWGN channels than the traditional method, but also make reasonable decisions in the fading channels.
Keywords
reinforcement learning; decision-making; Q-learning; cognitive radio; adaptive modulation and coding;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. G. Bilen, R. C. Reinhart, T. M. Hackett, R. Paffenroth, P. V. R. Ferreria, D.J. Mortensen, and A. M. Wyglinski, "Multi-objective reinforcement learning-based deep neural networks for cognitive space communications," in Proc. of Cognitive Communications for Aerospace Applications Workshop, pp. 1-8, 2017.
2 T. J. O'Shea, T. Roy and N. West, "Approximating the Void: Learning Stochastic Channel Models from Observation with Variational Generative Adversarial Networks," in Proc. of 2019 International Conference on Computing, Networking and Communications (ICNC), pp. 681-686, 2019.
3 A. Ikami, T. Hayashi and Y. Amano, "Dynamic Channel Allocation Algorithm for Spectrum Sharing between Different Radio Systems," in Proc. of 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, pp. 1-6, 2020.
4 B Li, C Ju, H Yang and G Liu, "Adaptive coded modulation based on LDPC codes," in Proc. of 2015 10th International Conference on Communications and Networking in China(China Com), pp. 648-651, 2015. .
5 W. Jouini, C. Moy, and J. Palicot, "Decision making for cognitive radio equipment: analysis of the first 10 years of exploration," EURASIP Journal on Wireless Communications and Networking, vol. 2012, no. 1, p. 26, 2012.
6 Passino K M., "Bacterial foraging optimization," International Journal of Swarm Intelligence Research (IJSIR), 1(1), 1-16, 2010.   DOI
7 R. S. Sutton and A. G. Barto, "Reinforcement Learning: An Introduction," IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 1054-1054, Sept. 1998.
8 A. Amanna, D. Ali, D. G. Fitch and J. H. Reed, "Design of experiments based empirical models to support cognitive radio decision making," in Proc. of 2015 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), pp. 1-7, 2015.
9 M. Bkassiny, Y. Li and S. K. Jayaweera, "A Survey on Machine-Learning Techniques in Cognitive Radios," IEEE Communications Surveys & Tutorials, vol. 15, no. 3, pp. 1136-1159, Third Quarter 2013.   DOI
10 L. Gavrilovska, V. Atanasovski, I. Macaluso and L. A. DaSilva, "Learning and Reasoning in Cognitive Radio Networks," IEEE Communications Surveys & Tutorials, vol. 15, no. 4, pp. 1761- 1777, Fourth Quarter 2013.   DOI
11 S. Wang, H. Liu, P. H. Gomes and B. Krishnamachari, "Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks," IEEE Transactions on Cognitive Communications and Networking, vol. 4, no. 2, pp. 257-265, June 2018.   DOI
12 J. Mitola and G. Q. Maguire, "Cognitive radio: making software radios more personal," IEEE Personal Communications, vol. 6, no. 4, pp. 13-18, Aug. 1999.   DOI
13 J. Mitola, "Software radios: Survey, critical evaluation and future directions," IEEE Aerospace and Electronic Systems Magazine, vol. 8, no. 4, pp. 25-36, April 1993.   DOI
14 Y. Yang, H. Jiang, C. Liu and Z. Lan, "Research on Cognitive Radio Engine Based on Genetic Algorithm and Radial Basis Function Neural Network," in Proc. of 2012 Spring Congress on Engineering and Technology, pp. 1-5, 2012.
15 R. Daniels and R. W. Heath, "Online adaptive modulation and coding with support vector machines," in Proc. of 2010 European Wireless Conference (EW), pp. 718-724, 2010.
16 Z. Dou, Y. Dong, and C. Li, "Intelligent decision modeling for communication parameter selection via back propagation neural network," in Proc. of International Conference on Advanced Hybrid Information Processing, Springer, pp. 465-472, 2017.
17 X Dong, Y Li, C Wu and Y Cai, "A learner based on neural network for cognitive radio," in Proc. of 2010 IEEE 12th International Conference on Communication Technology, pp. 893-896, 2010.
18 L. Zhang, M. Zhao, C. Tan, G. Li and C. Lv, "Research on Spectrum Sensing System Based on Composite Neural Network," in Proc. of 2020 2nd International Conference on Advances in Computer Technology, Information Science and Communications (CTISC), pp. 22-26, 2020.
19 Y. Yang, Q. Zhang, Y. Wang, T. Emoto, M. Akutagawa and S. Konaka, "Multi-strategy dynamic spectrum access in cognitive radio networks: Modeling, analysis and optimization," China Communications, vol. 16, no. 3, pp. 103-121, March 2019.   DOI
20 S. Haykin, "Cognitive radio: brain-empowered wireless communications," IEEE Journal on Selected Areas in Communications, vol. 23, no. 2, pp. 201-220, Feb. 2005.   DOI
21 A. S. Margulies and J. Mitola, "Software defined radios: a technical challenge and a migration strategy," in Proc. of 1998 IEEE 5th International Symposium on Spread Spectrum Techniques and Applications - Proceedings. Spread Technology to Africa (Cat. No.98TH8333), vol.2, pp. 551-556, 1998.
22 C. J. Rieser, "Biologically inspired cognitive radio engine model utilizing distributed genetic algorithms for secure and robust wireless communications and networking," Ph.D. dissertation, Virginia Tech, 2004.
23 Xu. H and Zhou. Z, "Cognitive radio decision engine using hybrid binary particle swarm optimization," in Proc. of 2013 13th International Symposium on Communications and Information Technologies (ISCIT), pp. 143-147, 2013.
24 Xu H, Zhou Z, "Hill-climbing genetic algorithm optimization in cognitive radio decision engine," in Proc. of 15th IEEE International Conference on Communication Technology, pp. 115-119, 2013.
25 P. M. Pradhan and G. Panda, "Comparative performance analysis of evolutionary algorithm based parameter optimization in cognitive radio engine: A survey," Ad Hoc Networks, vol. 17, pp. 129-146, 2014.   DOI
26 N. Abbas, Y. Nasser, and K. E. Ahmad, "Recent advances on artificial intelligence and learning techniques in cognitive radio networks," EURASIP Journal on Wireless Communications and Networking, vol. 2015, no. 1, p. 174, 2015.
27 S. Tubachi, M. Venkatesan and A. V. Kulkarni, "Predictive learning model in cognitive radio using reinforcement learning," in Proc. of 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI), pp. 564-567, 2017.
28 R. Bruno, A. Masaracchia and A. Passarella, "Robust Adaptive Modulation and Coding (AMC) Selection in LTE Systems Using Reinforcement Learning," in Proc. of 2014 IEEE 80th Vehicular Technology Conference (VTC2014-Fall), pp. 1-6, 2014.
29 J. P. Leite, P. H. P. de Carvalho and R. D. Vieira, "A flexible framework based on reinforcement learning for adaptive modulation and coding in OFDM wireless systems," in Proc. of 2012 IEEE Wireless Communications and Networking Conference (WCNC), pp. 809-814, 2012.
30 P. Ferreira, R. Paffenroth, A. Wyglinski, T. M. Hackett, S. Bilen, R. Rein- hart, and D. Mortensen, "Multi-objective reinforcement learning for cognitive radio-based satellite communications," in Proc. of 34th AIAA International Communications Satellite Systems Conference, p.5726, 2016.
31 C Yu, X Wen, X Lin and Z Wei, "Research on the modulation and coding scheme in LTE TDD wireless network," in Proc. of 2009 International Conference on Industrial Mechatronics and Automation, pp. 468-471, 2009.