Browse > Article
http://dx.doi.org/10.3745/KTSDE.2021.10.7.263

The Design and Practice of Disaster Response RL Environment Using Dimension Reduction Method for Training Performance Enhancement  

Yeo, Sangho (아주대학교 인공지능학과)
Lee, Seungjun (아주대학교 인공지능학과)
Oh, Sangyoon (아주대학교 소프트웨어학과)
Publication Information
KIPS Transactions on Software and Data Engineering / v.10, no.7, 2021 , pp. 263-270 More about this Journal
Abstract
Reinforcement learning(RL) is the method to find an optimal policy through training. and it is one of popular methods for solving lifesaving and disaster response problems effectively. However, the conventional reinforcement learning method for disaster response utilizes either simple environment such as. grid and graph or a self-developed environment that are hard to verify the practical effectiveness. In this paper, we propose the design of a disaster response RL environment which utilizes the detailed property information of the disaster simulation in order to utilize the reinforcement learning method in the real world. For the RL environment, we design and build the reinforcement learning communication as well as the interface between the RL agent and the disaster simulation. Also, we apply the dimension reduction method for converting non-image feature vectors into image format which is effectively utilized with convolution layer to utilize the high-dimensional and detailed property of the disaster simulation. To verify the effectiveness of our proposed method, we conducted empirical evaluations and it shows that our proposed method outperformed conventional methods in the building fire damage.
Keywords
Reinforcement Learning Environment; Disaster Response Simulation; Dimension Reduction Method; PCA;
Citations & Related Records
연도 인용수 순위
  • Reference
1 V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing atari with deep reinforcement learning," arXiv preprint arXiv:1312.5602, 2013.
2 B. R. Kiran, I, Sobh, V. Talpaert., P. Mannion, A. A. A. Sallab, S. Yogamani, and P. Perez, "Deep reinforcement learning for autonomous driving: A survey," arXiv preprint arXiv:2002.00444, 2020.
3 J. Boyan and M. Littman, "Packet routing in dynamically changing networks: A reinforcement learning approach," Advances in Neural Information Processing Systems, pp.671-678, 1994.
4 L. Nguyen, Z. Yang, J. Zhu, J. Li, and F. Jin, "Coordinating disaster emergency response with heuristic reinforcement learning," arXiv preprint arXiv:1811.05010, 2018
5 J. Sharma, P. A. Andersen, O. C. Granmo, and M. Goodwin, "Deep Q-Learning with Q-Matrix transfer learning for novel fire evacuation environment," IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020.
6 H. R. Lee and T. Lee, "Multi-agent reinforcement learning algorithm to solve a partially-observable multi-agent problem in disaster response," European Journal of Operational Research, Vol.291, No.1, pp.296-308, 2021.   DOI
7 C. Skinner and S. Ramchurn, "The robocup rescue simulation platform," In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1, pp.1647-1648, May 2010.
8 B. Kovalerchu, B. Agarwal, and D. C. Kalla, "Solving Nonimage Learning Problems by Mapping to Images," International Conference Information Visualization, pp.264-269, 2020.
9 L. Buturovic and D. Miljkovic, "A novel method for classification of tabular data using convolutional neural networks," BioRxiv, 2020.
10 O. Kramer, "Scikit-learn," In Machine Learning for Evolution Strategies, pp.45-53. Springer, Cham, 2016.
11 T. Korhonen and S. Hostikka, "Fire dynamics simulator with evacuation: Fds+ Evac: Technical reference and user's guide," 2009.
12 P. I. Wojcik and M. Kurdziel, "Training neural networks on high-dimensional data using random projection," Pattern Analysis and Applications, Vol.22, No.3, pp.1221-1231. 2019.   DOI
13 KISTI, NEURON computing environment [Internet], https://www.ksc.re.kr/mobile/ggspcpt/neuron
14 A. Sharma, E. Vans, D. Shigemizu, K. A. Boroevich, and T. Tsunoda, "DeepInsight: A methodology to transform a non-image data to an image for convolution neural network architecture," Scientific Reports, Vol.9, No.1, pp.1-7, 2019.   DOI
15 A. Sharma and D. Kumar, "Non-image data classification with convolutional neural networks," arXiv preprint arXiv: 2007.03218, 2020.
16 A. Goyal, "Multi-agent deep reinforcement learning for robocup rescue simulator," The Graduate School of The University of Texas at Austin, May 2020.
17 S. Mika, B. Scholkopf, A. Smola, K. R. Muller, M. Scholz, and G. Ratsch, "Kernel PCA and de-noising in feature spaces," Advances in Neural Information Processing Systems, Vol.11, pp.536-542, 1998.
18 scikit-learn, sklearn.preprocessing.MinMaxScaler [Internet], https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html
19 G. M. Kurtzer, V. Sochat, and M. W. Bauer, "Singularity: Scientific containers for mobility of compute," PloS one, Vol.12, No.5, e0177459, 2017.   DOI
20 A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, and A. Desmaison, "Pytorch: An imperative style, high-performance deep learning library," In Advances in Neural Information Processing Systems, pp.8026-8037, 2019.
21 G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, "Openai gym," arXiv preprint arXiv:1606.01540, 2016.