Browse > Article
http://dx.doi.org/10.9717/kmms.2022.25.2.145

Digital Twin and Visual Object Tracking using Deep Reinforcement Learning  

Park, Jin Hyeok (Dept. of IT Convergence and Application Engineering, PuKyong National University)
Farkhodov, Khurshedjon (Dept. of IT Convergence and Application Engineering, PuKyong National University)
Choi, Piljoo (Dept. of IT Convergence and Application Engineering, PuKyong National University)
Lee, Suk-Hwan (Dept. of Computer Engineering, Dong-A University)
Kwon, Ki-Ryong (Dept. of IT Convergence and Application Engineering, PuKyong National University)
Publication Information
Journal of Korea Multimedia Society / v.25, no.2, 2022 , pp. 145-156 More about this Journal
Abstract
Nowadays, the complexity of object tracking models among hardware applications has become a more in-demand duty to complete in various indeterminable environment tracking situations with multifunctional algorithm skills. In this paper, we propose a virtual city environment using AirSim (Aerial Informatics and Robotics Simulation - AirSim, CityEnvironment) and use the DQN (Deep Q-Learning) model of deep reinforcement learning model in the virtual environment. The proposed object tracking DQN network observes the environment using a deep reinforcement learning model that receives continuous images taken by a virtual environment simulation system as input to control the operation of a virtual drone. The deep reinforcement learning model is pre-trained using various existing continuous image sets. Since the existing various continuous image sets are image data of real environments and objects, it is implemented in 3D to track virtual environments and moving objects in them.
Keywords
Object Tracking; Object Detection; Deep Learning; Deep Reinforcement Learning; Deep Q-Network; DQN; Virtual Drone; AirSim;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 J.F. Henriques, R. Caseiro, P. Martins, and J. Batista, "High Speed Tracking with Kernelized Correlation Filters," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, No. 3, pp. 583-596, 2015.   DOI
2 M. Danelljan, G. Hager, F. Shahbaz Khan, and M. Felsberg, "Convolutional Features for Correlation Filter Based Visual Tracking," Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 58-66, 2015.
3 J. Xie. E. Stensrud, and T. Skramstad, "Detection-Based Object Tracking Applied to Remote Ship Inspection," Sensors, Vol. 21, No. 3, 761, 2021.   DOI
4 L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. Torr. "Fully-Convolutional Siamese Networks for Object Tracking," Proceeding of the European Conference on Computer Vision, pp. 850-865, 2016.
5 B. Babenko, M.-H. Yang, and S. Belongie. "Robust Object Tracking with Online Multiple Instance Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 8, pp. 1619-1632, 2011.   DOI
6 X. Farhodov, O.-H. Kwon, K.-S. Moon, O.-J. Kwon, S.-H. Lee, and K.-R. Kwon. "A New CSR-DCF Tracking Algorithm Based on Faster RCNN Detection Model and CSRT Tracker for Drone Data," Journal of Korea Multimedia Society, Vol. 22, No. 12, pp. 1415-1429, 2019.   DOI
7 R.S. Sutton. Introduction to Reinforcement Learning, The MIT Press, Cambridge, Massachusetts, London, England, 2015.
8 J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. "Long-Term Recurrent Convolutional Networks for Visual Recognition and Description," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2625-2634, 2015.
9 N.O. Salscheider, "Object Tracking by Detection with Visual Motion Cues," arXiv Preprint, arXiv:2101.07549, 2021.
10 S. Shah, D. Dey, C. Lovett, and A. Kapoor, "AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles," Proceeding of the 11th Conference on Field and Service Robotics, pp. 621-635, 2018.
11 J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. "You Only Look Once: Unified, Real-time Object Detection," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
12 G. Ning, Z. Zhang, C. Huang, Z. He, X. Ren, and H. Wang. "Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking," Proceedings of the IEEE International Symposium on Circuits and Systems, pp. 1-4, 2017.
13 J.C. Caicedo and S. Lazebnik. "Active Object Localization with Deep Reinforcement Learning," Proceedings of the IEEE International Conference on Computer Vision, pp. 2488-2496, 2015.
14 D. Jayaraman and K. Grauman. "Look-Ahead before You Leap: End-to-End Active Recognition by Forecasting the Effect of Motion," Proceedings of the European Conference on Computer Vision, pp. 489-505, 2016.
15 V. Mnih, K. Koray, D. Silver, A.A. Rusu, J. Veness, M.G. Bellemare, et al., "Human-Level Control through Deep Reinforcement Learning," Nature, Vol. 518, pp. 529-533, 2015.   DOI