Browse > Article
http://dx.doi.org/10.6109/jkiice.2021.25.9.1244

Determining Whether to Enter a Hazardous Area Using Pedestrian Trajectory Prediction Techniques and Improving the Training of Small Models with Knowledge Distillation  

Choi, In-Kyu (Intelligent Image Processing Research Center, Korea Electronics Technology Institute)
Lee, Young Han (Intelligent Image Processing Research Center, Korea Electronics Technology Institute)
Song, Hyok (Intelligent Image Processing Research Center, Korea Electronics Technology Institute)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.25, no.9, 2021 , pp. 1244-1253 More about this Journal
Abstract
In this paper, we propose a method for predicting in advance whether pedestrians will enter the hazardous area after the current time using the pedestrian trajectory prediction method and an efficient simplification method of the trajectory prediction network. In addition, we propose a method to apply KD(Knowledge Distillation) to a small network for real-time operation in an embedded environment. Using the correlation between predicted future paths and hazard zones, we determined whether to enter or not, and applied efficient KD when learning small networks to minimize performance degradation. Experimentally, it was confirmed that the model applied with the simplification method proposed improved the speed by 37.49% compared to the existing model, but led to a slight decrease in accuracy. As a result of learning a small network with an initial accuracy of 91.43% using KD, It was confirmed that it has improved accuracy of 94.76%.
Keywords
Pedestrian trajectory prediction; Model compression; Knowledge distillation; Prediction of entry into hazardous areas;
Citations & Related Records
연도 인용수 순위
  • Reference
1 G. Chen, W. Choi, X. Yu, T. Han, and M. Chandraker, "Learning efficient object detection models with knowledge distillation," in Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 742-751, Dec. 2017.
2 B. Zhou, X. Wang, and X. Tang, "Understanding collective crowd behaviors: Learning a mixture model of dynamic pedestrian-agents," in Computer Vision and Pattern Recognition (CVPR), pp. 2871-2878, 2012.
3 C. Zhang, H. Li, X. Wang, and X. Yang, "Cross-scene crowd counting via deep convolutional neural networks," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 833-841, 2015.
4 R. Mehran, A. Oyama, and M. Shah, "Abnormal crowd behavior detection using social force model," in Computer Vision and Pattern Recognition, pp. 935-942, 2009.
5 S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding," arXiv preprint arXiv: 1510.00149, 2015.
6 B. Cancela, A. Iglesias, M. Ortega, and M. G. Penedo, "Unsupervised trajectory modelling using temporal information via minimal paths," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2553-2560, 2014.
7 D. Lin, S. Talathi, and S. Annapureddy, "Fixed point quantization of deep convolutional networks," in International conference on machine learning, pp. 2849-2858, June. 2016.
8 A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, "Social lstm: Human trajectory prediction in crowded spaces," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 961-971, 2016
9 M. R. U. Saputra, P. P. de Gusmao, Y. Almalioglu, A. Markham, and N. Trigoni, "Distilling knowledge from a deep pose regressor network," in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 263-272, 2019.
10 S. H. Lee, D. H. Kim, and B. C. Song, "Self-supervised knowledge distillation using singular value decomposition," in Proceedings of the European Conference on Computer Vision (ECCV), pp. 335-350, 2018.
11 H. Wang, H. Zhao, X. Li and X. Tan, "Progressive Blockwise Knowledge Distillation for Neural Network Acceleration," in International Joint Conference on Artificial Intelligence, pp. 2769-2775, Jan. 2018.
12 Y. Xu, Z. Piao, and S. Gao, "Encoding crowd interaction with deep neural network for pedestrian trajectory prediction," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5275-5284, 2018.
13 X. Wang, X. Ma, and W. E. L. Grimson, "Unsupervised activity perception in crowded and complicated scenes using hierarchical bayesian models," IEEE Transactions on pattern analysis and machine intelligence, vol. 31, no. 3, pp. 539-555, 2009.   DOI
14 M. Takamoto, Y. Morishita, and H. Imaoka, "An efficient method of training small models for regression problems with knowledge distillation," in 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 67-72, Aug. 2020.
15 A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, "Fitnets: Hints for thin deep nets," arXiv preprint arXiv:1412.6550, 2014.
16 A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, and H. Adam, "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017.
17 S. Yi, H. Li, and X. Wang, "Pedestrian behavior understanding and prediction with deep neural networks," in European Conference on Computer Vision, pp. 263-279, Oct. 2016.
18 P. Zhang, W. Ouyang, P. hang, J. Xue, and N. Zheng, "Sr-lstm: State refinement for lstm towards pedestrian trajectory prediction," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12085-12094, 2019.
19 J. Amirian, J. B. Hayet, and J. Pettre, "Social ways: Learning multi-modal distributions of pedestrian trajectories with gans," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
20 G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," arXiv preprint arXiv: 1503.02531, 2015.
21 D. Lopez-Paz, L. Bottou, B. Scholkopf, and V. Vapnik, "Unifying distillation and privileged information," arXiv preprint arXiv:1511.03643, 2015.
22 A. Polino, R. Pascanu, and D. Alistarh, "Model compression via distillation and quantization," arXiv preprint arXiv: 1802.05668, 2018.
23 S. Yi, H. Li, and X. Wang, "Understanding pedestrian behaviors from stationary crowd groups," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3488-3496, 2015.
24 J. Yim, D. Joo, J. Bae, and J. Kim, "A gift from knowledge distillation: Fast optimization, network minimization and transfer learning," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4133-4141, 2017.
25 M. Kang and S. Kang, "Data-free knowledge distillation in neural networks for regression," Expert Systems with Applications, vol. 175, no. 114813, 2021.
26 J. Yang, B. Marinez, S. A. Center, A. Bulat and G. Tzimiropoulos, "Knowledge distillation via softmax regression representation learning," International Conference on Learning Representations (ICLR), May. 2020.
27 K. Sharma and D. D. Londhe, "Human Safety Devices Using IoT and Machine Learning: A Review," in 2018 3rd International Conference for Convergence in Technology (I2CT), pp. 1-7, Apr. 2018.
28 A. I. Maqueda, A. Loquercio, G. Gallego, N. Garcia, and D. Scaramuzza, "Event-based vision meets deep learning on steering prediction for self-driving cars," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5419-5427, 2018.
29 Y. Huang and Y. Chen, "Survey of State-of-Art Autonomous Driving Technologies with Deep Learning," in 2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C), pp. 221-228, Dec. 2020.
30 D. Tabernik, S. Sela, J. Skvarc, and D. Skocaj, "Segmentationbased deep-learning approach for surface-defect detection," Journal of Intelligent Manufacturing, vol. 31, no. 3, pp. 759-776, 2020.   DOI
31 M. Rudolph, B. Wandt, and B. Rosenhahn, "Same same but differnet: Semi-supervised defect detection with normalizing flows," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1907-1916, 2021.
32 J. Yuan and C. Guo, "A deep learning method for detection of dangerous equipment," in 2018 Eighth International Conference on Information Science and Technology (ICIST), pp. 159-164, June. 2018.