Journal of the Korean Society for Aeronautical & Space Sciences (한국항공우주학회지)

The Korean Society for Aeronautical and Space Sciences (한국항공우주학회)
권호 표지
DOI QR코드

DOI QR Code

A Study of Automatic Recognition on Target and Flame Based Gradient Vector Field Using Infrared Image

적외선 영상을 이용한 Gradient Vector Field 기반의 표적 및 화염 자동인식 연구

  • Received : 2020.06.26
  • Accepted : 2020.12.07
  • Published : 2021.01.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a algorithm for automatic target recognition robust to the influence of the flame in order to track the target by EOTS(Electro-Optical Targeting System) equipped on UAV(Unmanned Aerial Vehicle) when there is aerial target or marine target with flame at the same time. The proposed method converts infrared images of targets and flames into a gradient vector field, and applies each gradient magnitude to a polynomial curve fitting technique to extract polynomial coefficients, and learns them in a shallow neural network model to automatically recognize targets and flames. The performance of the proposed technique was confirmed by utilizing the various infrared image database of the target and flame. Using this algorithm, it can be applied to areas where collision avoidance, forest fire detection, automatic detection and recognition of targets in the air and sea during automatic flight of unmanned aircraft.

본 논문은 공중 혹은 해상배경에 표적과 화염이 동시에 존재할 때, 무인항공기에 장착된 EOTS(Electro-Optical Targeting System; 전자광학 추적장비)가 표적을 추적하기 위해 화염의 영향에 강건하도록 표적을 자동 인식하는 기법을 제안한다. 제안한 기법은 표적과 화염의 적외선 영상을 Gradient Vector Field로 변환하고, 각 Gradient magnitude를 Polynomial Curve Fitting 도구에 적용하여 다항식 계수를 추출 및 얕은 신경망 모델에 학습함으로써, 표적과 화염을 자동으로 인식한다. 확보한 표적 및 화염의 다양한 적외선 영상 DB를 학습데이터, 검증데이터, 시험데이터로 분류하여 제안한 기법의 표적 및 화염 자동 인식 성능을 확인하였다. 본 알고리듬을 활용하여 무인항공기의 자동비행 중 충돌회피, 산불탐지, 공중 및 해상의 목표물을 자동탐지 및 인식하는 분야에 적용될 수 있다.

Keywords

References

  1. Wee, J. H. and Hong, S. K., "Design of the Autopilot Algorithm for Unmanned Aerial Vehicle (UAV) & its Flight Test," Proceedings of the International Conference on Control, Automation and Systems, October 2001, pp. 2395-2399.
  2. Shakhatreh, H., Sawalmeh, A. H., Al-Fuqaha, A., Dou, Z., Almaita, E., Khalil, I., Othman, N. S., Khreishah, A. and Guizani, M., "Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges," IEEE Access, Vol. 7, 2019, pp. 48572-48634. https://doi.org/10.1109/access.2019.2909530
  3. Stojcsics, D., Somlyai, L. and Molnar, A., "Unmanned aerial vehicle guidance methods for 3D aerial image reconstruction," ICCC 2013.IEEE 9th International Conference on Computational Cybermetics, 2013, pp. 321-326.
  4. Lee, S. H. and Bang, H. C., "Research on the Vision-Based Landing Guidance of Rotorcraft Unmanned Aerial Vehicle using Deep Deterministic Policy Gradient," Proceeding of The Korean Society for Aeronautical and Space Sciences Spring Conference, April 2018, pp. 202-203.
  5. Hong, K. W. and Park, J. W., "Vision-based Recognition and Navigation for UAVs," Proceeding of The Korean Society for Aeronautical and Space Sciences Fall Conference, November 2013, pp. 1191-1194.
  6. Kim, C. H. and Bang, H. C., "Object Recognition using Information from an Aerial Image," Proceeding of The Korean Society for Aeronautical and Space Sciences Spring Conference, April 2014, pp. 623-626.
  7. Jeon, B. Y., Baek, K. Y. and Bang, H. C., "Target Detection and Tracking on Aerial Images from an UAV," Proceeding of The Korean Society for Aeronautical and Space Sciences Spring Conference, April 2013, pp. 425-428.
  8. Yu, D. Y., Won, D. Y. and Tahk, M. J., "Vision Based Collision Avoidance of Multi-Rotor UAVs using Optical Flow," Proceeding of The Korean Society for Aeronautical and Space Sciences Fall Conference, November 2010, pp. 494-497.
  9. Kang, M. H., Kim, M. J. and Kim, B. H., "Operational Concept and System Control Design of Electro-Optical Payload for UAV," Proceeding of The Korean Society for Aeronautical and Space Sciences Fall Conference, November 2014, pp. 1229-1232.
  10. Jeong, Y. Y. and Bang, H. C., "Onboard Target Recognition and Tracking Algorithm for UAV based Template Matching," Proceeding of The Korean Society for Aeronautical and Space Sciences Fall Conference, November 2012, pp. 582-588.
  11. Lee, S. H., Kim, S. H. and Choi, H. L., "Vision-Based Multi-Object Detection and Localization Using CNN," Proceeding of The Korean Society for Aeronautical and Space Sciences Fall Conference, November 2017, pp. 326-327.
  12. Gao, J. Y., Lin, Z. P. and An, A. W., "Infrared Small Target Detection Using a Temporal Variance and Spatial Patch Contrast Filter," IEEE Access, Vol. 7, 2019, pp. 32217-32226. https://doi.org/10.1109/access.2019.2903808
  13. Zhanga, X., Dingf, Q., Luoa, H., Huia, B., Changa, Z. and Zhanga, J., "Infrared small target detection based on local intensity and gradient properties," Elsevier, Vol. 99, 2019, pp. 55-63.
  14. Ozbay, M. and Sahingil, M., "A Fast and Robust Automatic Object Detection Algorithm to Detect Small Objects in Infrared Images," IEEE, 25th ISU, 2017, pp. 1-4.
  15. Lee, J., Lim, J., Baek, H., Kim, C., Park, J. and Koh, E., "A Design of Du-CNN based on the Hybrid Machine Characters to Classify Target and Clutter in The IR Image," Journal of the Korea Institute of Military Science and Technology, Vol. 20, No. 6, 2017, pp. 758-766. https://doi.org/10.9766/KIMST.2017.20.6.758
  16. Ryu, J. and Kim, S., "Heterogeneous Gray-Temperature Fusion-Based Deep Learning Architecture for Far Infrared Small Target Detection," Hindawi Journal of Sensors, 2019, pp. 1-15.
  17. Phillips, W., Shan, M. and Lobo, N. V., "Flame recognition in video," Pattern Recognition Letters, Vol. 23, No. 1-3, 2002, pp. 319-327. https://doi.org/10.1016/S0167-8655(01)00135-0
  18. Chen, T. H., Kao, C. L. and Chang, S. M., "An intelligent real-time fire-detection method based on video processing," Proceeding of the IEEE 37th Annual International Carnahan Conference on Security Technology, 2003, pp. 104-111.
  19. Toreyin, B. U., Dedeoglu, Y., Gudukbay, U. and E.Cetin, A., "Computer vision based method for real-time fire and flame detection," Pattern Recognition Letters, Vol. 27, No. 1, 2006, pp. 49-58. https://doi.org/10.1016/j.patrec.2005.06.015
  20. Chen, J., He, Y. and Wang, J., "Multi-feature fusionbased fast video flame detection," Building and Environment, Vol. 45, No. 5, 2010, pp. 1113-1122. https://doi.org/10.1016/j.buildenv.2009.10.017
  21. Celik, T. and Demirel, H., "Fire detection in video sequences using a generic color model," Fire Safety Journal, Vol. 44, No. 2, 2009, pp. 147-158. https://doi.org/10.1016/j.firesaf.2008.05.005
  22. Horng, W. B., Peng, J. W. and Chen, C. Y., "A new image-based real-time flame detection method using color analysis," Proceeding of the IEEE Networking, Sensing and Control, 2005, pp. 100-105.
  23. Verstockt, S., Vanoosthuyse, A., Hoecke, S. V., Lambert, P. and Van de Walle, R., "Multi-sensor fire detection by fusing visual and non-visual flame features," Proceedings of the 4th International Conference on Image and Signal Processing (ICISP 10), 2010, pp. 333-341.
  24. Toreyin, B. U., Cinbis, R. G., Dedeoglu, Y., Cetin, A. E., "Fire detection in infrared video using wavelet analysis," Optical Engineering, Vol. 46, No. 6, 067204, 2007, pp. 1-9. https://doi.org/10.1117/1.2748752
  25. Grimson, W., Stauffer, C., Romano, R. and Lee, L., "Using adaptive tracking to classify and monitor activities in a site," Computer Vision and Pattern Recognition, 1998, pp. 22-29.
  26. Yuan, F., GuangXuan, L., WeiCheng, F. and Heqin, Z., "Vision based fire detection using mixture Gaussian model," Proceedings of the 8th International Symposium on Fire Safety Science, 2005, pp. 1575-1583.
  27. Yu, C., Zhang, Y., Fang, J. and Wang, J., "Video smoke recognition based on optical flow," Proceedings of the 2nd IEEE International Conference on Advanced Computer Control (ICACC), 2010, pp. 16-21.
  28. Kolesov, I., Karasev, P., Tannenbaum, A. and Haber, H., "Fire and smoke detection in video with optimal mass transport based optical flow and neural networks," Proceedings of IEEE 17th International Conference on Image Processing, 2010, pp. 761-764.
  29. Chacon-Murguia, M. I. and Perez-Vargas, F. J., "Thermal Video Analysis for Fire Detection Using Shape Regularity and Intensity Saturation Features," Mexican Conference on Pattern Recognition (MCPR), 2011, pp. 118-126.
  30. McNeil, J. G., Lattimer, B. Y. and Hughes, J., "Robotic Fire Suppression Through Autonomous Feedback Control," Fire Technology, Vol. 53, 2017, pp. 1171-1199. https://doi.org/10.1007/s10694-016-0623-1
  31. Shin, M. C., Basic Statistics for Business and Economics, Changmin Publisher, 2010, pp. 174-176.
  32. Muhammad, K., Ahmad, J., Mehmood, I., Rho, S. and Baik, S. W., "Convolutional neural networks based fire detection in surveillance videos," IEEE Access, Vol. 6, 2018, pp. 18174-18183. https://doi.org/10.1109/access.2018.2812835
  33. Muhammad, K., Ahmad, J., Lv, Z., Bellavista, P., Yang, P. and Baik, S. W., "Efficient deep CNN-based fire detection and localization in video surveillance applications," IEEE Transactions on Systems, Man, and Cybernetics : Systems, Vol. 49, 2019, pp. 1419-1434. https://doi.org/10.1109/tsmc.2018.2830099
  34. Liu, D., Cao, L., Li, Z., Liu, T. and Che, P., "Infrared Small Target Detection Based on Flux Density and Direction Diversity in Gradient Vector Field," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Vol. 11, No. 7, 2018, pp. 2528-2554. https://doi.org/10.1109/jstars.2018.2828317
  35. Ostertagova, E., "Modelling using Polynomial Regreesion," Procedia Engineering, Vol. 48, 2012, pp. 500-506. https://doi.org/10.1016/j.proeng.2012.09.545