The Journal of The Korea Institute of Intelligent Transport Systems (한국ITS학회 논문지)

The Korea Institute of Intelligent Transport Systems (한국ITS학회)
권호 표지
DOI QR코드

DOI QR Code

Class 1·3 Vehicle Classification Using Deep Learning and Thermal Image

열화상 카메라를 활용한 딥러닝 기반의 1·3종 차량 분류

  • Jung, Yoo Seok (Korea Institute of Civil Engineering and Building Technology) ;
  • Jung, Do Young (Korea Institute of Civil Engineering and Building Technology)
  • 정유석 (한국건설기술연구원 미래융합연구본부) ;
  • 정도영 (한국건설기술연구원 미래융합연구본부)
  • Received : 2020.11.09
  • Accepted : 2020.12.14
  • Published : 2020.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

To solve the limitation of traffic monitoring that occur from embedded sensor such as loop and piezo sensors, the thermal imaging camera was installed on the roadside. As the length of Class 1(passenger car) is getting longer, it is becoming difficult to classify from Class 3(2-axle truck) by using an embedded sensor. The collected images were labeled to generate training data. A total of 17,536 vehicle images (640x480 pixels) training data were produced. CNN (Convolutional Neural Network) was used to achieve vehicle classification based on thermal image. Based on the limited data volume and quality, a classification accuracy of 97.7% was achieved. It shows the possibility of traffic monitoring system based on AI. If more learning data is collected in the future, 12-class classification will be possible. Also, AI-based traffic monitoring will be able to classify not only 12-class, but also new various class such as eco-friendly vehicles, vehicle in violation, motorcycles, etc. Which can be used as statistical data for national policy, research, and industry.

본 연구에서는 루프 센서를 통한 교통량 수집방식의 오류를 해결하기 위해 1종(승용차)과 3종(일반 트럭)의 구분이 어려운 부분 및 영상 이미지의 단점을 보완하기 위해 도로변에 열화상 카메라를 설치하여 영상 이미지를 수집하였다. 수집된 영상 이미지를 레이블링 단계를 거쳐 1종(승용차)과 3종(일반 트럭)의 학습데이터를 구성하였다. 정지영상을 대상으로 labeling을 진행하였으며, 총 17,536대의 차량 이미지(640x480 pixel)에 대해 시행하였다. 열화상 영상 기반의 차종 분류를 달성하기 위해 CNN(Convolutional Neural Network)을 이용하였으며, 제한적인 데이터량과 품질에도 불구하고 97.7%의 분류정확도를 나타내었다. 이는 AI 영상인식 기반의 도로 교통량 데이터 수집 가능성을 보여주는 것이라 판단되며, 향후 더욱더 많은 학습데이터를 축적한다면 12종 차종 분류가 가능할 것이다. 또한, AI 기반 영상인식으로 도로 교통량의 12종 차종뿐만 아니라 다양한(친환경 차량, 도로 법규 위반차량, 이륜자동차 등) 차종 분류를 할 수 있을 것이며, 이는 국가정책, 연구, 산업 등의 통계 데이터로 활용도가 높을 것으로 판단된다.

Keywords

References

  1. Ali R.(2019), Deep learning-and infrared thermography-based subsurface damage detection in a steel bridge, University of Manitoba.
  2. Audebert N., Le Saux B. and Lefèvre S.(2017), "Segment-before-detect: Vehicle detection and classification through semantic segmentation of aerial images," Remote Sensing, vol. 9, no. 4, 368. https://doi.org/10.3390/rs9040368
  3. Bajwa R. et al.(2011), "In-pavement wireless sensor network for vehicle classification," In Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks, IEEE, pp.85-96.
  4. Bang H. T., Park S. and Jeon H.(2020), "Defect identification of composites via thermography and deep learning techniques," Composite Structures, vol. 246, 112405. https://doi.org/10.1016/j.compstruct.2020.112405
  5. Burnos P. et al.(2011), "Road traffic parameters measuring system with variable structure," Metrology and Measurement Systems, vol. 18, no. 4, pp.659-666. https://doi.org/10.2478/v10178-011-0062-8
  6. Cao Y. et al.(2020), "Two-stream convolutional neural network for non-destructive subsurface defect detection via similarity comparison of lock-in thermography signals," NDT & E International, vol. 112, 102246. https://doi.org/10.1016/j.ndteint.2020.102246
  7. Chen Z., Ellis T. and Velastin S. A.(2012), "Vehicle detection, tracking and classification in urban traffic," In 2012 15th International IEEE Conference on Intelligent Transportation Systems, IEEE, pp.951-956.
  8. Cheng C., Shang Z. and Shen Z.(2019), "CNN-based deep architecture for reinforced concrete delamination segmentation through thermography'," In Computing in Civil Engineering 2019: Smart Cities, Sustainability, and Resilience, American Society of Civil Engineers Reston, VA, pp.50-57.
  9. Cheung S. Y. et al.(2005), "Traffic measurement and vehicle classification with single magnetic sensor'," Transportation Research Record, vol. 1917, no. 1, pp.173-181. https://doi.org/10.1177/0361198105191700119
  10. Choi Y. H. and Cho W. S.(2018), "Research of Automatic Water Leak Detection Technology Used on Thermography and Deep Learning," The Journal of Korean Institute of Information Technology, vol. 16, no. 8, pp.1-9, doi:10.14801/jkiit.2018.16.8.1.
  11. Chun C. and Ryu S. K.(2019), "Road Surface Damage Detection Using Fully Convolutional Neural Networks and Semi-Supervised Learning," Sensors, vol. 19, no. 24, 5501, doi:10.3390/s19245501.
  12. Duarte M. F. and Hu Y. H.(2004), "Vehicle classification in distributed sensor networks," Journal of Parallel and Distributed Computing, vol. 64, no. 7, pp.826-838. https://doi.org/10.1016/j.jpdc.2004.03.020
  13. Fahmy M. and Moselhi O.(2009), "Detecting and locating leaks in underground water mains using thermography," In Proceedings of the 26th International Symposium on Automation and Robotics in Construction(ISARC 2009), Austin, TX, USA, pp.24-27.
  14. Kaewkamnerd S. et al.(2009), "Automatic vehicle classification using wireless magnetic sensor'," In 2009 IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, IEEE, pp.420-424.
  15. Lee J. T. and Chung Y.(2017), "Deep learning-based vehicle classification using an ensemble of local expert and global networks," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.47-52.
  16. Liu W. et al.(2017), "An ensemble deep learning method for vehicle type classification on visual traffic surveillance sensors," IEEE Access, vol. 5, pp.24417-24425. https://doi.org/10.1109/ACCESS.2017.2766203
  17. Ma W. et al.(2013), "A wireless accelerometer-based automatic vehicle classification prototype system," IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 1, pp.104-111. https://doi.org/10.1109/TITS.2013.2273488
  18. MacCarley A. and Slonaker J.(2008), Video Vehicle Detector Verification System(V2DVS).
  19. Mita Y. and Imazu K.(1995), "Range-measurement-type optical vehicle detector," Intelligent Transport Systems World Congress.
  20. Peiyu L., Dapeng T. and Boyu L.(2006), "Embedded flexible assembly system for car latch based on laser identification," International Technology and Innovation Conference 2006(ITIC 2006).
  21. Ramesh V.(2017), "A Review on Application of Deep Learning in Thermography," International Journal of Engineering and Management Research(IJEMR), vol. 7, no. 3, pp.489-493.
  22. Sentas A. et al.(2020), "Performance evaluation of support vector machine and convolutional neural network algorithms in real-time vehicle type and color classification," Evolutionary Intelligence, vol. 13, no. 1, pp.83-91. https://doi.org/10.1007/s12065-018-0167-z
  23. Taghvaeeyan S. and Rajamani R.(2013), "Portable roadside sensors for vehicle counting, classification, and speed measurement," IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 1, pp.73-83. https://doi.org/10.1109/TITS.2013.2273876
  24. Tang Y. et al.(2017), "Vehicle detection and recognition for intelligent traffic surveillance system," Multimedia tools and applications, vol. 76, no. 4, pp.5817-5832. https://doi.org/10.1007/s11042-015-2520-x
  25. Valev K. et al.(2018), "A systematic evaluation of recent deep learning architectures for fine-grained vehicle classification," In Pattern Recognition and Tracking XXIX, International Society for Optics and Photonics, 1064902.
  26. Wen X. et al.(2015), "A rapid learning algorithm for vehicle classification," Information sciences, vol. 295, pp.395-406. https://doi.org/10.1016/j.ins.2014.10.040
  27. Wu J. et al.(2019), "Automatic vehicle classification using roadside LiDAR data," Transportation Research Record, vol. 2673, no. 6, pp.153-164.
  28. Xie R. et al.(2016), "Resource-constrained implementation and optimization of a deep neural network for vehicle classification," In 2016 24th European Signal Processing Conference(EUSIPCO), IEEE, pp.1862-1866.
  29. Yousaf K., Iftikhar A. and Javed A.(2012), "Comparative analysis of automatic vehicle classification techniques: A survey," International Journal of Image, Graphics and Signal Processing, vol. 4, no. 9, pp.52-59. https://doi.org/10.5815/ijigsp.2012.09.08
  30. Zhang G., Avery R. P. and Wang Y.(2007), "Video-based vehicle detection and classification system for real-time traffic data collection using uncalibrated video cameras," Transportation Research Record, vol. 1993, no. 1, pp.138-147. https://doi.org/10.3141/1993-19
  31. Zhao D., Chen Y. and Lv L.(2016), "Deep reinforcement learning with visual attention for vehicle classification," IEEE Transactions on Cognitive and Developmental Systems, vol. 9, no. 4, pp.356-367. https://doi.org/10.1109/TCDS.2016.2614675
  32. Zhou Y. and Cheung N. M.(2016), Vehicle classification using transferable deep neural network features, arXiv preprint arXiv:1601.01145.
  33. Zhuo L. et al.(2017), "Vehicle classification for large-scale traffic surveillance videos using convolutional neural networks," Machine Vision and Applications, vol. 28, no. 7, pp.793-802. https://doi.org/10.1007/s00138-017-0846-2