Journal of Korea Multimedia Society (한국멀티미디어학회논문지)

Korea Multimedia Society (한국멀티미디어학회)
권호 표지
DOI QR코드

DOI QR Code

Multi-spectral Vehicle Detection based on Convolutional Neural Network

  • Choi, Sungil (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Kim, Seungryong (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Park, Kihong (Dept. of Electrical and Electronic Engineering, Yonsei University) ;
  • Sohn, Kwanghoon (Dept. of Electrical and Electronic Engineering, Yonsei University)
  • Received : 2016.05.26
  • Accepted : 2016.11.21
  • Published : 2016.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a unified framework for joint Convolutional Neural Network (CNN) based vehicle detection by leveraging multi-spectral image pairs. With the observation that under challenging environments such as night vision and limited light source, vehicle detection in a single color image can be more tractable by using additional far-infrared (FIR) image, we design joint CNN architecture for both RGB and FIR image pairs. We assume that a score map from joint CNN applied to overall image can be considered as confidence of vehicle existence. To deal with various scale ratios of vehicle candidates, multi-scale images are first generated scaling an image according to possible scale ratio of vehicles. The vehicle candidates are then detected on local maximal on each score maps. The generation of overlapped candidates is prevented with non-maximal suppression on multi-scale score maps. The experimental results show that our framework have superior performance than conventional methods with a joint framework of multi-spectral image pairs reducing false positive generated by conventional vehicle detection framework using only single color image.

Keywords

References

  1. National Highway Traffic Safety Administration, Traffic Safety Facts, Annals of Emergency Medicine, 2013.
  2. D.W. James and V. Sharma, "Background-Subtraction in Thermal Imagery Using Contour Saliency," International Journal of Computer Vision, Vol. 71, No. 2, pp. 161-181, 2007. https://doi.org/10.1007/s11263-006-4121-7
  3. L. Walchshäusl, R. Lindl, K. Vogel, and T. Tatschke, Advanced Microsystems for Automotive Applications, Springer Publishers, Berlin Heidelberg, 2006.
  4. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, Overfeat: Integrated Recognition, Localization and Detection Using Convolutional Networks, arXiv preprint arXiv:1312.6229, 2013.
  5. S. Hwang, J. Park, N. Kim, Y. Choi, and S. Kweon, "Multispectral Pedestrian Detection: Benchmark Dataset and Baseline," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1037-1045, 2015.
  6. LSI Far Infrared Pedestrian Dataset, http://www.uc3m.es/islab/repository (accessed July, 2013).
  7. C.J. Burges, "A Tutorial on Support Vector Machines for Pattern Recognition," Data Mining and Knowledge Discovery, Vol. 2, No. 2, pp. 121-167, 1988. https://doi.org/10.1023/A:1009715923555
  8. F. Yoav and S.E. Robert, "A Decision-theoretic Generalization of On-line Learning and an Application to Boosting," Journal of Computer and System Sciences, Vol. 55, No. 1, pp. 119-139, 1997. https://doi.org/10.1006/jcss.1997.1504
  9. D. Navneet and T. Bill, "Histograms of Oriented Gradients for Human Detection," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 886-893, 2005.
  10. L. Rainer and M. Jochen, "An Extended Set of Haar-like Features for Rapid Object Detection," Proceeding of International Conference on Image Processing, pp. 900-903, 2002.
  11. S. Zehang, B. George, and M. Ronald, "Onroad Vehicle Detection Using Gabor Filters and Support Vector Machines," Proceeding of International Conference on Digital Signal Processing, pp. 1019-1022, 2002.
  12. J. Seo and K. Sohn, "Superpixel-based Vehicle Detection Using Plane Normal Vector in Disparity Space," Journal of Korea Multimedia Society, Vol. 19, No. 6, pp. 1003-1013, 2016. https://doi.org/10.9717/kmms.2016.19.6.1003
  13. Y. Lee, T. Kim, and J. Shim, "Two-wheeler Detection System Using Histogram of Oriented Gradients Based on Local Correlation Coefficients and Curvature," Journal of Korea Multimedia Society, Vol. 2, No. 4, pp. 303-310, 2015.
  14. F. Pedro, M. David, and R. Deva, "A Discriminatively Trained, Multiscale, Deformable Part Model," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
  15. P. Dollar, R. Appel, and S. Belongie, "Fast Feature Pyramids for Object Detection," Journal of Pattern Analysis and Machine Intelligence, Vol. 36, No. 8, pp. 1532-1545, 2014. https://doi.org/10.1109/TPAMI.2014.2300479
  16. J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. Fei-Fei, "ImageNet: Large-scale Hierarchical Image Database," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009.
  17. G. Ross, D. Jeff, D. Trevor, and M. Jitendra, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580-587, 2014.
  18. X. Glorot and Y. Bengio, "Understanding the Difficulty of Training Deep Feedforward Neural Networks," Proceeding of International Conference on Artificial Intelligence and Statistics, pp. 249-256, 2010.
  19. M. Tarek, A. Nabil, S.A. Domingo, A. Cristhian, and T. Ricardo, "Multispectral Stereo Odometry," IEEE Transactions on Intelligent Transportation Systems, Vol. 16, No. 3, pp. 1210-1224, 2015. https://doi.org/10.1109/TITS.2014.2354731
  20. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, T. Darrell, et al., "DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition," Proceeding of the International Confidence on Machine Learning, pp. 647-655, 2014.
  21. J. Long, E. Shelhamer, and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431-3440, 2015.
  22. C. Papageorgiou and T. Poggio, "A Trainable System for Object Detection," International Journal of Computer Vision, Vol. 38, No. 1, pp. 15-33, 2000. https://doi.org/10.1023/A:1008162616689
  23. N. Srivastava, G. Hinton, A. Krizhevsky, Alex, I. Sutskever, and R. Salakhutdinov, "Dropout: A Simple Way to Prevent Neural Networks from Overfitting," The Journal of Machine Learning Research, Vol. 15, No. 1, pp. 1929-1958, 2014.
  24. A. Vedaldi and K. Lenc, "MatConvNet-Convolutional Neural Networks for MATLAB," Proceedings of the 23rd ACM International Confidence on Multimedia, pp. 689-692, 2015.
  25. [Available] P. Dollar, Piotr's Computer Vision Matlab Toolbox, http://vision.ucsd.edu/˜pdollar/toolbox/doc/index.html.

Cited by

  1. 시간에 따라 변화하는 빗줄기 장면을 이용한 딥러닝 기반 비지도 학습 빗줄기 제거 기법 vol.22, pp.1, 2019, https://doi.org/10.9717/kmms.2019.22.1.001