대한임베디드공학회논문지 (IEMEK Journal of Embedded Systems and Applications)

대한임베디드공학회 (Institute of Embedded Engineering of Korea)
권호 표지
DOI QR코드

DOI QR Code

3차원 합성곱 신경망 기반 향상된 스테레오 매칭 알고리즘

Enhanced Stereo Matching Algorithm based on 3-Dimensional Convolutional Neural Network

  • 투고 : 2021.08.20
  • 심사 : 2021.09.23
  • 발행 : 2021.10.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

For stereo matching based on deep learning, the design of network structure is crucial to the calculation of matching cost, and the time-consuming problem of convolutional neural network in image processing also needs to be solved urgently. In this paper, a method of stereo matching using sparse loss volume in parallax dimension is proposed. A sparse 3D loss volume is constructed by using a wide step length translation of the right view feature map, which reduces the video memory and computing resources required by the 3D convolution module by several times. In order to improve the accuracy of the algorithm, the nonlinear up-sampling of the matching loss in the parallax dimension is carried out by using the method of multi-category output, and the training model is combined with two kinds of loss functions. Compared with the benchmark algorithm, the proposed algorithm not only improves the accuracy but also shortens the running time by about 30%.

키워드

과제정보

본 논문은 2021년도 정부 (산업통상자원부)의 재원으로 한국에너지기술평가원의 지원을 받아 수행된 연구임 (20213030020120, 해상풍력발전 블레이드의 전주기 신뢰성 향상을 위한 생산품질 및 유지관리 기술 개발).

참고문헌

  1. V.D. Nguyen, D.D. Nguyen, S.J. Lee, J,W. Jeon,, "Local Density Encoding for Robust Stereo Matching," Transactions on Circuits and Systems for Video Technology, Vol. 24, No. 12, pp. 2049-2062, 2014. https://doi.org/10.1109/TCSVT.2014.2334053
  2. R.A. Hamzah, H. Ibrahim, A.H.A. Hassan, "Stereo Matching Algorithm Based on per Pixel Difference Adjustment, Iterative Guided Filter and Graph Segmentation," Journal of Visual Communication and Image Representation, pp. 145-160, 2017.
  3. S.Q. Ren, K.M. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks," Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 6, pp. 1137-1149, 2015.
  4. E. Shelhamer, J. Long, T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 4, pp. 640-651, 2017. https://doi.org/10.1109/TPAMI.2016.2572683
  5. J.S. Xiao, H. Tian, W.T. Zou, "Stereo Matching Based on Convolutional Neural Network," Acta Optica Sinica, 2018.
  6. J. Zbontar, Y. LeCun, "Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches," Journal of Machine Learning Research, Vol. 17, No. 1, pp. 2287-2318, 2016.
  7. N. Mayer, E. Ilg, P. Hausser, "A Large Dataset to Train Convolutional Network for Disparity, Optical Flow, and Scene Flow Estimation," Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, New York, pp. 4040-4048, 2016.
  8. J.H. Pang, W.X. Sun, J.S. Ren, "Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching," International Conference on Computer Vision Workshops, Venice, Italy. New York, pp. 878-886, 2017.
  9. A. Kendall, H. Martirosyan, S. Dasgupta, P. Henry, R. Kennedy, A. Bachrach, A. Bry, "End-to-end Learning of Geometry and Context for Deep Stereo Regression", International Conference on Computer Vision, Venice, Italy. New York, pp. 66-75, 2017.
  10. J.R. Chang, Y.S. Chen, "Pyramid Stereo Matching Network," Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. New York, pp. 5410-5418, 2018.
  11. W.J. Luo, A.G. Schwing, R. Urtasun, "Efficient Deep Learning for Stereo Matching," Conference on Computer Vision and Pattern Recognition, Las Vegas, UT, USA. New York, pp. 5695-5703, 2016.
  12. Z.F. Liang, Y.L. Feng, Y.L. Guo, H. Liu, W. Chen, L. Qiao, L. Zhou, J. Zhang, "Learning for Disparity Estimation Through Feature Constancy," Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. New York, pp. 2811-2820, 2018.
  13. Z.Q. Jie, P.F. Wang, Y.G Ling, B. Zhao, Y. Wei, J. Feng, W. Liu, "Left-right Comparative Recurrent Model for Stereo Matching," Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. New York, pp. 3838-3846, 2018.
  14. L.D. Yu, Y.C. Wang, Y.W. Wu, Y.D. Jia, "Deep Stereo Matching with Explicit Cost Aggregation Sub-architecture," http://cn.arxiv.org/abs/1801.04065, 2018.
  15. N. Smolyanskiy, A. Kamenev, S. Birchfield, "On the Importance of Stereo for Accurate Depth Estimation: an Efficient Semi-supervised Deep Neural Network approach," Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA. New York, pp. 1120-1128, 2018.
  16. F.H. Zhang, V Prisacariu, R. Yang, P.H.S. Torr, "GA-Net: Guided Aggregation Net for End-To-End Stereo Matching," IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, CA, USA, pp. 185-194, 2019.
  17. J.H. Huang, R.F. Zhang, Y.H. Liu, "Research on An Optimized Deep Learning Stereo Matching Algorithm," Laser & Optoelectronics Progress, Vol. 58, No. 24, pp. 37-55, 2021.
  18. R. Girshick, "Fast R-CNN," International Conference on Computer Vision, Santiago, Chile. New York, pp. 1440-1448, 2015.
  19. D.P. Kingma, J. Ba, "Adam: A Method for Stochastic Optimization," http://cn.arxiv.org/abs/1412.6980, 2017.