Browse > Article
http://dx.doi.org/10.14372/IEMEK.2021.16.5.179

Enhanced Stereo Matching Algorithm based on 3-Dimensional Convolutional Neural Network  

Wang, Jian (Kunsan National University)
Noh, Jackyou (Kunsan National University)
Publication Information
IEMEK Journal of Embedded Systems and Applications / v.16, no.5, 2021 , pp. 179-186 More about this Journal
Abstract
For stereo matching based on deep learning, the design of network structure is crucial to the calculation of matching cost, and the time-consuming problem of convolutional neural network in image processing also needs to be solved urgently. In this paper, a method of stereo matching using sparse loss volume in parallax dimension is proposed. A sparse 3D loss volume is constructed by using a wide step length translation of the right view feature map, which reduces the video memory and computing resources required by the 3D convolution module by several times. In order to improve the accuracy of the algorithm, the nonlinear up-sampling of the matching loss in the parallax dimension is carried out by using the method of multi-category output, and the training model is combined with two kinds of loss functions. Compared with the benchmark algorithm, the proposed algorithm not only improves the accuracy but also shortens the running time by about 30%.
Keywords
Stereo matching; 3D Convolutional Neural Network; Parallax dimension; Computation cost; Network structure;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J.R. Chang, Y.S. Chen, "Pyramid Stereo Matching Network," Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. New York, pp. 5410-5418, 2018.
2 Z.Q. Jie, P.F. Wang, Y.G Ling, B. Zhao, Y. Wei, J. Feng, W. Liu, "Left-right Comparative Recurrent Model for Stereo Matching," Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. New York, pp. 3838-3846, 2018.
3 F.H. Zhang, V Prisacariu, R. Yang, P.H.S. Torr, "GA-Net: Guided Aggregation Net for End-To-End Stereo Matching," IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Long Beach, CA, USA, pp. 185-194, 2019.
4 D.P. Kingma, J. Ba, "Adam: A Method for Stochastic Optimization," http://cn.arxiv.org/abs/1412.6980, 2017.
5 N. Smolyanskiy, A. Kamenev, S. Birchfield, "On the Importance of Stereo for Accurate Depth Estimation: an Efficient Semi-supervised Deep Neural Network approach," Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA. New York, pp. 1120-1128, 2018.
6 E. Shelhamer, J. Long, T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 4, pp. 640-651, 2017.   DOI
7 L.D. Yu, Y.C. Wang, Y.W. Wu, Y.D. Jia, "Deep Stereo Matching with Explicit Cost Aggregation Sub-architecture," http://cn.arxiv.org/abs/1801.04065, 2018.
8 J.H. Huang, R.F. Zhang, Y.H. Liu, "Research on An Optimized Deep Learning Stereo Matching Algorithm," Laser & Optoelectronics Progress, Vol. 58, No. 24, pp. 37-55, 2021.
9 R. Girshick, "Fast R-CNN," International Conference on Computer Vision, Santiago, Chile. New York, pp. 1440-1448, 2015.
10 Z.F. Liang, Y.L. Feng, Y.L. Guo, H. Liu, W. Chen, L. Qiao, L. Zhou, J. Zhang, "Learning for Disparity Estimation Through Feature Constancy," Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. New York, pp. 2811-2820, 2018.
11 R.A. Hamzah, H. Ibrahim, A.H.A. Hassan, "Stereo Matching Algorithm Based on per Pixel Difference Adjustment, Iterative Guided Filter and Graph Segmentation," Journal of Visual Communication and Image Representation, pp. 145-160, 2017.
12 S.Q. Ren, K.M. He, R. Girshick, J. Sun, "Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks," Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, No. 6, pp. 1137-1149, 2015.
13 J.S. Xiao, H. Tian, W.T. Zou, "Stereo Matching Based on Convolutional Neural Network," Acta Optica Sinica, 2018.
14 J. Zbontar, Y. LeCun, "Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches," Journal of Machine Learning Research, Vol. 17, No. 1, pp. 2287-2318, 2016.
15 J.H. Pang, W.X. Sun, J.S. Ren, "Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching," International Conference on Computer Vision Workshops, Venice, Italy. New York, pp. 878-886, 2017.
16 A. Kendall, H. Martirosyan, S. Dasgupta, P. Henry, R. Kennedy, A. Bachrach, A. Bry, "End-to-end Learning of Geometry and Context for Deep Stereo Regression", International Conference on Computer Vision, Venice, Italy. New York, pp. 66-75, 2017.
17 W.J. Luo, A.G. Schwing, R. Urtasun, "Efficient Deep Learning for Stereo Matching," Conference on Computer Vision and Pattern Recognition, Las Vegas, UT, USA. New York, pp. 5695-5703, 2016.
18 V.D. Nguyen, D.D. Nguyen, S.J. Lee, J,W. Jeon,, "Local Density Encoding for Robust Stereo Matching," Transactions on Circuits and Systems for Video Technology, Vol. 24, No. 12, pp. 2049-2062, 2014.   DOI
19 N. Mayer, E. Ilg, P. Hausser, "A Large Dataset to Train Convolutional Network for Disparity, Optical Flow, and Scene Flow Estimation," Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, New York, pp. 4040-4048, 2016.