Browse > Article
http://dx.doi.org/10.5909/JBE.2020.25.6.954

Depth Map Correction Algorithm based on Segmentation in Multi-view Systems  

Jung, Woo-Kyung (Sejong University, Dept. of Electrical Engineering)
Han, Jong-Ki (Sejong University, Dept. of Electrical Engineering)
Publication Information
Journal of Broadcast Engineering / v.25, no.6, 2020 , pp. 954-964 More about this Journal
Abstract
In immersive media, the most important factor that provides immersion is depth information. Therefore, it is essential to obtain high quality depth information in order to produce high quality immersive media. In this paper we propose an algorithm to improve depth map, considering the segmentation of images and the relationship between multiple views in multi-view systems. The proposed algorithm uses a super-pixel segmentation technique to divide the depth map of the reference view into several segments, and project each segment into adjacent view. Subsequently, the depth map of the adjacent view is improved using plane estimation using the information of the projected segment, and then reversed to the reference view. This process is repeated for several adjacent views to improve the reference depth map by updating the values of the improved adjacent views and the initial depth map of the reference view. Through simulation, the proposed algorithm is shown to surpass the conventional algorithm subjectively and objectively.
Keywords
depth map; multi-view system; super-pixel based segmentation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 H. Hirschmuller, "Stereo Processing by Semiglobal Matching and Mutual Information," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008.   DOI
2 A. Dziembowski, A. Grzelka, D. Mieloch, O. Stankiewicz and M. Domanski, "Depth map upsampling and refinement for FTV systems", 2016 International Conference on Signals and Electronic Systems (ICSES), Krakow, pp. 89-92, 2016.
3 T. Chang, J. Kuo and J. Yang, "Efficient hole filling and depth enhancement based on texture image and depth map consistency," 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), Jeju, pp. 192-195, 2016.
4 J. Lu, D. Min, R. S. Pahwa and M. N. Do, "A revisit to MRF-based depth map super-resolution and enhancement," 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, pp. 985-988, 2011.
5 D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms" Int. Jour. Computer Vision, 47(1/2/3):7-42, 2002   DOI
6 T. Schops, J. L. Schonberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, A. Geiger, "A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos", Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
7 H. Shih and H. Hsiao, "A depth refinement algorithm for multi-view video synthesis," 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, pp. 742-745, 2010
8 M. Kurc, O. Stankiewicz and M. Domanski, "Depth map inter-view consistency refinement for multiview video," 2012 Picture Coding Symposium, Krakow, pp. 137-140, 2012.
9 R. Xiaofeng and J. Malik. "Learning a classification model for segmentation." Proceedings Ninth IEEE International Conference on Computer Vision, pp. 10-17 vol.1, 2003.
10 S. M. Seitz, B. Curless, J. Diebel, D. Scharstein and R. Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms," 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), New York, NY, USA, 2006, pp. 519-528, doi: 10.1109/CVPR.2006.19.   DOI
11 Arno Knapitsch, , Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. "Tanks and Temples: Benchmarking Large-Scale Scene Reconstruction".ACM Transactions on Graphics 36, no.4 (2017).
12 M. A. Fischler and R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography'. Commun. ACM 24, pp. 381-395, 1981.   DOI
13 T. Schops, J. L. Schonberger, S. Galliani, T. Sattler, K. Schindler, M. Pollefeys, A. Geiger, "A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos", Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
14 https://github.com/cdcseacave/openMVS