Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

A method of improving the quality of 3D images acquired from RGB-depth camera

깊이 영상 카메라로부터 획득된 3D 영상의 품질 향상 방법

  • Park, Byung-Seo (Department of Electronic Materials Engineering, Kwangwoon University) ;
  • Kim, Dong-Wook (Department of Electronic Materials Engineering, Kwangwoon University) ;
  • Seo, Young-Ho (Department of Electronic Materials Engineering, Kwangwoon University)
  • Received : 2020.10.07
  • Accepted : 2021.04.16
  • Published : 2021.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

In general, in the fields of computer vision, robotics, and augmented reality, the importance of 3D space and 3D object detection and recognition technology has emerged. In particular, since it is possible to acquire RGB images and depth images in real time through an image sensor using Microsoft Kinect method, many changes have been made to object detection, tracking and recognition studies. In this paper, we propose a method to improve the quality of 3D reconstructed images by processing images acquired through a depth-based (RGB-Depth) camera on a multi-view camera system. In this paper, a method of removing noise outside an object by applying a mask acquired from a color image and a method of applying a combined filtering operation to obtain the difference in depth information between pixels inside the object is proposed. Through each experiment result, it was confirmed that the proposed method can effectively remove noise and improve the quality of 3D reconstructed image.

일반적으로, 컴퓨터 비전, 로보틱스, 증강현실 분야에서 3차원 공간 및 3차원 객체 검출 및 인식기술의 중요성이 대두되고 있다. 특히, 마이크로소프트사의 키넥트(Microsoft Kinect) 방식을 사용하는 영상 센서를 통하여 RGB 영상과 깊이 영상을 실시간 획득하는 것이 가능해짐으로 인하여 객체 검출, 추적 및 인식 연구에 많은 변화를 가져오고 있다. 본 논문에서는 다시점 카메라 시스템 상에서의 깊이 기반(RGB-Depth) 카메라를 통해 획득된 영상을 처리하여 3D 복원 영상의 품질을 향상하는 방법을 제안한다. 본 논문에서는 컬러 영상으로부터 획득한 마스크 적용을 통해 객체 바깥쪽 잡음을 제거하는 방법과 객체 안쪽의 픽셀 간 깊이 정보 차이를 구하는 필터링 연산을 결합하여 적용하는 방법을 제시하였다. 각 실험 결과를 통해 제시한 방법이 효과적으로 잡음을 제거하여 3D 복원 영상의 품질을 향상할 수 있음을 확인하였다.

Keywords

Acknowledgement

This work was supported by the Technology development Program (S2949268) funded by the Ministry of SMEs and Startups (MSS, Korea)

References

  1. R. Schafer, P. Kauff, R. Skupin, Y. Sanchez, and C. Weissig, "Interactive Steaming of Panoramas and VR Worlds," in SMPTE Motion Imaging Journal, vol. 126, no. 1, pp. 35-42, Jan. 2017. https://doi.org/10.5594/JMI.2016.2640058
  2. Z. Zhan, G. Zhou, and X. Yang, "A Method of Hierarchical Image Retrieval for Real-Time Photogrammetry Based on Multiple Features," in IEEE Access, vol. 8, pp. 21524-21533, 2020. https://doi.org/10.1109/ACCESS.2020.2969287
  3. Photogrammetry [Internet]. Available: https://en.wikipedia.org/wiki/Photogrammetry.
  4. K. Guo, P. Lincoln, P. Davidson, J. Busch, and X. Yu, "The Relightables: Volumetric Performance Capture of Humans with Realistic Relighting," ACM Transactions on Graphics, vol. 38, no. 6, pp. 1-19, Nov. 2019.
  5. M. Dou, P. Davidson, S. R. Fanello, S. Khamis, A. Kowdle, C. Rhemann, V. Tankovich, and S. Izadi, "Motion2fusion: real-time volumetric performance capture," ACM Transactions on Graphics, vol. 35, no. 4, pp. 1-16, Jul. 2016.
  6. M. Dou, S. Khamis, Y. Degtyarev, P. Davidson, S. R. Fanello, A. Kowdle, S. O. Escolano, C. Rhemann, D. Kim, J. Taylor, P. Kohli, V. Tankovich, and S. Izadi, "Fusion4D: real-time performance capture of challenging scenes," ACM Transactions on Graphics, vol. 36, no. 6, pp. 1-13, Nov. 2017.
  7. Open Source Computer Vision. Detection of ChArUco Corners [Internet]. Available: https://docs.opencv.org/3.4/df/d4a/tutorial_charuco_detection.html.
  8. G. An, S. Lee, M. Seo, K. Yun, W. Cheong, and S. Kang, "Charuco Board-Based Omnidirectional Camera Calibration Method," in Electronics, vol. 7, no. 12, pp. 421-436, 2018. https://doi.org/10.3390/electronics7120421
  9. Open Source Computer Vision, Calibration with ArUco and ChArUco [Internet]. Available: https://docs.opencv.org/master/da/d13/tutorial_aruco_calibration.html.
  10. P. Rathnayaka, S. Baek, and S. Park, "Calibration of a Different Field-of-view Stereo Camera System using an Embedded Checker- board Pattern," International Conference on Computer Vision Theory and Applications, pp. 294-300, 2017.
  11. K. Kim, B. Park, J. Kim, D. Kim, and Y. Seo, "Holographic augmented reality based on three-dimensional volumetric imaging for a photorealistic scene," Optics Express, vol. 28, pp. 35972-35985, 2020. https://doi.org/10.1364/OE.411141
  12. G. H. An, S. Lee, M. Seo, K. Yun, W. Cheong, and S. Kang, "Charuco board-based omnidirectional camera calibration method," Electronics, vol. 7, no. 12, pp. 421, 2018. https://doi.org/10.3390/electronics7120421
  13. S. Ruder, "An overview of gradient descent optimization algorithms," arXiv preprint, 2016.