Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

3D Mesh Reconstruction Technique from Single Image using Deep Learning and Sphere Shape Transformation Method

딥러닝과 구체의 형태 변형 방법을 이용한 단일 이미지에서의 3D Mesh 재구축 기법

  • Kim, Jeong-Yoon (Dept. Electronic Engineering, Hanbat National University) ;
  • Lee, Seung-Ho (Dept. Electronic Engineering, Hanbat National University)
  • Received : 2022.03.30
  • Accepted : 2022.05.31
  • Published : 2022.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a 3D mesh reconstruction method from a single image using deep learning and a sphere shape transformation method. The proposed method has the following originality that is different from the existing method. First, the position of the vertex of the sphere is modified to be very similar to the 3D point cloud of an object through a deep learning network, unlike the existing method of building edges or faces by connecting nearby points. Because 3D point cloud is used, less memory is required and faster operation is possible because only addition operation is performed between offset value at the vertices of the sphere. Second, the 3D mesh is reconstructed by covering the surface information of the sphere on the modified vertices. Even when the distance between the points of the 3D point cloud created by correcting the position of the vertices of the sphere is not constant, it already has the face information of the sphere called face information of the sphere, which indicates whether the points are connected or not, thereby preventing simplification or loss of expression. can do. In order to evaluate the objective reliability of the proposed method, the experiment was conducted in the same way as in the comparative papers using the ShapeNet dataset, which is an open standard dataset. As a result, the IoU value of the method proposed in this paper was 0.581, and the chamfer distance value was It was calculated as 0.212. The higher the IoU value and the lower the chamfer distance value, the better the results. Therefore, the efficiency of the 3D mesh reconstruction was demonstrated compared to the methods published in other papers.

본 논문에서는 딥러닝과 구체의 형태 변형 방법을 이용한 단일 이미지에서의 3D mesh 재구축 기법을 제안한다. 제안한 기법은 기존의 방식과 다른 다음과 같은 독창성이 있다. 첫 번째, 기존의 근처의 가까운 점들을 연결하여 모서리 또는 면을 구축하는 방식과 다르게 딥러닝 네트워크을 통하여 구체의 꼭짓점의 위치를 사물의 3D 포인트 클라우드와 매우 유사하게 수정한다. 3D 포인트 클라우드를 이용하므로 메모리가 적게 필요하며 구체의 꼭짓점에 오프셋 값 사이에 덧셈 연산만을 수행하기 때문에 더 빠른 연산이 가능하다. 두 번째, 수정한 꼭짓점에 구체의 면 정보를 씌워 3D mesh를 재구축한다. 구체의 꼭짓점의 위치를 수정하여 생성한 3D 포인트 클라우드의 점들의 간격이 일정하지 않을 때에도 이미 점들 사이의 연결 여부를 나타내는 구체의 면 정보라는 3D mesh의 면 정보를 가지고 있어 표현의 단순화나 결손을 방지할 수 있다. 제안하는 기법의 객관적인 신뢰성을 평가하기 위해 공개된 표준 데이터셋인 ShapeNet 데이터셋을 이용하여 비교 논문들과 같은 방법으로 실험한 결과, 본 논문에서 제안하는 기법의 IoU 값이 0.581로, chamfer distance 값은 0.212로 산출되었다. IoU 값은 수치가 높을수록, chamfer distance 값은 수치가 낮을수록 우수한 결과를 나타내므로 다른 논문에서 발표한 기법들보다 3D mesh 재구축의 결과에서 성능의 효율성이 입증되었다.

Keywords

References

  1. CIPRESSO, Pietro, et al. "The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature," Frontiers in psychology 2086, 2018. DOI: 10.3389/fpsyg.2018.02086
  2. Gokturk, S. Burak, Hakan Yalcin, and Cyrus Bamji. "A time-of-flight depth sensor-system description, issues and solutions," 2004 Conference on Computer Vision and Pattern Recognition Workshop. IEEE, 2004. DOI: 10.1109/CVPR.2004.291
  3. KINGMA, Diederik P.; WELLING, Max. "Autoencoding variational bayes," International Conference on Learning Representations (ICLR), 2014.
  4. Choy, Christopher B., et al. "3d-r2n2: A unified approach for single and multi-view 3d object reconstruction," European conference on computer vision. Springer, Cham, 2016.
  5. Fan, Haoqiang, Hao Su, and Leonidas J. Guibas. "A point set generation network for 3d object reconstruction from a single image," Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. DOI: 10.48550/arXiv.1612.00603
  6. Wang, Nanyang, et al. "Pixel2mesh: Generating 3d mesh models from single rgb images." Proceedings of the European Conference on Computer Vision (ECCV). 2018.
  7. Groueix, Thibault, et al. "A papier-mache approach to learning 3d surface generation," Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
  8. Mescheder, Lars, et al. "Occupancy networks: Learning 3d reconstruction in function space." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.