Broadcasting and Media Magazine (방송과미디어)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지

2차원 영상으로부터 3차원 영상을 모델링하는 기술 동향

  • Published : 2021.10.30
Download PDF
⟨ Previous Next ⟩

Abstract

2차원 영상을 3차원 모델 영상으로 변환하는 방식이 다양하게 발전해오고 있다. 딥러닝의 발전 중 특히 GAN의 다양한 연구는 2차원 영상의 생성뿐만 아니라 다양한 3차원 영상의 생성에도 진전을 보였다. 본 고에서는 2차원 영상을 3차원 영상으로 변환하는 연구의 필요성을 바탕으로 관련 연구의 내용과 동향을 분석하였다. 주요 내용으로는 딥러닝 기반의 3차원 객체인식, 2D로부터 3D 변환을 위한 신경망에 대한 연구, 생성적 기법을 적용한 연구, 3D 모델링 도구 등이 포함된다. 관련 연구의 전반적인 흐름을 고려했을 때 향후 3D 모델링의 정교한 표현력 향상, 고속의 고해상도 렌더링, 편리한 온라인 접근성 등을 예상하게 된다. 관련 산업 종사자들에게는 생성시간의 단축을 가져올 수 있고 일반인은 전문적인 3D 기술이 없어도 우수한 3D 모델을 생성하고 활용할 수 있을 것으로 기대한다.

Keywords

Acknowledgement

이 글은 2021년도 과학기술정보통신부의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임(No.2021-0-00751, 0.5mm급 이하 초정밀 가시·비가시 정보 표출을 위한 다차원 시각화 디지털 트윈 프레임워크 기술개발)

References

  1. 인공지능과 최적설계, https://www.koreascience.or.kr/article/JAKO202017054987861.pdf(Accessed September 26, 2021)
  2. 딥 러닝 생성 설계의 외계인 스타일, https://medium.com/intuitionmachine/the-alien-look-of-deep-learning-generative-design5c5f871f7d10(Accessed September 26, 2021)
  3. Shaoshuai Shi Xiaogang Wang Hongsheng Li, PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, https://arxiv.org/pdf/1812.04244.pdf, arXiv:1812.04244v2[cs.CV] 16, May, 2019
  4. Mingyang Jiang,Yiran Wu,Tianqi Zhao,Zelin Zhao, PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation,https://arxiv.org/pdf/1807.00652.pdf, arXiv:1807.00652v2 [cs.CV] 24, Nov, 2018
  5. 임보트넷(ImVotenet), https://medium.com/codex/imvotenet-paper-review-and-code-analysis-bf103117b32e, (Accessed September 26, 2021)
  6. Loic Landrieu,Martin Simonovsky, Universite Paris-Est, LASTIG MATIS IGN, ENSG, Universite Paris-Est, Ecole des Ponts ParisTech,Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs, https://arxiv.org/pdf/1711.09869.pdf, arXiv:1711.09869v2[cs.CV] 28, Mar, 2018
  7. Martin Simon, Stefan Milz, Karl Amende, Horst-Michael Gross, Complex-YOLO: An Euler-Region-Proposal for Real-time 3D Object Detection on Point Clouds,https://arxiv.org/pdf/1803.06199.pdf, arXiv:1803.06199v2 [cs.CV] 24, Sep, 2018
  8. Timo Hackel, Nikolay Savinov, Lubor Ladicky, Jan D. Wegner, Konrad Schindler, Marc Pollefeys, SEMANTIC3D.NET: A NEW LARGE-SCALE POINT CLOUD CLASSIFICATION BENCHMARK,https://arxiv.org/pdf/1704.03847.pdf, arXiv:1704.03847v1 [cs.CV] 12, Apr, 2017
  9. Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, Andrew Markham, University of Oxford, Sun Yat-sen University, National University of Defense Technology, RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds, https://openaccess.thecvf.com/content_CVPR_2020/papers/Hu_RandLA-Net_Efficient_ Semantic_Segmentation_of_Large-Scale_Point_Clouds_CVPR_2020_paper.pdf, ICCV 2021: October 11th - 17th, Virtual
  10. Quang-Hieu Pham, Duc Thanh Nguyen, Binh-Son Hua, Gemma Roig, Sai-Kit Yeung, Singapore University of Technology and Design, Deakin University, The University of Tokyo, Hong Kong University of Science and Technology, JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields,https://arxiv.org/pdf/1904.00699.pdf, arXiv:1904.00699v2 [cs.CV] 5, Apr, 2019
  11. Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, Bharath Hariharan, Cornell University, Cornell Tech, NVIDIA,PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows, https://arxiv.org/pdf/1906.12320.pdf,arXiv:1906.12320v3 [cs.CV] 2, Sep, 2019
  12. Tai Wang, Xinge Zhu, Jiangmiao Pang, Dahua Lin, FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection, https://arxiv.org/pdf/2104.10956v3.pdf, arXiv:2104.10956v3 [cs.CV] 24, Sep, 2021
  13. Despoina Paschalidou, Angelos Katharopoulos, Andreas Geiger, Sanja Fidler,Max Planck Institute for Intelligent Systems Tubingen, University of Tubingen,Idiap Research Institute, Switzerland, Ecole Polytechique Federale de Lausanne(EPFL),Max Planck ETH Center for Learning Systems, NVIDIA, University of Toronto, Vector Institute,Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks, https://arxiv.org/pdf/2103.10429.pdf, arXiv:2103.10429v1 [cs.CV] 18, Mar, 2021
  14. Alex Yu Vickie Ye Matthew Tancik Angjoo Kanazawa, pixelNeRF: Neural Radiance Fields from One or Few Images,https://arxiv.org/pdf/2012.02190.pdf,arXiv:2012.02190v3 [cs.CV] 30, May, 2021
  15. Vincent Sitzmann, Semon Rezchikov, William T. Freeman, Joshua B. Tenenbaum, Fr?do Durand, MIT CSAIL , Columbia University, NSF IAFI, MIT BCS , NSF CBMM, Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering, https://arxiv.org/pdf/2106.02634.pdf, arXiv:2106.02634v1 [cs.CV] 4, Jun, 2021
  16. Vincent Sitzmann, Michael Zollh?fer, Gordon Wetzstein,Scene Representation Networks: Continuous Neural Scene 3D-Structure-Aware Representations, https://arxiv.org/pdf/1906.01618.pdf,arXiv:1906.01618v2 [cs.CV] 28, Jan, 2020
  17. Konstantinos Rematas, Ricardo Martin-Brualla, Vittorio Ferrari,ShaRF: Shape-conditioned Radiance Fields from a Single View, https://arxiv.org/pdf/2102.08860.pdf,arXiv:2102.08860v2 [cs.CV] 23, Jun, 2021
  18. Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler, NVIDIA, University of Toronto, Vector Institute, University of Waterloo,IMAGE GANS MEET DIFFERENTIABLE RENDERING FOR INVERSE GRAPHICS AND INTERPRETABLE 3D NEURAL RENDERING,https://arxiv.org/pdf/2010.09125.pdf,arXiv:2010.09125v2 [cs.CV] 20, Apr, 2021
  19. Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, Sanja Fidler,NVIDIA, University of Toronto, Vector Institute, McGill University, Aalto University, Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer, https://nv-tlabs.github.io/DIB-R/files/diff_shader.pdf, arXiv:1908.01210v2 [cs.CV] 21, Nov, 2019
  20. Jiajun Wu, Chengkai Zhang, Tianfan Xue,William T. Freeman, Joshua B. Tenenbaum,Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling,http://3dgan.csail.mit.edu/papers/3dgan_nips.pdf,arXiv: 1610.07584v2 [cs.CV] 4, Jan, 2017
  21. Eric R. Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein, Stanford University,pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis,https://arxiv.org/pdf/2012.00926.pdf,arXiv:2012.00926v2 [cs.CV] 5, Apr, 2021
  22. 3D-FCR-alphaGAN, https://github.com/yunishi3/3D-FCR-alphaGAN(Accessed September 26, 2021)
  23. Edward J. Smith, David Meger,Improved Adversarial Systems for 3D Object Generation and Reconstruction, https://arxiv.org/pdf/1707.09557.pdf,arXiv:1707.09557v3 [cs.CV] 30, Oct, 2017
  24. Michael Niemeyer, Andreas Geiger, Max Planck Institute for Intelligent Systems, Tubingen, University of Tubingen, GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields, http://www.cvlibs.net/publications/Niemeyer2021CVPR.pdf,CVPR 2021(oral, best paper award)
  25. Mattia Segu, Margarita Grinvald, Roland Siegwart, Federico Tombari, 3DSNet: Unsupervised Shape-to-Shape 3D Style Transfer, https://arxiv.org/pdf/2011.13388.pdf,arXiv:2011.13388v4 [cs.CV] 18, May, 2021
  26. NVIDIA Kaolin, https://developer.nvidia.com/nvidia-kaolin(Accessed September 26, 2021)
  27. GAN을 사용하여 환상적인 생물 만들기, https://ai.googleblog.com/2020/11/using-gans-to-create-fantastical.html,(Accessed September 26, 2021)