Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A Progressive Rendering Method to Enhance the Resolution of Point Cloud Contents

포인트 클라우드 콘텐츠 해상도 향상을 위한 점진적 렌더링 방법

  • Lee, Heejea (Department of Computer Science, Hanyang University) ;
  • Yun, Junyoung (Department of Computer Science, Hanyang University) ;
  • Kim, Jongwook (Department of Computer Science, Hanyang University) ;
  • Kim, Chanhee (Department of Computer Science, Hanyang University) ;
  • Park, Jong-Il (Department of Computer Science, Hanyang University)
  • 이희제 (한양대학교 컴퓨터소프트웨어학과) ;
  • 윤준영 (한양대학교 컴퓨터소프트웨어학과) ;
  • 김종욱 (한양대학교 컴퓨터소프트웨어학과) ;
  • 김찬희 (한양대학교 컴퓨터소프트웨어학과) ;
  • 박종일 (한양대학교 컴퓨터소프트웨어학과)
  • Received : 2021.04.09
  • Accepted : 2021.05.24
  • Published : 2021.05.30
Download PDF
⟨ Previous Next ⟩

Abstract

Point cloud content is immersive content that represents real-world objects with three-dimensional (3D) points. In the process of acquiring point cloud data or encoding and decoding point cloud data, the resolution of point cloud content could be degraded. In this paper, we propose a method of progressively enhancing the resolution of sequential point cloud contents through inter-frame registration. To register a point cloud, the iterative closest point (ICP) algorithm is commonly used. Existing ICP algorithms can transform rigid bodies, but there is a disadvantage that transformation is not possible for non-rigid bodies having motion vectors in different directions locally, such as point cloud content. We overcome the limitations of the existing ICP-based method by registering regions with motion vectors in different directions locally between the point cloud content of the current frame and the previous frame. In this manner, the resolution of the point cloud content with geometric movement is enhanced through the process of registering points between frames. We provide four different point cloud content that has been enhanced with our method in the experiment.

포인트 클라우드 콘텐츠는 3차원 포인트로 실제 객체를 나타내는 몰입형 콘텐츠이다. 포인트 클라우드 데이터를 획득하거나 포인트 클라우드 데이터를 인코딩 및 디코딩하는 과정에서 포인트 클라우드 콘텐츠의 해상도가 저하될 수 있다. 본 논문에서는 프레임 간 정합을 통해 순차적으로 포인트 클라우드 콘텐츠의 해상도를 점진적으로 향상시키는 방법을 제안한다. 포인트 클라우드 데이터를 정합하기 위해 ICP(Iterative Closest Point) 알고리즘이 일반적으로 사용된다. 기존 ICP 알고리즘은 강체를 변환할 수 있지만 포인트 클라우드 콘텐츠와 같이 로컬에서 서로 다른 방향으로 모션 벡터를 갖는 비 강체에 대해서는 변환이 불가능하다는 단점이 있다. 현재 프레임의 포인트 클라우드와 이전 프레임 사이의 포인트를 쌍을 만들고 만들어진 쌍의 움직임양을 계산하여 보상해주는 방법으로 기존 ICP 정합에서의 한계를 극복하였다. 이러한 방식으로 프레임 사이에 포인트를 정합하는 과정을 통해 기하학적 움직임이 있는 포인트 클라우드 콘텐츠의 해상도가 향상됨을 보였다.

Keywords

Acknowledgement

이 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임(No.2020-0-00452, 적응형 뷰어 중심 포인트 클라우드AR/VR 스트리밍 플랫폼 기술 개발) This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. 2019R1F1A1041882).

References

  1. BHAYANI, Sam B.; ANDRIOLE, Gerald L. Three-dimensional (3D) vision: does it improve laparoscopic skills? An assessment of a 3D head-mounted visualization system. Reviews in urology, 7.4: 211, 2005.
  2. D. B. Payne and J. R. Stern, "Wavelength-switched passively coupled single-mode optical network," in Proc. IOOC-ECOC, Boston, MA, USA, pp. 585-590, 1985.
  3. MITRA, Niloy J., et al. Registration of point cloud data from a geometric optimization perspective. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, p. 22-31, 2004.
  4. Arun, Somani; Thomas S. Huang; Steven D. Blostein (1987). "Least-square fitting of two 3-D point sets". IEEE Pattern Analysis and Machine Intelligence, 1987
  5. Besl, Paul J.; N.D. McKay (1992). "A Method for Registration of 3-D Shapes". IEEE Transactions on Pattern Analysis and Machine Intelligence. 14 (2): 239-256. doi:10.1109/34.121791.
  6. Chen, Yang; Gerard Medioni (1991). "Object modelling by registration of multiple range images". Image Vision Computing. 10(3): 145-155. doi:10.1016/0262-8856(92)90066-C.
  7. Zhang, Zhengyou (1994). "Iterative point matching for registration of free-form curves and surfaces". International Journal of Computer Vision. 13 (12): 119-152. https://doi.org/10.1007/BF01427149
  8. Rusinkiewicz, Szymon; Marc Levoy (2001). Efficient Variants of the ICP Algorithm. Proceedings Third International Conference on 3-D Digital Imaging and Modeling. Quebec City, Quebec, Canada. pp. 145-152. doi:10.1109/IM.2001.924423.
  9. Pomerleau, Francois; Colas, Francis; Siegwart, Roland (2015). "A Review of Point Cloud Registration Algorithms for Mobile Robotics". Foundations and Trends in Robotics. 4 (1): 1-104. CiteSeerX 10.1.1.709.2212. doi:10.1561/2300000035.
  10. LOW, Kok-Lim. Linear least-squares optimization for point-to-plane icp surface registration. Chapel Hill, University of North Carolina, 2004, 4.10: 1-3.
  11. RUSINKIEWICZ, Szymon; LEVOY, Marc. Efficient variants of the ICP algorithm. In: Proceedings third international conference on 3-D digital imaging and modeling. IEEE, p. 145-152, 2001.
  12. SEGAL, Aleksandr; HAEHNEL, Dirk; THRUN, Sebastian. Generalized-icp. In: Robotics: science and systems. p. 435. 2009
  13. Francois Pomerleau, Francis Colas, Roland Siegwart, and Stephane Magnenat. Comparing ICP Variants on Real-World Data Sets. In Autonomous Robots, 34(3), pages 133-148, April 2013. https://doi.org/10.1007/s10514-013-9327-2
  14. MEN, Hao; GEBRE, Biruk; POCHIRAJU, Kishore. Color point cloud registration with 4D ICP algorithm. In: 2011 IEEE International Conference on Robotics and Automation. IEEE, p. 1511-1516, 2011.
  15. Holz, Dirk; Ichim, Alexandru E.; Tombari, Federico; Rusu, Radu B.; Behnke, Sven (2015). "Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D". IEEE Robotics Automation Magazine. 22 (4): 110-124. https://doi.org/10.1109/MRA.2015.2432331
  16. AOKI, Yasuhiro, et al. Pointnetlk: Robust & efficient point cloud registration using pointnet. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 7163-7172, 2019.
  17. POMERLEAU, Francois, et al. Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research, 31.14: 1705-1711, 2012. https://doi.org/10.1177/0278364912458814
  18. SCHUETZ, Markus; WIMMER, Michael. Progressive real-time rendering of unprocessed point clouds. In: ACM SIGGRAPH 2018 Posters, p. 1-2, 2018.
  19. HE, Jun; YAO, Xin. Drift analysis and average time complexity of evolutionary algorithms. Artificial intelligence, 127.1: 57-85, 2001. https://doi.org/10.1016/S0004-3702(01)00058-3
  20. ZWICKER, Matthias, et al. EWA splatting. IEEE Transactions on Visualization and Computer Graphics, 8.3: 223-238, 2002. https://doi.org/10.1109/TVCG.2002.1021576
  21. WU, Huikai; ZHANG, Junge; HUANG, Kaiqi. Point cloud super resolution with adversarial residual graph networks. arXiv preprint arXiv:1908.02111, 2019.
  22. YU, Lequan, et al. Pu-net: Point cloud upsampling network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 2790-2799, 2018.
  23. PUNET, Xavier, et al. Enhanced cell-material interactions through the biofunctionalization of polymeric surfaces with engineered peptides. Biomacromolecules, 14.8: 2690-2702, 2013. https://doi.org/10.1021/bm4005436
  24. ZHANG, Zhiyuan; DAI, Yuchao; SUN, Jiadai. Deep learning based point cloud registration: an overview. Virtual Reality & Intelligent Hardware, 2.3: 222-246, 2020. https://doi.org/10.1016/j.vrih.2020.05.002
  25. MIELEKAMP, Pieter Maria; HOMAN, Robert Johannes Frederik. Visualization of 3D images in combination with 2D projection images. U.S. Patent No 7,991,105, 2011.
  26. SARAVANAN, G.; YAMUNA, G.; NANDHINI, S. Real time implementation of RGB to HSV/HSI/HSL and its reverse color space models. In: 2016 International Conference on Communication and Signal Processing (ICCSP). IEEE, p. 0462-0466, 2016.
  27. HOUSHIAR, Hamidreza; NuCHTER, Andreas. 3D point cloud compression using conventional image compression for efficient data transmission. In: 2015 XXV International Conference on Information, Communication and Automation Technologies (ICAT). IEEE, p. 1-8, 2015.
  28. SHI, Shaoshuai; WANG, Xiaogang; LI, Hongsheng. Pointrcnn: 3d object proposal generation and detection from point cloud. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 770-779, 2019.
  29. Call for Proposals for Point Cloud Compression v2, Standard ISO/IECJTC1/SC29/WG11 MPEG2017/N16763, Hobart, AU, Apr. 2017.
  30. RHYU, Sungryeul, et al. Contextual Homogeneity-Based Patch Decomposition Method for Higher Point Cloud Compression. IEEE Access, 8: 207805-207812, 2020. https://doi.org/10.1109/ACCESS.2020.3038800
  31. MIAO, Yongwei; PAJAROLA, Renato; FENG, Jieqing. Curvature-aware adaptive re-sampling for point-sampled geometry. Computer-Aided Design, 2009, 41.6: 395-403, 2009. https://doi.org/10.1016/j.cad.2009.01.006
  32. CARDANI, Darrin. Adventures in hsv space. Laboratorio de Robotica, Instituto Tecnologico Autonomo de Mexico, 2001.
  33. H.Lee, J.Yun, J.Kim, J.Park, "ICP based Point Cloud Contents Quality Improvement Method," The Korean Institute of Broadcast and Media Engineers, p. 200-201, 2020.