Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A Technique for Interpreting and Adjusting Depth Information of each Plane by Applying an Object Detection Algorithm to Multi-plane Light-field Image Converted from Hologram Image

Light-field 이미지로 변환된 다중 평면 홀로그램 영상에 대해 객체 검출 알고리즘을 적용한 평면별 객체의 깊이 정보 해석 및 조절 기법

  • Young-Gyu Bae (School of electrical and electronic engineering, Kyungpook National University) ;
  • Dong-Ha Shin (School of electrical and electronic engineering, Kyungpook National University) ;
  • Seung-Yeol Lee (School of electrical and electronic engineering, Kyungpook National University)
  • 배영규 (경북대학교 전자전기공학부) ;
  • 신동하 (경북대학교 전자전기공학부) ;
  • 이승열 (경북대학교 전자전기공학부)
  • Received : 2022.12.02
  • Accepted : 2022.12.26
  • Published : 2023.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

Directly converting the focal depth and image size of computer-generated-hologram (CGH), which is obtained by calculating the interference pattern of light from the 3D image, is known to be quite difficult because of the less similarity between the CGH and the original image. This paper proposes a method for separately converting the each of focal length of the given CGH, which is composed of multi-depth images. Firstly, the proposed technique converts the 3D image reproduced from the CGH into a Light-Field (LF) image composed of a set of 2D images observed from various angles, and the positions of the moving objects for each observed views are checked using an object detection algorithm YOLOv5 (You-Only-Look-Once-version-5). After that, by adjusting the positions of objects, the depth-transformed LF image and CGH are generated. Numerical simulations and experimental results show that the proposed technique can change the focal length within a range of about 3 cm without significant loss of the image quality when applied to the image which have original depth of 10 cm, with a spatial light modulator which has a pixel size of 3.6 ㎛ and a resolution of 3840⨯2160.

재생하고자 하는 3차원 이미지에서 발현되는 빛의 간섭 무늬를 계산하여 얻게 되는 Computer Generated Hologram(CGH)은 본래의 3차원 이미지와 유사관계를 찾기 힘든 형태로 형성되기에 직접적인 초점 위치 혹은 크기 등의 변환이 어려운 것으로 알려져 있다. 본논문은 이러한 문제 중 하나인 다중 평면으로 구성된 3차원 이미지 CGH의 평면별 초점 거리를 변환하는 문제를 해결하는 기술을 제안한다. 제안하는 기술은 CGH로부터 재생되는 3차원 이미지를 여러 각도에서 관측한 2차원 이미지의 집합으로 구성된 Light-Field (LF) 이미지로 변환하고, 관측한 각도별로 이동하는 객체의 위치를 객체 탐지 알고리즘인 YOLOv5(You Only Look Once version 5)로 분석한 뒤, 이를 조절함으로써 초점 거리가 변환된 LF 이미지와 이를 역변환한 결과인 CGH를 생성한다. 해당 기술은 CGH의 픽셀 사이즈가 3.6 ㎛, 해상도가 3840⨯2160인 상황에서 10 cm 거리에 재생되는 상에 적용되어 영상 품질의 큰 손실 없이 약 3 cm 정도의 범위에서 초점 거리를 변환시킬 수 있음을 시뮬레이션 분석과 실제 실험 관측을 통해 확인하였다.

Keywords

Acknowledgement

이 논문은 2019년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2019-0-00001, 홀로그램 영상 서비스를 위한 Holo-TV 핵심 기술 개발)

References

  1. G. Ruffato, R. Rossi, M. Massari, E. Mafakheri, P. Capaldo, and F. Romanato, "Design, fabrication and characterization of Computer Generated Holograms for anti-counterfeiting applications using OAM beams as light decoders," Sci. Rep., Vol. 7, No. 18011, Dec 2017 doi: https://doi.org/10.1038/s41598-017-18147-7
  2. Y. Yang, Q. Li, and G. P. Wang, "Design and fabrication of diverse metamaterial structures by holographic lithography," Opt. Express, Vol. 16, No. 15, pp 11275-11280, Jul 2008 doi: https://doi.org/10.1364/OE.16.011275
  3. D. Gabor, "Microscopy by reconstructed wave-fronts," Proc. R. Soc. Lond., Vol. 197, No. 1051, pp 11275-487, Jul 1949 doi: https://doi.org/10.1098/rspa.1949.0075
  4. D. Pi, J. Liu, and Y. Wang, "Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display," Light Sci. Appl., Vol. 11, No. 231, Jul 2022 doi: https://doi.org/10.1038/s41377-022-00916-3
  5. E. Sahin, E. Stoykova, J. Makinen, and A. Gotchev, "ComputerGenerated Holograms for 3D Imaging: A Survey," ACM Comput. Surv., Vol. 53, No. 2, pp 1-35, Mar 2021 doi: https://doi.org/10.1145/3378444
  6. K. Matsushima, "Computer-generated holograms for three dimensional surface objects with shade and texture," Appl. Opt, Vol. 44, pp 4607-4614, Aug 2005 doi: https://doi.org/10.1364/AO.44.004607
  7. Y. Rivenson, A. Stern, and B. Javidi, "Compressive Fresnel Holography," J. Display Technol., Vol. 6, No. 10 pp 506-509, 2010 doi: https://doi.org/10.1109/JDT.2010.2042276
  8. Y. Peng, C. Suyeon, N. Padmanaban, G. Wetzstein, "Neural holography with camera-in-the-loop training," ACM Trans. Graph., Vol. 44, No. 6, pp 1-14, Dec 2020 doi: https://doi.org/10.1145/3414685.3417802
  9. C. Chen, B. Lee, N.-N. Li, M. Chae, D. Wang, Q.-H. Wang, and B. Lee, "Multi-depth hologram generation using stochastic gradient descent algorithm with complex loss function," Opt. Express, Vol. 29, No. 10, pp 15089-15103, Apr 2021. doi: https://doi.org/10.1364/OE.425077
  10. Z. Huang, and L. Cao, "Bicubic interpolation and extrapolation iteration method for high resolution digital holographic reconstruction," Opt. Lasers Eng., Vol. 130, Jul 2020 doi: https://doi.org/10.1016/j.optlaseng.2020.106090
  11. J. Zhao, Z. Gao, Y. Niu, and L. Deng, "A method for improving the resolution of single-frame off-axis holograms based on extrapolation iterations and Kronecker interpolation," Measurement, Vol. 199, Aug 2022 doi: https://doi.org/10.1016/j.measurement.2022.111531
  12. T. Shimobaba, M. Makowski, T. Kakue, M. Oikawa, N. Okada, Y. Endo, R. Hirayama, and T. Ito, "Lensless zoomable holographic projection using scaled Fresnel diffraction," Opt. Express, Vol. 21, No. 21, pp 25285-25290, Oct 2013 doi: https://doi.org/10.1364/OE.21.025285
  13. D.-Y. Park, and J.-H. Park, "Generation of distortion-free scaled holograms using light field data conversion," Opt. Express, Vol. 29, No. 1, pp 487-508, Jan 2021 doi: https://doi.org/10.1364/OE.412986
  14. R. Girshick, J. Donahue, T. Darrell, and J. Malik , "Rich feature hierarchies for accurate object detection and semantic segmentation," IEEE Conference on Computer Vision and Pattern Recognition, Ohio, USA, pp 580-587, Jun 2014
  15. K. He, X. Zhang, S. Ren, J. Sun, "Deep Residual Learning for Image Recognition," IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 770-778, Jun 2016
  16. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 779-788, Jun 2016
  17. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," European Conference on Computer Vision, Amsterdam, Netherlands, pp 21-37, Oct 2016