Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Watermarking for Digital Hologram by a Deep Neural Network and its Training Considering the Hologram Data Characteristics

딥 뉴럴 네트워크에 의한 디지털 홀로그램의 워터마킹 및 홀로그램 데이터 특성을 고려한 학습

  • Received : 2021.02.02
  • Accepted : 2021.04.09
  • Published : 2021.05.30
Download PDF
⟨ Previous Next ⟩

Abstract

A digital hologram (DH) is an ultra-high value-added video content that includes 3D information in 2D data. Therefore, its intellectual property rights must be protected for its distribution. For this, this paper proposes a watermarking method of DH using a deep neural network. This method is a watermark (WM) invisibility, attack robustness, and blind watermarking method that does not use host information in WM extraction. The proposed network consists of four sub-networks: pre-processing for each of the host and WM, WM embedding watermark, and WM extracting watermark. This network expand the WM data to the host instead of shrinking host data to WM and concatenate it to the host to insert the WM by considering the characteristics of a DH having a strong high frequency component. In addition, in the training of this network, the difference in performance according to the data distribution property of DH is identified, and a method of selecting a training data set with the best performance in all types of DH is presented. The proposed method is tested for various types and strengths of attacks to show its performance. It also shows that this method has high practicality as it operates independently of the resolution of the host DH and WM data.

디지털 홀로그램(digital hologram, DH)은 2차원 데이터에 3차원의 정보를 포함하는 초고부가가치의 영상 콘텐츠이다. 따라서 이 콘텐츠의 유통을 위해서는 그 지적재산권이 반드시 보호되어야 한다. 본 논문에서는 이를 위해서 최초로 딥 뉴럴 네트워크를 이용한 DH의 워터마킹 방법을 제안한다. 이 방법은 워터마크(watermark, WM)가 의 비가시성, 공격에 대한 강인성, WM 추출 시 호스트 정보를 사용하지 않는 blind 워터마킹 방법이다. 제안하는 네트워크는 호스트와 워터마크 각각의 전처리, WM 삽입, WM 추출의 네 부-네트워크로 구성된다. 이 네트워크는 고주파 성분이 강한 DH의 특성을 감안하여 호스트 데이터를 축소하지 않고 WM 데이터를 확장하여 호스트 데이터와 정합함으로써 WM를 삽입한다. 또한 이 네트워크의 학습에 있어서 DH의 데이터 분포특성에 따른 성능의 차이를 확인하고, 모든 종류의 DH에서 최고의 성능을 갖는 학습 데이터 세트를 선정하는 방법을 제시한다. 제안한 방법을 다양한 종류와 강도의 공격에 대해 실험을 수행하여 그 성능을 보인다. 또한 이 방법이 호스트 DH의 해상도와 WM 데이터에 독립적으로 동작하여 높은 실용성을 갖는다는 것을 보인다.

Keywords

Acknowledgement

The present research has been conducted by the Excellent researcher support project of Kwangwoon University in 2021. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2019R1F1A105452).

References

  1. Stephen A. Benton and V. Michael Bove Jr., Holographic Imaging, John Willy and Sons Inc., Hoboken, NJ, 2008.
  2. Ingemar J. Cox, Mattew L. Miller, Jeffrey A. Bloom, and Ton Kalker, Digital Watermarking and Steganography, Morgan Kaufman Publisher, Burlington, MA, 2008.
  3. S. Kishk and B. Javidi, "3D object watermarking by a 3D hidden object," Opt. Express 11, 874-888, 21 April 2003. https://doi.org/10.1364/OE.11.000874
  4. H. Kim, and Y H. Lee, "Optimal watermarking of digital hologram of 3-D object," Optics Express 13, 2881-2886, 18 April 2005. https://doi.org/10.1364/OPEX.13.002881
  5. H. Choi, Y. Seo, J. Yoo, and D. Kim, "An Electronic Watermarking Technique for Digital Holograms in a DWT Domain." In: Chang LW., Lie WN. (eds) Advances in Image and Video Technology. PSIVT 2006. Lecture Notes in Computer Science, vol 4319. Springer, Berlin, Heidelberg, December 2006.
  6. H. Choi, Y. Seo, J. Yoo, and D. Kim. "Digital watermarking technique for holography interference patterns in a transform domain," Optics and Lasers in Engineering. 46, 343-348, 1 April 2008. https://doi.org/10.1016/j.optlaseng.2007.11.005
  7. Y. Lee, Y. Seo, and D. Kim "Digital hologram watermarking by embedding Fresnel-diffracted watermark data," Optical Engineering 58(3), 035102, 20 March 2019.
  8. X. Li, Y. Wang, Q. Wang, S. Kim and X. Zhou, "Copyright Protection for Holographic Video Using Spatiotemporal Consistent Embedding Strategy," in IEEE Transactions on Industrial Informatics, vol. 15, no. 11, pp. 6187-6197, November 2019. https://doi.org/10.1109/tii.2019.2897733
  9. J. Zhu, R. Kaplan, J. Johnson, and L. Fei-Fei, "HiDDeN: hiding data with deep networks," arXiv:1807.09937, July, 2018.
  10. M. Ahmadi, A. Norouzi, S. M. Reza Soroushmehr, N. Karimi, K. Najarian, S. Samavi, and A. Emami, "ReDMark: framework for residual diffusion watermarking on deep networks," arXiv:1810.07248, Dec. 2018
  11. J. Lee, Y. Seo, D. Kim, "Convolutional Neural Network-Based Digital Image Watermarking Adaptive to the Resolution of Image and Watermark." Applied Sciences, 2020; 10(19):6854, 29 September 2020. https://doi.org/10.3390/app10196854
  12. J. Lee, J. Jeong, H. Cho, D. Yoo, B. Lee, and B. Lee, "Deep neural network for multi-depth hologram generation and its training strategy," Optics Express, Vol. 28, No. 18, pp. 27137-27154, 31 August 2020. https://doi.org/10.1364/OE.402317
  13. J. Kang, J. Lee, Y. Lee, D. Kim, and Y. Seo, "Interference pattern generation by using deep learning based on GAN," ITC-CSCC, pp. 1-2, 2019.
  14. Dataset of hologram data. URL http://plenodb.jpeg.org/
  15. "Open Access Database for Experimental Validations of Holographic Compression Engines" for the 7th International Workshop on Quality of Multimedia Experience (QoMex 2015).
  16. Sergey Loffe and Christian Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," arXiv:1502.03167v3 [csLG], 2 Mar 2015.
  17. Symeonidou, A., Blinder, D., Ahar, A., Schretter, C., Munteanu, A., and Schelkens, P., "Speckle noise reduction for computer generated holograms of objects with diffuse surfaces," Proc. SPIE 9896, (2016).
  18. A. Symeonidou, D. Blinder, P. Schelkens, "Colour computer-generated holography for point clouds utilizing the Phong illumination model," Optics Express (2018).
  19. Warsaw University of Technology , http://ds.erc-interfere.eu/holodb/
  20. A. Gilles, P. Gioia, R. Cozot, and L. Morin, "Hybrid approach for fast occlusion processing in computer-generated hologram calculation," Applied. Optics, vol. 55, no. 20, pp. 5459-5470, July. 2016. https://doi.org/10.1364/AO.55.005459
  21. A. Gilles, P. Gioia, R. Cozot, and L. Morin, "Computer generated hologram from Multiview-plus-Depth data considering specular reflections," in 2016 IEEE ICMEW, pp. 1-6, 2016.
  22. Marco V. Bernardo, Pedro Fernandes, Angelo Arrifano, Marc Antonini, Elsa Fonseca, Paulo T. Fiadeiro, Antonio M.G. Pinheiro, Manuela Pereira, "Holographic representation: Hologram plane vs. object plane," Signal Processing: Image Communication, Vol. 68, pp. 193-206, October 2018. https://doi.org/10.1016/j.image.2018.08.006
  23. Warsaw University of Technology, http://plenodb.jpeg.org/holo/WUT