Current Optics and Photonics

Optical Society of Korea (한국광학회)
권호 표지
DOI QR코드

DOI QR Code

Multimode-fiber Speckle Image Reconstruction Based on Multiscale Convolution and a Multidimensional Attention Mechanism

  • Kai Liu (School of Optical-electrical and Computer Engineering, University of Shanghai for Science and Technology) ;
  • Leihong Zhang (School of Optical-electrical and Computer Engineering, University of Shanghai for Science and Technology) ;
  • Runchu Xu (School of Optical-electrical and Computer Engineering, University of Shanghai for Science and Technology) ;
  • Dawei Zhang (School of Optical-electrical and Computer Engineering, University of Shanghai for Science and Technology) ;
  • Haima Yang (School of Optical-electrical and Computer Engineering, University of Shanghai for Science and Technology) ;
  • Quan Sun (College of Advanced Interdisciplinary Studies, National University of Defense Technology)
  • Received : 2024.07.30
  • Accepted : 2024.09.03
  • Published : 2024.10.25
Download PDF
⟨ Previous Next ⟩

Abstract

Multimode fibers (MMFs) possess high information throughput and small core diameter, making them highly promising for applications such as endoscopy and communication. However, modal dispersion hinders the direct use of MMFs for image transmission. By training neural networks on time-series waveforms collected from MMFs it is possible to reconstruct images, transforming blurred speckle patterns into recognizable images. This paper proposes a fully convolutional neural-network model, MSMDFNet, for image restoration in MMFs. The network employs an encoder-decoder architecture, integrating multiscale convolutional modules in the decoding layers to enhance the receptive field for feature extraction. Additionally, attention mechanisms are incorporated from both spatial and channel dimensions, to improve the network's feature-perception capabilities. The algorithm demonstrates excellent performance on MNIST and Fashion-MNIST datasets collected through MMFs, showing significant improvements in various metrics such as SSIM.

Keywords

Acknowledgement

The authors thank the National Natural Science Foundation of China, Shanghai Industrial Collaborative Innovation Project, and the Development Fund for Shanghai Talents for help in identifying collaborators for this work.

References

  1. Z. Wen, Z. Dong, Q. Deng, C. Pang, C. F. Kaminski, X. Xu, H. Yan, L. Wang, S.Liu, J. Tang, W. Chen, X. Liu, and Q. Yang, "Single multimode fibre for in vivo light-field-encoded endoscopic imaging," Nat. Photon. 17, 679-687 (2023). 
  2. R. Y. Gu, R. N. Mahalati, and J. M. Kahn, "Design of flexible multi-mode fiber endoscope," Opt. Express 23, 26905-26918 (2015). 
  3. D. Psaltis and C. Moser, "Imaging with multimode fibers," Opt. Photon. News 27, 24-31 (2016). 
  4. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. P. Pakan, N. L. Rochefort, and T. Cizmar, "High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging," Light Sci. Appl. 7, 92 (2018). 
  5. B. Rahmani, I. Oguz, U. Tegin, J. Hsieh, D. Psaltis, and C. Moser, "Learning to image and compute with multimode optical fibers," Nanophotonics 11, 1071-1082 (2022). 
  6. Z. Liu, L. Wang, Y. Meng, T. He, S. He, Y. Yang, L. Wang, J. Tian, D. Li, P. Yan, M. Gong, Q. Liu, and Q. Xiao, "All-fiber high-speed image detection enabled by deep learning," Nat. Commun. 13, 1433 (2022). 
  7. M. Rehan, R. Chowdhury, P. Biswas, M. S. Kang, and S. K. Varshney, "Low-threshold cascaded Raman scattering and intermodal four-wave mixing in cascaded multimode fiber system," J. Light. Technol. 42, 4636-4642 (2024). 
  8. D. Loterie, S. Farahi, I. Papadopoulos, A. Goy, D. Psaltis, and C. Moser, "Digital confocal microscopy through a multimode fiber," Opt. Express 23, 23845-23858 (2015). 
  9. S. Sivankutty, E. R. Andresen, R. Cossart, G. Bouwmans, S. Monneret, and H. Rigneault, "Ultra-thin rigid endoscope: Two-photon imaging through a graded-index multi-mode fiber," Opt. Express 24, 825-841 (2016). 
  10. R. Horisaki, R. Takagi, and J. Tanida, "Learning-based imaging through scattering media," Opt. Express 24, 13738-13743 (2016). 
  11. W. Fan, Z. Chen, Z. Gezhi, L. Chen, X. Wang, and J. Pu, "Control the normalized polarization ratio of a focal spot in speckle field formed by non-polarization-maintaining multimode fiber," J. Opt. 21, 045704 (2019). 
  12. I. Gusachenko, M. Chen, and K. Dholakia, "Raman imaging through a single multimode fibre," Opt. Express 25, 13782-13798 (2017). 
  13. I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, "Focusing and scanning light through a multimode optical fiber using digital phase conjugation," Opt. Express 20, 10583-10590 (2012). 
  14. C. Ma, J. Di, Y. Li, F. Xiao, J. Zhang, K. Liu, X. Bai, and J. Zhao, "Rotational scanning and multiple-spot focusing through a multimode fiber based on digital optical phase conjugation," Appl. Phys. Express 11, 062501 (2018). 
  15. C. Ma, J. Di, Y. Zhang, P. Li, F. Xiao, K. Liu, X. Bai, and J. Zhao, "Reconstruction of structured laser beams through a multimode fiber based on digital optical phase conjugation," Opt. Lett. 43, 3333-3336 (2018). 
  16. S. Shimizu, T. Matsuura, M. Umezawa, K. Hiramoto, N. Miyamoto, K. Umegaki, and H. Shirato, "Preliminary analysis for integration of spot-scanning proton beam therapy and real-time imaging and gating," Phys. Med. 30, 555-558 (2014). 
  17. M. N'Gom, T. B. Norris, E. Michielssen, and R. R. Nadakuditi, "Mode control in a multimode fiber through acquiring its transmission matrix from a reference-less optical system," Opt. Lett. 43, 419-422 (2018). 
  18. S. Resisi, Y. Viernik, S. M. Popoff, and Y. Bromberg, "Wavefront shaping in multimode fibers by transmission matrix engineering," APL Photonics 5, 036103 (2020). 
  19. R. Xu, L. Zhang, Z. Chen, Z. Wang, and D. Zhang, "High accuracy transmission and recognition of complex images through multimode fibers using deep learning," Laser Photonics Rev. 17, 2200339 (2023). 
  20. B. Song, C. Jin, J. Wu, W. Lin, B. Liu, W. Huang, and S. Chen, "Deep learning image transmission through a multimode fiber based on a small training dataset," Opt. Express 30, 5657-5672 (2022). 
  21. P. Tang, K. Zheng, W. Yuan, T. Pan, Y. Xu, S. Fu, Y. Wang, and Y. Qin, "Learning to transmit images through optical speckle of a multimode fiber with high fidelity," Appl. Phys. Lett. 121, 081107 (2022). 
  22. W. Zhang, S. Wang, H. Liu, C. Hu, Y. Zheng, and X. Yan, "REcaNet: Residual neural networks with initialized weights and attention mechanism for image propagating in multimode optical fiber restoration," Egypt. Inform. J. 24, 100404 (2023). 
  23. E. Kakkava, B. Rahmani, N. Borhani, U. Tegin, D. Loterie, G. Konstantinou, C. Moser, and D. Psaltis, "Imaging through multimode fibers using deep learning: The effects of intensity versus holographic recording of the speckle pattern," Opt. Fiber Technol. 52, 101985 (2019). 
  24. L. Zhang, R. Xu, H. Ye, K. Wang, B. Xu, and D. Zhang, "High definition images transmission through single multimode fiber using deep learning and simulation speckles," Opt. Lasers Eng. 140, 106531 (2021). 
  25. B. Rahmani, D. Loterie, G. Konstantinou, D. Psaltis, and C. Moser, "Multimode optical fiber transmission with a deep learning network," Light Sci. Appl. 7, 69 (2018). 
  26. P. Fan, T. Zhao, and L. Su, "Deep learning the high variability and randomness inside multimode fibers," Opt. Express 27, 20241-20258 (2019). 
  27. S. Resisi, S. M. Popoff, and Y. Bromberg, "Image transmission through a dynamically perturbed multimode fiber by deep learning," Laser Photonics Rev. 15, 2000553 (2021). 
  28. X. Hu, J. Zhao, J. E. Antonio-Lopez, R. A. Correa, and A. Schulzgen, "Unsupervised full-color cellular image reconstruction through disordered optical fiber," Light Sci. Appl. 12, 125 (2023). 
  29. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Proc. 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015) (Munich, Germany, Oct. 5-9, 2015), pp. 234-241. 
  30. L. Deng, "The MNIST database of handwritten digit images for machine learning research [Best of the web]," IEEE Signal Process Mag. 29, 141-142 (2012). 
  31. H. Xiao, K. Rasul, and R. Vollgraf, "Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms," arXiv: 1708.07747 (2017). 
  32. A. Nawaz, U. Akram, A. A. Salam, A. R. Ali, A. U. Rehman, and J. Zeb, "VGG-UNET for brain tumor segmentation and ensemble model for survival prediction," in Proc. 2021 International Conference on Robotics and Automation in Industry (ICRAI) (Karachi, Pakistan, Nov. 21-22, 2021), pp. 1-6. 
  33. M. Z. Alom, M. Hasan, C. Yakopcic, T. M. Taha, and V. K. Asari, "Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation," arXiv: 1802.06955 (2018).