Current Optics and Photonics

Optical Society of Korea (한국광학회)
권호 표지
DOI QR코드

DOI QR Code

Volume-sharing Multi-aperture Imaging (VMAI): A Potential Approach for Volume Reduction for Space-borne Imagers

  • Jun Ho Lee (Department of Optical Engineering, Kongju National University) ;
  • Seok Gi Han (Department of Optical Engineering, Kongju National University) ;
  • Do Hee Kim (Department of Optical Engineering, Kongju National University) ;
  • Seokyoung Ju (Department of Optical Engineering, Kongju National University) ;
  • Tae Kyung Lee (Department of Mathematical Sciences, Seoul National University) ;
  • Chang Hoon Song (Department of Mathematical Sciences, Seoul National University) ;
  • Myoungjoo Kang (Department of Mathematical Sciences, Seoul National University) ;
  • Seonghui Kim (Telepix Ltd.) ;
  • Seohyun Seong (Telepix Ltd.)
  • Received : 2023.08.01
  • Accepted : 2023.09.02
  • Published : 2023.10.25
Download PDF
⟨ Previous Next ⟩

Abstract

This paper introduces volume-sharing multi-aperture imaging (VMAI), a potential approach proposed for volume reduction in space-borne imagers, with the aim of achieving high-resolution ground spatial imagery using deep learning methods, with reduced volume compared to conventional approaches. As an intermediate step in the VMAI payload development, we present a phase-1 design targeting a 1-meter ground sampling distance (GSD) at 500 km altitude. Although its optical imaging capability does not surpass conventional approaches, it remains attractive for specific applications on small satellite platforms, particularly surveillance missions. The design integrates one wide-field and three narrow-field cameras with volume sharing and no optical interference. Capturing independent images from the four cameras, the payload emulates a large circular aperture to address diffraction and synthesizes high-resolution images using deep learning. Computational simulations validated the VMAI approach, while addressing challenges like lower signal-to-noise (SNR) values resulting from aperture segmentation. Future work will focus on further reducing the volume and refining SNR management.

Keywords

Acknowledgement

Challengeable Future Defense Technology Research and Development Program through the Agency for Defense Development (ADD) funded by the Defense Acquisition Program Administration in 2021 (No. 915020201).

References

  1. Q. Zhao, L. Yu, Z. Du, D. Peng, P. Hao, Y. Zhang, and P. Gong, "An overview of the applications of Earth observation satellite data: Impacts and future trends," Remote Sens. 14, 1863 (2022).
  2. J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts & Co., USA, 2005).
  3. S. E. Qian, Optical Payloads for Space Missions (Wiley, USA, 2015).
  4. C. Toth and G. Jozkow, "Remote sensing platforms and sensors: A survey," ISPRS J. Photogramm. Remote Sens. 115, 22-36 (2016). https://doi.org/10.1016/j.isprsjprs.2015.10.004
  5. R. Roy and J. Miller, "Miniaturization of image sensors: The role of innovations in complementary technologies in overcoming technological trade-offs associated with product innovation," J. Eng. Technol. Manag. 44, 58-69 (2017). https://doi.org/10.1016/j.jengtecman.2017.01.002
  6. R. N. Wilson, Reflecting telescope optics I: Basic design theory and its historical development, 2nd Ed. (Springer Berlin, Germany, 2004).
  7. D. Koresh, Reflective Optics (Academic Press, USA, 1991).
  8. V. Costes, G. Cassar, and L. Escarrat, "Optical design of a compact telescope for the next generation Earth observation system," Proc. SPIE 10564, 1056516 (2017).
  9. S. Grabarnik, M. Taccola, L. Maresi, V. Moreau, L. de Vos, J. Versluys, and G. Gubbels, "Compact multispectral and hyperspectral imagers based on a wide field of view TMA," Proc. SPIE 10565, 105605 (2017).
  10. B. Fan, W.-J. Cai, and Y. Huang, "Design and test of a high performance off-axis TMA telescope," Proc. SPIE 10564, 1056417 (2017).
  11. S.-T. Chang, Y.-C. Lin, C.-C. Lien, T.-M. Huang, H.-L. Tsay, and J.-J. Miau, "The design and assembly of a long-focal-length telescope with aluminum mirrors," Proc. SPIE 11180, 111806U (2019).
  12. M. Metwally, T. M. Bazan, and F. Eltehamy, "Design of very high-resolution satellite telescopes part I: Optical system design," IEEE Trans. Aerosp. Electron. Syst. 56, 1202-1208 (2020). https://doi.org/10.1109/TAES.2019.2929969
  13. J.-I. Bae, H.-B. Lee, J.-W. Kim, and M.-W. Kim, "Design of all-SiC lightweight secondary and tertiary mirrors for use in spaceborne telescopes," Curr. Opt. Photonics 6, 60-68 (2022).
  14. R. L. Kendrick, A. Duncan, C. Ogden, J. Wilm, and S.T. Thurman, "Segmented planar imaging detector for EO reconnaissance," in Computational Optical Sensing and Imaging 2013 (Optica Publishing Group, 2013), paper CM4C.1.
  15. G. Carles, G. Muyo, N. Bustin, A. Wood, and A. R. Harvey, "Compact multi-aperture imaging with high angular resolution," J. Opt. Soc. Am. A 32, 411-419 (2015). https://doi.org/10.1364/JOSAA.32.000411
  16. G. Carles and A. R. Harvey, "Multi-aperture imaging for flat cameras," Opt. Lett. 45, 6182-6185 (2020). https://doi.org/10.1364/OL.405702
  17. G. Lv, H. Xu, H. Feng, Z. Xu, H. Zhou, Q. Li, and Y. Chen, "A full-aperture imaging synthesis method for the rotating rectangular aperture system using Fourier spectrum restoration," Photonics 8, 522 (2021).
  18. D. J. Brady, W. Pang, H. Li, Z. Ma, Y. Tao, and X. Cao, "Parallel cameras," Optica 5, 127-137 (2018). https://doi.org/10.1364/OPTICA.5.000127
  19. E. Tseng, S. Colburn, J. Whitehead, L. Huang, S.-H. Baek, A. Majumdar, and F. Heide, "Neural nano-optics for high-quality thin lens imaging," Nat. Commun. 12, 6493 (2021).
  20. X. Liu, J. Deng, K. F. Li, M. Jin, Y. Tang, X. Zhang, X. Cheng, H. Wang, W. Liu, and G. Li, "Optical telescope with Cassegrain metasurfaces," Nanophotonics 9, 3263-3269 (2020). https://doi.org/10.1515/nanoph-2020-0012
  21. K. Zhang, C. Yang, X. Li, C. Zhou, and R. Zhong, "High-efficiency microsatellite-using super-resolution algorithm based on the multi-modality super-CMOS sensor," Sensors 20, 4019 (2020).
  22. L. Ma, Y. Liu, X. Zhang, Y. Ye, G. Yin, and B. A. Johnson, "Deep learning in remote sensing applications: A meta-analysis and review," ISPRS J. Photogramm. Remote Sens. 152, 166-177 (2019). https://doi.org/10.1016/j.isprsjprs.2019.04.015
  23. S. Li, X. Kang, L. Fang, J. Hu, and H. Yin, "Pixel-level image fusion: A survey of the state of the art," Inform. Fusion 33, 100-112 (2017). https://doi.org/10.1016/j.inffus.2016.05.004
  24. H. Kaur, D. Koundal, and V. Kdyan, "Image Fusion chnqieus: A survey," Arch. Computat. Methods Eng. 28, 4425-4447 (2021). https://doi.org/10.1007/s11831-021-09540-7
  25. R. D. Flete and B. D. Paul, "Modelling the optical transfer function in the imaging chain," Opt. Eng. 53, 083013 (2014).
  26. A. M. John, K. Khanna, R. R. Prasad, and L. G. Pillai, "A review on application of fourier transform in image restoration," in Proc. 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) (Palladam, India, 2020), pp. 389-397.
  27. J. Su, B. Xu, and H. Yin, "A survey of deep learning approaches to image restoration," Neurocomputing 487, 46-65 (2022). https://doi.org/10.1016/j.neucom.2022.02.046
  28. H.-W. Chen, "Spatial resolution enhancement of oversmapled images using regression decomposition and systhesis," ISPRS Archives XLVI-4/W3-2021, 71-77 (2021).
  29. J. Benediksson, J. Chanussot, and W. Moon, "Advances in very-high-resolution remote sensing," Proc. IEEE. 101, 566-569 (2013). https://doi.org/10.1109/JPROC.2012.2237076
  30. J. Liang, G. Sun, K. Zhang, L. van Gool, and R. Timofte, "Mutual affine network for spatially variant kernel estimation in blind image super-resolution," in Proc. IEEE/CVF International Conference on Computer Vision (Montreal, Canada, Oct. 11-17, 2021), pp. 4096-4105.
  31. G. Hwang, C. Song, T. Lee, H. Na, and M. Kang, "Multi-aperture image processing using deep learning," J. Korean Soc. Indust. Appl. Math. 27, 56-74 (2023).
  32. L. Lin, Y. Liu, Y. Hu, X. Yan, K. Xie, and H. Huang, "Capturing, reconstructing, and simulating: The urbanscene3D dataset," (Visual Computing Research Center, Shenzhen University, Published date: 2022), https://vcc.tech/UrbanScene3D (Accessed Date: May. 1, 2022).
  33. L. Lin, U. Liu, Y. Hu, X. Yan, K. Xie, and H. Huang, "Capturing, reconstructing, and simulating: The urbanscene3D dataset," in Proc. Computer Vision - ECCV 2022: 17th European Conference (Tel Aviv, Israel, October 23-27, 2022), Part VIII, pp. 93-109.
  34. Earth Engine Data Catalog, "A planetary-scale platform for Earth science data & analysis," (Google), https://developers.google.com/earth-engine/datasets (Accessed Date: May. 1, 2022).
  35. A. Hore and D. Ziou, "Image quality metrics: PSNR vs. SSIM," in Proc. 20th International Conference on Pattern Recognition (Istanbul, Turkey, Aug. 23-26, 2010), pp. 2366-2369.
  36. Teledyne Imaging, "TDI imagers for space," (Teledyne Technologies Inc.), https://www.teledyneimaging.com/en/aerospace-and-defense/products/tdi-imagers-for-space/ (Accessed Date: Jul. 1, 2023).