Journal of Astronomy and Space Sciences

The Korean Space Science Society (한국우주과학회)
권호 표지
DOI QR코드

DOI QR Code

Design and Verification of Spacecraft Pose Estimation Algorithm using Deep Learning

  • Shinhye Moon (Department of Astronomy, Yonsei University) ;
  • Sang-Young Park (Department of Astronomy, Yonsei University) ;
  • Seunggwon Jeon (Department of Astronomy, Yonsei University) ;
  • Dae-Eun Kang (Department of Astronomy, Yonsei University)
  • Received : 2024.05.03
  • Accepted : 2024.05.22
  • Published : 2024.06.15
Download PDF
⟨ Previous Next ⟩

Abstract

This study developed a real-time spacecraft pose estimation algorithm that combined a deep learning model and the least-squares method. Pose estimation in space is crucial for automatic rendezvous docking and inter-spacecraft communication. Owing to the difficulty in training deep learning models in space, we showed that actual experimental results could be predicted through software simulations on the ground. We integrated deep learning with nonlinear least squares (NLS) to predict the pose from a single spacecraft image in real time. We constructed a virtual environment capable of mass-producing synthetic images to train a deep learning model. This study proposed a method for training a deep learning model using pure synthetic images. Further, a visual-based real-time estimation system suitable for use in a flight testbed was constructed. Consequently, it was verified that the hardware experimental results could be predicted from software simulations with the same environment and relative distance. This study showed that a deep learning model trained using only synthetic images can be sufficiently applied to real images. Thus, this study proposed a real-time pose estimation software for automatic docking and demonstrated that the method constructed with only synthetic data was applicable in space.

Keywords

Acknowledgement

This study was supported by the Challengeable Future Defense Technology Research and Development Program (Grant No. 912908601) of the Agency for Defense Development (2023).

References

  1. Chen B, Cao J, Parra A, Chin TJ, Satellite pose estimation with deep landmark regression and nonlinear pose refinement, in 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea, 27-28 Oct 2019.
  2. Choi AJ, Yang HH, Han JH, Study on robust aerial docking mechanism with deep learning based drogue detection and docking, Mech. Syst. Signal Process. 154, 107579 (2021). https://doi.org/10.1016/j.ymssp.2020.107579
  3. Eun Y, Park SY, Kim GN, Development of a hardware-in-the-loop testbed to demonstrate multiple spacecraft operations in proximity, Acta Astronaut. 147, 48-58 (2018). https://doi.org/10.1016/j.actaastro.2018.03.030
  4. Garcia A, Musallam MA, Gaudilliere V, Ghorbel E, Ismaeil KA, et al., LSPnet: a 2D localization-oriented spacecraft pose estimation neural network, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Nashville, TN, 19-25 Jun 2021.
  5. Hyun J, Eun Y, Park SY, Experimental study of spacecraft pose estimation algorithm using vision-based sensor, J. Astron. Space Sci. 35, 263-277 (2018). https://doi.org/10.5140/JASS.2018.35.4.263
  6. Kim GN, Park SY, Kang DE, Son J, Lee T, et al., Development of CubeSats for CANYVAL-C mission in formation flying, in APISAT 2019: asia pacific international symposium on aerospace technology, (Engineers Australia, Gold Coast, Australia, 2019) 813-824.
  7. Kingma DP, Ba J, Adam: a method for stochastic optimization, Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, 7-9 May 2015.
  8. Kisantal M, Sharma S, Park TH, Izzo D, Martens M, et al., Satellite pose estimation challenge: dataset, competition design, and results, IEEE Trans. Aerosp. Electron. Syst. 56, 4083-4098 (2020). https://doi.org/10.1109/TAES.2020.2989063
  9. Mayfield M, Industry offering on-orbit satellite servicing, Natl Def. 105, 25-26 (2021).
  10. Moon S, Design and verification of spacecraft pose estimation algorithm using deep learning, Master Thesis, Yonsei University (2022).
  11. Nazare TS, Paranhos da Costa GB, Contato WA, Ponti M, Deep convolutional neural networks and noisy images, in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 22nd Iberoamerican Congress, CIARP 2017, Valparaiso, Chile, 7-10 Nov 2017.
  12. Opromolla R, Fasano G, Rufino G, Grassi M, Uncooperative pose estimation with a LIDAR-based system, Acta Astronaut. 110, 287-297 (2015). https://doi.org/10.1016/j.actaastro.2014.11.003
  13. Phisannupawong T, Kamsing P, Torteeka P, Channumsin S, Sawangwit U, et al., Vision-based spacecraft pose estimation via a deep convolutional neural network for noncooperative docking operations, Aerospace 7, 126 (2020). https://doi.org/10.3390/aerospace7090126
  14. Proenca PF, Gao Y, Deep learning for spacecraft pose estimation from photorealistic rendering, in 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 31 May-31 Aug 2020.
  15. Sanders A, An Introduction to Unreal Engine 4 (1st ed.) (A K Peters, New York, 2016).
  16. Sharma S, D'Amico S, Pose estimation for non-cooperative rendezvous using neural networks, in AIAA/AAS Space Flight Mechanics Meeting, Maui, HI, 13-17 Jan 2019.
  17. Sun K, Xiao B, Liu D, Wang J, Deep high-resolution representation learning for human pose estimation, Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, 15-20 Jun 2019.
  18. Zhang Z, A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330-1334 (2000). https://doi.org/10.1109/34.888718