Journal of Aerospace System Engineering (항공우주시스템공학회지)

The Society for Aerospace System Engineering (항공우주시스템공학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on AR-based Interface Technique for efficient UAV Operation using a See-through HMD

투시형 HMD를 이용한 효율적인 UAV 운용을 위한 증강현실 기반의 인터페이스 기법에 대한 연구

  • Wan Joo Cho (Div. of Aerospace Engineering, Gyeongsang National University) ;
  • Hyun Joon Chang (School of Mechanical and Aerospace Engineering, Gyeongsang National University) ;
  • Yong Ho Moon (School of Mechanical and Aerospace Engineering, Gyeongsang National University)
  • 조완주 (경상국립대학교 항공우주특성화대학원 항공우주공학과) ;
  • 장현준 (경상국립대학교 일반대학원 기계항공우주공학부) ;
  • 문용호 (경상국립대학교 일반대학원 기계항공우주공학부)
  • Received : 2023.08.18
  • Accepted : 2023.11.29
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In order to effectively prevent and respond to disasters, several techniques have been developed in which the pilot wearing a see-through Head Mounted Display (HMD) performs disaster-related rescue activities using images transmitted from an Unmanned Aerial Vehicle (UAV). However, these techniques have limitations in quickly determining and executing tasks appropriate to the on-site situation because the pilot cannot recognize the entire field in an integrated manner. In order to overcome these problems, we propose an AR based-interface technique that allows the rescuer wearing a see-through HMD to operate a UAV efficiently. Simulation results show that the proposed interface technique allows the rescuer wearing a see-through HMD to control the gimbal and flight of the UAV at a high speed based on finger gestures in a visibility situation.

재난 및 재해를 효과적으로 예방하고 대응하기 위하여 투시형 HMD를 착용한 조종사가 무인항공기에서 전송되는 영상을 이용하여 재난 및 재해 관련 구조 작업을 수행하는 기법들이 개발되어 왔다. 그러나 투시형 HMD를 착용한 조종사는 현장 전체를 통합적으로 인지할 수 없기 때문에 현장 상황에 적합한 작업들을 신속하게 결정하고 실행하기에는 한계가 존재한다. 이러한 문제점을 극복하기 위해 본 논문에서는 투시형 HMD를 착용한 구조요원이 무인항공기를 효율적으로 운용할 수 있는 증강현실 기반의 인터페이스 기법을 제안한다. 모의실험 결과는 시야가 확보되는 상황에서 투시형 HMD를 착용한 구조요원이 손가락 제스처를 기반으로 짐벌과 무인항공기의 비행을 고속으로 제어할 수 있음을 보여준다.

Keywords

Acknowledgement

본 논문은 정부(교육부)의 재원으로 한국연구재단의 지원을 받아 수행된 기초연구사업의 연구 성과입니다.(No. NRF-2018R1D1A1B07043365)

References

  1. F. Ahmed, J. C. Mohanta, A. Keshari, and P. S. Yadav, "Recent advances in unmanned aerial vehicles: A review," Arabian Journal for Science and Engineering, vol. 47, pp. 7963-7984, Apr. 2022 https://doi.org/10.1007/s13369-022-06738-0
  2. S. G. Lee, D. S. Har, and D. S. Kum, "Drone-assisted disaster management: finding victims via infrared camera and lidar sensor fusion," Proc. of 3rd Asia-Pacific World Congress Computer Science and Engineering, Dec. 2016
  3. D. Camara., "Cavalry to the Rescue : Drones fleet to help rescuers operations over disasters scenarios," Proc. of IEEE Conference on Antenna Measurements & Application, Nov. 2014
  4. M. Prexl, K. Struebig, J. Harder, and A. Hoehn, "User studies of a head-mounted display for search and rescue teleoperation of UAVs via satellite link," Proc. of IEEE Aerospace Conference, Mar. 2017.
  5. J. M. Teixeira, R. Ferreira, M. Santos, and V. Teichrie, "Teleoperation using google glass and AR, drone for structural inspection," Proc of XVI Symposium on Virtual and Augmented Reality, May 2014.
  6. R. Llasag, D. Marcillo, C. Grilo, and C. Silva, "Human detection for search and rescue applications with UAVs and mixed reality interfaces," Proc. of 14th Iberian Conference on Information Systems and Technologies (CISTI), Jun. 2019.
  7. A. Akter, A. Islam, D. E. Widiyanti, and S. Y. Shin, "MR-Drone: Mixed reality-based drone-assisted search and rescue system," Proc. of Symposium of the Korean Institute of communications and Information Sciences, Feb. 2020.
  8. M. G. Jeong, H. S. Lee, M. N. Bae, D. B. Shin, S. H. Lim, and K. B. Lee, "Development and application of the smart helmet for disaster and safety," Proc. of International Conference on Information and Communication Technology Convergence (ICTC), Oct. 2018.
  9. R. T. Azuma, "A survey of augmented reality," Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355-385, Aug. 1997. https://doi.org/10.1162/pres.1997.6.4.355
  10. J. H. Hyun, M. H. Kang, and Y. H. Moon, "Mixed reality image generation method for HMD-based flight simulator," Journal of Aerospace System Engineering, vol. 17, no.1, pp. 59-67, Apr. 2023.
  11. Hanseul Jun and Gunhee Kim, "A calibration method for optical see-through head-mounted displays with a depth camera," Proc. of IEEE Conference on Virtual Reality, Mar. 2016.
  12. K. S. Ham, D. H. Lee, H. J. Hong, S. J. Park, and J. W. Kim, "An experimental research on the usability of indirect control using finger gesture interaction in three dimensional space," Journal of Contents, vol. 14, no. 11, pp. 519-532, Nov. 2014 https://doi.org/10.5392/JKCA.2014.14.11.519
  13. MS Hololens (https://support.microsoft.com/en-us/help/12644/hololens-use-gestures)