한국산업정보학회논문지 (Journal of Korea Society of Industrial Information Systems)

한국산업정보학회 (Korea Society of Industrial Information Systems)
권호 표지
DOI QR코드

DOI QR Code

가상 환경과 실제 환경의 병행 강화학습을 통한 실내 자율주행

Indoor Autonomous Driving through Parallel Reinforcement Learning of Virtual and Real Environments

  • 정유석 (군산대학교 컴퓨터정보공학과) ;
  • 이창우 (군산대학교 컴퓨터정보공학과)
  • 투고 : 2021.08.02
  • 심사 : 2021.08.25
  • 발행 : 2021.08.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

강화 학습을 통한 실내 자율주행을 위해 가상 환경과 실제 환경에서 학습을 병행하는 방법을 제안한다. 실제 환경에서만 학습을 진행했을 경우 80시간 정도의 소요 시간이 필요하지만, 실제 환경과 가상 환경을 병행하며 학습을 진행했을 경우 50시간의 소요 시간이 필요하다. 가상 환경과 실제 환경에서 학습을 병행하면서 빠른 학습으로 다양한 실험을 거쳐 최적화된 파라미터를 얻을 수 있는 이점이 있다. 실내복도 이미지를 이용하여 가상 환경을 구성한 후 데스크톱으로 선행학습을 진행하였고 실제 환경에서의 학습은 Jetson Xavier를 기반으로 다양한 센서와 연결하여 학습을 진행하였다. 또한, 실내복도 환경의 반복되는 텍스처에 따른 정확도 문제를 해결하기 위해 복도 벽의 아랫선을 강조하는 특징점 검출을 학습하여 복도 벽 객체를 판단하고 정확도를 높일 수 있었다. 학습을 진행할수록 실험 차량은 실내복도 환경에서 복도 중앙을 기준으로 주행하며 평균 70회의 조향명령을 통해 움직인다.

We propose a method that combines learning in a virtual environment and a real environment for indoor autonomous driving through reinforcement learning. In case of learning only in the real environment, it takes about 80 hours, but in case of learning in both the real and virtual environments, it takes 40 hours. There is an advantage in that it is possible to obtain optimized parameters through various experiments through fast learning while learning in a virtual environment and a real environment in parallel. After configuring a virtual environment using indoor hallway images, prior learning was carried out on the desktop, and learning in the real environment was conducted by connecting various sensors based on Jetson Xavier. In addition, in order to solve the accuracy problem according to the repeated texture of the indoor corridor environment, it was possible to determine the corridor wall object and increase the accuracy by learning the feature point detection that emphasizes the lower line of the corridor wall. As the learning progresses, the experimental vehicle drives based on the center of the corridor in an indoor corridor environment and moves through an average of 70 steering commands.

키워드

과제정보

이 논문은 한국산업기술진흥원(KIAT)의 지원을 받아 수행된 연구임. (2021년 미래형자동차 R&D 전문인력 양성사업, 과제번호 : N0002428)

참고문헌

  1. Kim, J. Y., and Lee, S. J., et al. (2017). A Study on the National Policy Agenda based on Science & Tehnology.ICT for leading the 4th Industrial Revolution, Ministry of Science, ICT and Future Planning
  2. Commercializations Promotion Agency for R&D Outcomes. (2019). Artificial intelligence (big data) market and technology trends, S&T Market Report, Vol. 71
  3. Mnih, V., et al. (2013). Playing Atari with Deep Reinforcement Learning, NIPS Deep Learning Workshop 2013.
  4. Keskar, N. S., et all. (2017). on Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima, ICLR 2017.
  5. Krizhevsky, A., et all. (2012). Imagenet Classification with Deep Convolutional Neural Networks, NIPS 2012.
  6. Yang, J., Coughlin, J. F. (2014). In-vehicle technology for self-driving cars: Advantages and challenges for aging drivers, International Journal of Automotive Technology, vol.15, no.2, pp333-340. https://doi.org/10.1007/s12239-014-0034-6
  7. Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot, Nando de Freitas. (2016). Dueling Network Architectures for Deep Reinforcement Learning, Proceedings of the 33rd International 35 Conference on International Conference on Machine Learning (PMLR), Vol.48, pp1995-2003.
  8. Lillicrap, Timothy P., et al. (2019). Continuous control with deep reinforcement learning, arXiv preprint arXiv:1509.02971v6.
  9. Szegedy, Christian, et al. (2015). Going deeper with convolutions, IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  10. Simonyan, Karen, and Andrew Zisserman. (2015). Very deep convolutional networks for large-scale image recognition, arXivpreprint arXiv:1409.1556v6.