Multi-modal Sensor System and Database for Human Detection and Activity Learning of Robot in Outdoor

실외에서 로봇의 인간 탐지 및 행위 학습을 위한 멀티모달센서 시스템 및 데이터베이스 구축

Abstract

Robots which detect human and recognize action are important factors for human interaction, and many researches have been conducted. Recently, deep learning technology has developed and learning based robot's technology is a major research area. These studies require a database to learn and evaluate for intelligent human perception. In this paper, we propose a multi-modal sensor-based image database condition considering the security task by analyzing the image database to detect the person in the outdoor environment and to recognize the behavior during the running of the robot.

Fig. 1. Multi-modal sensor system configuration.

Fig. 2. Multi-modal sensor module interface.

Fig. 3. Multi-modal sensor module Head design.

Fig. 4. Wire based vibration damper.

Fig. 5. Wire based vibration damper.

Fig. 6. The muliti-modal sensor module mounted on robot platform.

Fig. 7. An example of mulit-modal sensor data.

Fig. 8. Result of vibration damping. (a) Original, (b) Proposed method, (c) Experimental Platform

Fig. 9. Take 1: Day and Night time database in the lawn playground.

Fig. 10. Take 2: Lying database in the driveway.

Fig. 11. Take 3: Heavy rain and thunder in the driveway.

Fig. 12. Take 4: Daytime in the park.

Table 1. The behavior definition in normal and abnormal situations

Table 2. The database configuration