• 한국정보통신학회:학술대회논문집
• /
• 한국정보통신학회 2022년도 춘계학술대회
• /
• pp.47-49
• /
• 2022

Robot navigation has seen a major improvement since the the rediscovery of the potential of Artificial Intelligence (AI) and the attention it has garnered in research circles. A notable achievement in the area was Deep Learning (DL) application in computer vision with outstanding daily life applications such as face-recognition, object detection, and more. However, robotics in general still depend on human inputs in certain areas such as localization, navigation, etc. In this paper, we propose a study case of robot navigation based on deep reinforcement technology. We look into the benefits of switching from traditional ROS-based navigation algorithms towards machine learning approaches and methods. We describe the state-of-the-art technology by introducing the concepts of Reinforcement Learning (RL), Deep Learning (DL) and DRL before before focusing on visual navigation based on DRL. The case study preludes further real life deployment in which mobile navigational agent learns to navigate unbeknownst areas.

• Journal of Positioning, Navigation, and Timing
• /
• 제11권2호
• /
• pp.109-117
• /
• 2022

This paper proposes a method for detecting flight anomalies of drones through the difference between the command of flight controller (FC) and the navigation solution. If the drones make a flight normally, control errors generated by the difference between the desired control command of FC and the navigation solution should converge to zero. However, there is a risk of sudden change or divergence of control errors when the FC control feedback loop preset for the normal flight encounters interferences such as strong winds or navigation sensor abnormalities. In this paper, we propose the method with a deep neural network model that predicts the control error in the normal flight so that the abnormal flight state can be detected. The performance of proposed method was evaluated using the real-world flight data. The results showed that the method effectively detects anomalies in various situation.

• Journal of Positioning, Navigation, and Timing
• /
• 제10권2호
• /
• pp.113-120
• /
• 2021

The Distance Measuring Equipment (DME) is a ground-based aircraft navigation system and is considered as an infrastructure that ensures resilient aircraft navigation capability during the event of a Global Navigation Satellite System (GNSS) outage. The main problem of DME as a GNSS back up is a poor positioning accuracy that often reaches over 100 m. In this paper, a novel approach of applying deep reinforcement learning to a DME pulse design is introduced to improve the DME distance measuring accuracy. This method is designed to develop multipath-resistant DME pulses that comply with current DME specifications. In the research, a Markov Decision Process (MDP) for DME pulse design is set using pulse shape requirements and a timing error. Based on the designed MDP, we created an Environment called PulseEnv, which allows the agent representing a DME pulse shape to explore continuous space using the Soft Actor Critical (SAC) reinforcement learning algorithm.

• 로봇학회논문지
• /
• 제19권1호
• /
• pp.92-97
• /
• 2024

This paper provides a study of distributional perspective on reinforcement learning for application in mobile robot navigation. Mapless navigation algorithms based on deep reinforcement learning are proven to promising performance and high applicability. The trial-and-error simulations in virtual environments are encouraged to implement autonomous navigation due to expensive real-life interactions. Nevertheless, applying the deep reinforcement learning model in real tasks is challenging due to dissimilar data collection between virtual simulation and the physical world, leading to high-risk manners and high collision rate. In this paper, we present distributional reinforcement learning architecture for mapless navigation of mobile robot that adapt the uncertainty of environmental change. The experimental results indicate the superior performance of distributional soft actor critic compared to conventional methods.

• Journal of Positioning, Navigation, and Timing
• /
• 제10권1호
• /
• pp.49-54
• /
• 2021

Indoor positioning system becomes of increasing interests due to the demands for accurate indoor location information where Global Navigation Satellite System signal does not approach. Wi-Fi access points (APs) built in many construction in advance helps developing a Wi-Fi Received Signal Strength Indicator (RSSI) based indoor localization. This localization method first collects pairs of position and RSSI measurement set, which is called fingerprint database, and then estimates a user's position when given a query measurement set by comparing the fingerprint database. The challenge arises from nonlinearity and noise on Wi-Fi RSSI measurements and complexity of handling a large amount of the fingerprint data. In this paper, machine learning techniques have been applied to implement Wi-Fi based localization. However, most of existing indoor localizations focus on single position estimation. The main contribution of this paper is to develop multi-target localization by using deep neural, which is beneficial when a massive crowd requests positioning service. This paper evaluates the proposed multilocalization based on deep learning from a multi-story building, and analyses its learning effect as increasing number of target positions.

• 로봇학회논문지
• /
• 제19권1호
• /
• pp.117-129
• /
• 2024

This paper introduces collective navigation through a narrow gap using a curriculum-based deep reinforcement learning algorithm for a swarm of unmanned aerial vehicles (UAVs). Collective navigation in complex environments is essential for various applications such as search and rescue, environment monitoring and military tasks operations. Conventional methods, which are easily interpretable from an engineering perspective, divide the navigation tasks into mapping, planning, and control; however, they struggle with increased latency and unmodeled environmental factors. Recently, learning-based methods have addressed these problems by employing the end-to-end framework with neural networks. Nonetheless, most existing learning-based approaches face challenges in complex scenarios particularly for navigating through a narrow gap or when a leader or informed UAV is unavailable. Our approach uses the information of a certain number of nearest neighboring UAVs and incorporates a task-specific curriculum to reduce learning time and train a robust model. The effectiveness of the proposed algorithm is verified through an ablation study and quantitative metrics. Simulation results demonstrate that our approach outperforms existing methods.

• 대한임베디드공학회논문지
• /
• 제19권3호
• /
• pp.131-139
• /
• 2024

Crew Transfer Vessels (CTVs) are primarily used for the maintenance of offshore wind farms. Despite being manually operated by professional captains and crew, collisions with other ships and marine structures still occur. To prevent this, the introduction of autonomous navigation systems to CTVs is necessary. In this study, research on the obstacle avoidance system of the autonomous navigation system for CTVs was conducted. In particular, research on obstacle avoidance simulation for CTVs using deep reinforcement learning was carried out, taking into account the currents and wind loads in offshore wind farms. For this purpose, 3 degrees of freedom ship maneuvering modeling for CTVs considering the currents and wind loads in offshore wind farms was performed, and a simulation environment for offshore wind farms was implemented to train and test the deep reinforcement learning agent. Specifically, this study conducted research on obstacle avoidance maneuvers using MATD3 within deep reinforcement learning, and as a result, it was confirmed that the model, which underwent training over 10,000 episodes, could successfully avoid both static and moving obstacles. This confirms the conclusion that the application of the methods proposed in this study can successfully facilitate obstacle avoidance for autonomous navigation CTVs within offshore wind farms.

• 한국항해항만학회지
• /
• 제47권2호
• /
• pp.75-84
• /
• 2023

Ships need to ensure safety during their navigation, which makes route determination highly important. It must be accompanied by a route following controller that can accurately follow the route. This study proposes a method for automatically generating the ship route based on deep reinforcement learning algorithm and following it using a route following controller. To generate a ship route, under keel clearance was applied to secure the ship's safety and navigation chart information was used to apply ship navigation related regulations. For the experiment, a target ship with a draft of 8.23 m was designated. The target route in this study was to depart from Busan port and arrive at the pilot boarding place of the Ulsan port. As a route following controller, a velocity type fuzzy P ID controller that could compensate for the limitation of a linear controller was applied. As a result of using the deep Q network, a route with a total distance of 62.22 km and 81 waypoints was generated. To simplify the route, the Douglas-Peucker algorithm was introduced to reduce the total distance to 55.67 m and the number of way points to 3. After that, an experiment was conducted to follow the path generated by the target ship. Experiment results revealed that the velocity type fuzzy P ID controller had less overshoot and fast settling time. In addition, it had the advantage of reducing the energy loss of the ship because the change in rudder angle was smooth. This study can be used as a basic study of route automatic generation. It suggests a method of combining ship route generation with the route following control.

• 한국항해항만학회지
• /
• 제47권6호
• /
• pp.376-385
• /
• 2023

통계에 따르면 어선의 전복 사고는 전체 전복 사고의 절반 이상을 차지한다. 이는 미숙한 조업, 기상 악화, 정비 미흡 등 다양한 원인으로 발생할 수 있다. 업계 규모와 영향도, 기술 복잡성, 지역적 다양성 등으로 인해 어선은 상선에 비해 상대적으로 연구가 부족한 실정이다. 본 연구에서는 이미지 기반 딥러닝 모델을 활용하여 어선의 횡동요 시계열을 예측하고자 한다. 이미지 기반 딥러닝은 시계열의 다양한 패턴을 학습하여 높은 성능을 낼 수 있다. 이를 위해 Xception, ResNet50, CRNN의 3가지의 이미지 기반 딥러닝 모델을 활용하였다. Xception과 ResNet50은 각각 177, 184개의 층으로 구성되어 있으며 이에 반해 CRNN은 22개의 비교적 얇은 층으로 구성되어 있다. 실험 결과 Xception 딥러닝 모델이 가장 낮은 0.04291의 sMAPE와 0.0198의 RMSE를 기록하였다. ResNet50과 CRNN은 각각 0.0217, 0.022의 RMSE를 기록하였다. 이를 통해 상대적으로 층이 더 깊은 모델의 정확도가 높음을 확인할 수 있다.

• 한국항해항만학회:학술대회논문집
• /
• 한국항해항만학회 2022년도 추계학술대회
• /
• pp.170-171
• /
• 2022

A ship's maneuvering motion model is important in a computer simulation, especially under the trend of intelligent navigation. This model is usually constructed by the hydrodynamic parameters of the ship which are generated by the principles of hydrodynamics. Ship's motion model is a nonlinear function. By using this function, ships' motion elements can be calculated, then the ship's trajectory can be predicted. Deeping neural networks can construct any linear or non-linear equation theoretically if there have enough and sufficient training data. This study constructs some kinds of deep Networks and trains this network by real ship motion data, and chooses the best one of the networks, uses real data to train it, then uses it to predict the ship's trajectory, getting some conclusions and experiences.