• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.05a
• /
• pp.47-49
• /
• 2022

Robot navigation has seen a major improvement since the the rediscovery of the potential of Artificial Intelligence (AI) and the attention it has garnered in research circles. A notable achievement in the area was Deep Learning (DL) application in computer vision with outstanding daily life applications such as face-recognition, object detection, and more. However, robotics in general still depend on human inputs in certain areas such as localization, navigation, etc. In this paper, we propose a study case of robot navigation based on deep reinforcement technology. We look into the benefits of switching from traditional ROS-based navigation algorithms towards machine learning approaches and methods. We describe the state-of-the-art technology by introducing the concepts of Reinforcement Learning (RL), Deep Learning (DL) and DRL before before focusing on visual navigation based on DRL. The case study preludes further real life deployment in which mobile navigational agent learns to navigate unbeknownst areas.

• Journal of Positioning, Navigation, and Timing
• /
• v.11 no.2
• /
• pp.109-117
• /
• 2022

This paper proposes a method for detecting flight anomalies of drones through the difference between the command of flight controller (FC) and the navigation solution. If the drones make a flight normally, control errors generated by the difference between the desired control command of FC and the navigation solution should converge to zero. However, there is a risk of sudden change or divergence of control errors when the FC control feedback loop preset for the normal flight encounters interferences such as strong winds or navigation sensor abnormalities. In this paper, we propose the method with a deep neural network model that predicts the control error in the normal flight so that the abnormal flight state can be detected. The performance of proposed method was evaluated using the real-world flight data. The results showed that the method effectively detects anomalies in various situation.

• Journal of Positioning, Navigation, and Timing
• /
• v.10 no.2
• /
• pp.113-120
• /
• 2021

The Distance Measuring Equipment (DME) is a ground-based aircraft navigation system and is considered as an infrastructure that ensures resilient aircraft navigation capability during the event of a Global Navigation Satellite System (GNSS) outage. The main problem of DME as a GNSS back up is a poor positioning accuracy that often reaches over 100 m. In this paper, a novel approach of applying deep reinforcement learning to a DME pulse design is introduced to improve the DME distance measuring accuracy. This method is designed to develop multipath-resistant DME pulses that comply with current DME specifications. In the research, a Markov Decision Process (MDP) for DME pulse design is set using pulse shape requirements and a timing error. Based on the designed MDP, we created an Environment called PulseEnv, which allows the agent representing a DME pulse shape to explore continuous space using the Soft Actor Critical (SAC) reinforcement learning algorithm.

• The Journal of Korea Robotics Society
• /
• v.19 no.1
• /
• pp.92-97
• /
• 2024

This paper provides a study of distributional perspective on reinforcement learning for application in mobile robot navigation. Mapless navigation algorithms based on deep reinforcement learning are proven to promising performance and high applicability. The trial-and-error simulations in virtual environments are encouraged to implement autonomous navigation due to expensive real-life interactions. Nevertheless, applying the deep reinforcement learning model in real tasks is challenging due to dissimilar data collection between virtual simulation and the physical world, leading to high-risk manners and high collision rate. In this paper, we present distributional reinforcement learning architecture for mapless navigation of mobile robot that adapt the uncertainty of environmental change. The experimental results indicate the superior performance of distributional soft actor critic compared to conventional methods.

• Journal of Positioning, Navigation, and Timing
• /
• v.10 no.1
• /
• pp.49-54
• /
• 2021

Indoor positioning system becomes of increasing interests due to the demands for accurate indoor location information where Global Navigation Satellite System signal does not approach. Wi-Fi access points (APs) built in many construction in advance helps developing a Wi-Fi Received Signal Strength Indicator (RSSI) based indoor localization. This localization method first collects pairs of position and RSSI measurement set, which is called fingerprint database, and then estimates a user's position when given a query measurement set by comparing the fingerprint database. The challenge arises from nonlinearity and noise on Wi-Fi RSSI measurements and complexity of handling a large amount of the fingerprint data. In this paper, machine learning techniques have been applied to implement Wi-Fi based localization. However, most of existing indoor localizations focus on single position estimation. The main contribution of this paper is to develop multi-target localization by using deep neural, which is beneficial when a massive crowd requests positioning service. This paper evaluates the proposed multilocalization based on deep learning from a multi-story building, and analyses its learning effect as increasing number of target positions.

• The Journal of Korea Robotics Society
• /
• v.19 no.1
• /
• pp.117-129
• /
• 2024

This paper introduces collective navigation through a narrow gap using a curriculum-based deep reinforcement learning algorithm for a swarm of unmanned aerial vehicles (UAVs). Collective navigation in complex environments is essential for various applications such as search and rescue, environment monitoring and military tasks operations. Conventional methods, which are easily interpretable from an engineering perspective, divide the navigation tasks into mapping, planning, and control; however, they struggle with increased latency and unmodeled environmental factors. Recently, learning-based methods have addressed these problems by employing the end-to-end framework with neural networks. Nonetheless, most existing learning-based approaches face challenges in complex scenarios particularly for navigating through a narrow gap or when a leader or informed UAV is unavailable. Our approach uses the information of a certain number of nearest neighboring UAVs and incorporates a task-specific curriculum to reduce learning time and train a robust model. The effectiveness of the proposed algorithm is verified through an ablation study and quantitative metrics. Simulation results demonstrate that our approach outperforms existing methods.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.19 no.3
• /
• pp.131-139
• /
• 2024

Crew Transfer Vessels (CTVs) are primarily used for the maintenance of offshore wind farms. Despite being manually operated by professional captains and crew, collisions with other ships and marine structures still occur. To prevent this, the introduction of autonomous navigation systems to CTVs is necessary. In this study, research on the obstacle avoidance system of the autonomous navigation system for CTVs was conducted. In particular, research on obstacle avoidance simulation for CTVs using deep reinforcement learning was carried out, taking into account the currents and wind loads in offshore wind farms. For this purpose, 3 degrees of freedom ship maneuvering modeling for CTVs considering the currents and wind loads in offshore wind farms was performed, and a simulation environment for offshore wind farms was implemented to train and test the deep reinforcement learning agent. Specifically, this study conducted research on obstacle avoidance maneuvers using MATD3 within deep reinforcement learning, and as a result, it was confirmed that the model, which underwent training over 10,000 episodes, could successfully avoid both static and moving obstacles. This confirms the conclusion that the application of the methods proposed in this study can successfully facilitate obstacle avoidance for autonomous navigation CTVs within offshore wind farms.

• Journal of Navigation and Port Research
• /
• v.47 no.2
• /
• pp.75-84
• /
• 2023

Ships need to ensure safety during their navigation, which makes route determination highly important. It must be accompanied by a route following controller that can accurately follow the route. This study proposes a method for automatically generating the ship route based on deep reinforcement learning algorithm and following it using a route following controller. To generate a ship route, under keel clearance was applied to secure the ship's safety and navigation chart information was used to apply ship navigation related regulations. For the experiment, a target ship with a draft of 8.23 m was designated. The target route in this study was to depart from Busan port and arrive at the pilot boarding place of the Ulsan port. As a route following controller, a velocity type fuzzy P ID controller that could compensate for the limitation of a linear controller was applied. As a result of using the deep Q network, a route with a total distance of 62.22 km and 81 waypoints was generated. To simplify the route, the Douglas-Peucker algorithm was introduced to reduce the total distance to 55.67 m and the number of way points to 3. After that, an experiment was conducted to follow the path generated by the target ship. Experiment results revealed that the velocity type fuzzy P ID controller had less overshoot and fast settling time. In addition, it had the advantage of reducing the energy loss of the ship because the change in rudder angle was smooth. This study can be used as a basic study of route automatic generation. It suggests a method of combining ship route generation with the route following control.

• Journal of Navigation and Port Research
• /
• v.47 no.6
• /
• pp.376-385
• /
• 2023

Fishing boat capsizing accidents account for more than half of all capsize accidents. These can occur for a variety of reasons, including inexperienced operation, bad weather, and poor maintenance. Due to the size and influence of the industry, technological complexity, and regional diversity, fishing ships are relatively under-researched compared to commercial ships. This study aimed to predict the rolling motion time series of fishing boats using an image-based deep learning model. Image-based deep learning can achieve high performance by learning various patterns in a time series. Three image-based deep learning models were used for this purpose: Xception, ResNet50, and CRNN. Xception and ResNet50 are composed of 177 and 184 layers, respectively, while CRNN is composed of 22 relatively thin layers. The experimental results showed that the Xception deep learning model recorded the lowest Symmetric mean absolute percentage error(sMAPE) of 0.04291 and Root Mean Squared Error(RMSE) of 0.0198. ResNet50 and CRNN recorded an RMSE of 0.0217 and 0.022, respectively. This confirms that the models with relatively deeper layers had higher accuracy.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2022.11a
• /
• pp.170-171
• /
• 2022

A ship's maneuvering motion model is important in a computer simulation, especially under the trend of intelligent navigation. This model is usually constructed by the hydrodynamic parameters of the ship which are generated by the principles of hydrodynamics. Ship's motion model is a nonlinear function. By using this function, ships' motion elements can be calculated, then the ship's trajectory can be predicted. Deeping neural networks can construct any linear or non-linear equation theoretically if there have enough and sufficient training data. This study constructs some kinds of deep Networks and trains this network by real ship motion data, and chooses the best one of the networks, uses real data to train it, then uses it to predict the ship's trajectory, getting some conclusions and experiences.