• 전자통신동향분석
• /
• 제35권6호
• /
• pp.137-149
• /
• 2020

Several multi-agent reinforcement learning (MARL) algorithms have achieved overwhelming results in recent years. They have demonstrated their potential in solving complex problems in the field of real-time strategy online games, robotics, and autonomous vehicles. However these algorithms face many challenges when dealing with massive problem spaces in sparse reward environments. Based on the centralized training and decentralized execution (CTDE) architecture, the MARL algorithms discussed in the literature aim to solve the current challenges by formulating novel concepts of inter-agent modeling, credit assignment, multiagent communication, and the exploration-exploitation dilemma. The fundamental objective of this paper is to deliver a comprehensive survey of existing MARL algorithms based on the problem statements rather than on the technologies. We also discuss several experimental frameworks to provide insight into the use of these algorithms and to motivate some promising directions for future research.

• 제어로봇시스템학회논문지
• /
• 제8권9호
• /
• pp.795-802
• /
• 2002

We present a new dynamic routing scheme for multiple autonomous guided vehicles (AGVs) systems. There have been so many results concerned with scheduling and routing of multiple AGV systems; however, most of them are only applicable to systems with a small number of AGVs under a low degree of concurrency. With an increased number of AGVs in recent applications, these AGV systems are faced with another problem that has never been occurred in a system with a small number AGVs. This is the stop propagation problem. That is, if a leading AGV stops then all the following AGVs must stop to avoid any collision. In order to resolve this problem, we propose a nonstop preferential detour (NPD) algorithm which is a new dynamic routing scheme employing an election algorithm. For real time computation, we introduce two stage control scheme and propose a new path searching scheme, k-via shortest path scheme for an efficient dynamic routing algorithm. Finally, the proposed new dynamic routing scheme is illustrated by an example.

• Ocean Systems Engineering
• /
• 제9권4호
• /
• pp.391-407
• /
• 2019

Tracking the location (position) of a surface or underwater marine vehicle is important as part of guidance and navigation. While the Global Positioning System (GPS) works well in an open sea environment but its use is limited whenever testing scaled-down models of such vehicles in the laboratory environment. This paper presents the design, development and implementation of a low energy ultrasonic augmented single beacon-based localization technique suitable for such requirements. The strategy consists of applying Extended Kalman Filter (EKF) to achieve location tracking from basic dynamic distance measurements of the moving model from a fixed beacon, while on-board motion sensor measures heading angle and velocity. Iterative application of the Extended Kalman Filter yields x and y co-ordinate positions of the moving model. Tests performed on a free-running ship model in a wave basin facility of dimension 30 m by 30 m by 3 m water depth validate the proposed model. The test results show quick convergence with an error of few centimeters in the estimated position of the ship model. The proposed technique has application in the real field scenario by replacing the ultrasonic sensor with industrial grade long range acoustic modem. As compared with the existing systems such as LBL, SBL, USBL and others localization techniques, the proposed technique can save deployment cost and also cut the cost on number of acoustic modems involved.

• 한국군사과학기술학회지
• /
• 제17권6호
• /
• pp.810-820
• /
• 2014

Recently many studies of Radar systems mounted on ground vehicles for autonomous driving, SLAM (Simultaneous localization and mapping) and collision avoidance are reported. In near field, several hits per an object are generated after signal processing of Radar data. Hence, clustering is an essential technique to estimate their shapes and positions precisely. This paper proposes a method of grouping hits in range-doppler domains into clusters which represent each object, according to the pre-defined rules. The rules are based on the perceptual cues to separate hits by object. The morphological connectedness between hits and the characteristics of SNR distribution of hits are adopted as the perceptual cues for clustering. In various simulations for the performance assessment, the proposed method yielded more effective performance than other techniques.

• 한국해양공학회지
• /
• 제25권1호
• /
• pp.27-31
• /
• 2011

The previously developed method of estimating the propulsion performance of a UUV was applied to the high speed UUV, which is propelled by commercial thrusters. The thrusters were selected with an overdesign mentality; in other words, their capacities were excessive. At that point, the designer's concern was focused on a question regarding at what rpm the UUV could reach the design speed. Because the developed method required thrust coefficient curve data, the researchers asked for the POW data of the thrusters from the manufacturer. From the data, the researchers extracted the thrust coefficient and estimated the rpm value of design speed for the UUV. Finally, the researchers compared the estimated value and the result from a self-propulsion test using a VPMM (Vertical planar motion mechanism) test at a towing tank in MOERI.

• Advances in aircraft and spacecraft science
• /
• 제7권1호
• /
• pp.53-78
• /
• 2020

Automatic trajectory planning is an important task that will have to be performed by truly autonomous vehicles. The main method proposed, for unmanned airplanes to do this, consists in concatenating elementary segments of trajectories such as rectilinear, circular and helical segments. It is argued here that because these cannot be expected to all be flyable at a same constant speed, it is necessary to consider segments on which the airplane accelerates or decelerates. In order to preserve the planning advantages that result from having the speed constant, it is proposed to do all speed changes at maximum deceleration or acceleration, so that they are as brief as possible. The constraints on the load factor, the lift and the power required for the motion are derived. The equation of motion for such accelerated motions is solved numerically. New results are obtained concerning the value of the angle and the speed for which the longest distance and the longest duration glides happen, and then for which the steepest, the fastest and the most fuel economical climbs happen. The values obtained differ from those found in most airplane dynamics textbooks. Example of tables are produced that show how general speed changes can be effected efficiently; showing the time required for the changes, the horizontal distance traveled and the amount of fuel required. The results obtained apply to all internal combustion engine-propeller driven airplanes.

• 한국시뮬레이션학회논문지
• /
• 제27권2호
• /
• pp.101-114
• /
• 2018

무인 비행체의 실시간 경로계획 생성 시 최적의 경로를 찾기 위한 다양한 연구가 진행되어 왔다. 본 논문에서는 진화알고리즘을 통한 무인비행체의 경로계획 생성을 수행하고, 이를 에이전트 기반 시뮬레이션 환경에서 구현 및 테스트가 가능함을 검증하였다. 이를 위해, Repast toolkit에 JGAP 패키지를 탑재하여 Java 기반의 유전 알고리즘 프로그래밍을 통한 무인 비행체의 경로 계획을 생성하였고, 해당 결과를 에이전트 기반으로 시뮬레이션을 수행하였다. 본 논문에서는 에이전트 기반 시뮬레이션 소프트웨어를 소프트웨어 공학 개발 생명주기에 맞춰 문서화하여 설계 및 구현되었으며, 에이전트 모델링 설계는 자동화, 적응성 및 에이전트 간의 상호 작용에 초점을 맞추었다. 또한, 시뮬레이션을 통해 에이전트 기반 환경에서 설계한 모델 및 시나리오를 검증하여 다수의 비행 에이전트에 내재된 동적 경로계획 알고리즘이 실시간으로 자율적인 경로 생성이 가능함을 증명하였다.

• 제어로봇시스템학회:학술대회논문집
• /
• 제어로봇시스템학회 2005년도 ICCAS
• /
• pp.463-467
• /
• 2005

In this paper, the guidance law applicable to formation flight of UAV in three-dimensional space is proposed. The concept of miss distance, which is commonly used in the missile guidance laws, and Lyapunov stability theorem are effectively combined to obtain the guidance commands of the wingmen. The propose guidance law is easily integrated into the existing flight control system because the guidance commands are given in terms of velocity, flight path angle and heading angle to form the prescribed formation. In this guidance law, communication is required between the leader and the wingmen to achieve autonomous formation. The wingmen are only required the current position and velocity information of the leader vehicle. The performance of the proposed guidance law is evaluated using the complete nonlinear 6-DOF aircraft system. This system is integrated with nonlinear aerodynamic and engine characteristics, actuator servo limitations for control surfaces, various stability and control augmentation system, and autopilots. From the nonlinear simulation results, the new guidance law for formation flight shows that the vehicles involved in formation flight are perfectly formed the prescribed formation satisfying the several constraints such as final velocity, flight path angle, and heading angle.

• 제어로봇시스템학회논문지
• /
• 제20권11호
• /
• pp.1098-1102
• /
• 2014

Localization of aerial vehicles and map building of flight environments are key technologies for the autonomous flight of small UAVs. In outdoor environments, an unmanned aircraft can easily use a GPS (Global Positioning System) for its localization with acceptable accuracy. However, as the GPS is not available for use in indoor environments, the development of a SLAM (Simultaneous Localization and Mapping) system that is suitable for small UAVs is therefore needed. In this paper, we suggest a vision-based SLAM system that uses vision sensors and an AHRS (Attitude Heading Reference System) sensor. Feature points in images captured from the vision sensor are obtained by using GPU (Graphics Process Unit) based SIFT (Scale-invariant Feature Transform) algorithm. Those feature points are then combined with attitude information obtained from the AHRS to estimate the position of the small UAV. Based on the location information and color distribution, a Gaussian process model is generated, which could be a map. The experimental results show that the position of a small unmanned aircraft is estimated properly and the map of the environment is constructed by using the proposed method. Finally, the reliability of the proposed method is verified by comparing the difference between the estimated values and the actual values.

• Nuclear Engineering and Technology
• /
• 제52권7호
• /
• pp.1436-1442
• /
• 2020

Nuclear infrastructure systems play an important role in national security. The functions and missions of nuclear infrastructure systems are vital to government, businesses, society and citizen's lives. It is crucial to design nuclear infrastructure for scalability, reliability and robustness. To do this, we can use machine learning, which is a state of the art technology used in various fields ranging from voice recognition, Internet of Things (IoT) device management and autonomous vehicles. In this paper, we propose to design and develop a machine learning algorithm to perform predictive maintenance of nuclear infrastructure. Support vector machine and logistic regression algorithms will be used to perform the prediction. These machine learning techniques have been used to explore and compare rare events that could occur in nuclear infrastructure. As per our literature review, support vector machines provide better performance metrics. In this paper, we have performed parameter optimization for both algorithms mentioned. Existing research has been done in conditions with a great volume of data, but this paper presents a novel approach to correlate nuclear infrastructure data samples where the density of probability is very low. This paper also identifies the respective motivations and distinguishes between benefits and drawbacks of the selected machine learning algorithms.