• Journal of the Korean Institute of Intelligent Systems
• /
• v.21 no.6
• /
• pp.737-742
• /
• 2011

Navigation problems include the task of determining the control input under various constraints for systems such as mobile robots subject to uncertain disturbance. Such tasks can be modeled as constrained stochastic control problems. In order to solve these control problems, one may try to utilize the dynamic programming(DP) methods which rely on the concept of optimal value function. However, in most real-world problems, this trial would give us many difficulties; for examples, the exact system model may not be known; the computation of the optimal control policy may be impossible; and/or a huge amount of computing resource may be in need. As a strategy to overcome the difficulties of DP, one can utilize ADP(approximate dynamic programming) methods, which find suboptimal control policies resorting to approximate value functions. In this paper, we apply recently proposed ADP methods to a class of navigation problems having complex constraints, and observe the resultant performance characteristics.

• Journal of the Korean Society for Aeronautical & Space Sciences
• /
• v.50 no.4
• /
• pp.287-295
• /
• 2022

As low altitude long-range artillery threat has been strengthened, the development of anti-artillery interception system to protect assets against its attacks will be kicked off. We view the defense of long-range artillery attacks as a typical dynamic weapon target assignment (DWTA) problem. DWTA is a sequential decision process in which decision making under future uncertain attacks affects the subsequent decision processes and its results. These are typical characteristics of Markov decision process (MDP) model. We formulate the problem as a MDP model to examine the assignment policy for the defender. The proximity of the capital of South Korea to North Korea border limits the computation time for its solution to a few second. Within the allowed time interval, it is impossible to compute the exact optimal solution. We apply approximate dynamic programming (ADP) approach to check if ADP approach solve the MDP model within processing time limit. We employ Shoot-Shoot-Look policy as a baseline strategy and compare it with ADP approach for three scenarios. Simulation results show that ADP approach provide better solution than the baseline strategy.

• Proceedings of the Korean Information Science Society Conference
• /
• 2011.06a
• /
• pp.286-289
• /
• 2011

제약을 갖는 부분 관찰 의사결정 과정(Constrained Partially Observable Markov Decision Process; CPOMDP)는 정책이 제약(constraint)를 만족하면서 가치 함수를 최적화하도록 일반적인 부분 관찰 의사결정과정(POMDP)을 확장한 모델이다. CPOMDP는 제한된 자원을 가지거나 여러 개의 목적 함수를 가지는 문제를 자연스럽게 모델링할 수 있기 때문에 일반적인 POMDP에 비해 더 실용적인 장점을 가진다. 본 논문에서는 CPOMDP의 확률적 최적 정책 및 근사 최적 정책을 계산할 수 있는 최적 및 근사 동적 프로그래밍 알고리즘을 제안한다. 최적 알고리즘은 동적 프로그래밍의 각 단계마다 미니맥스 이차 제약 계획 문제를 계산해야 하는 반면에 근사 알고리즘은 선형 계획 문제만을 필요로 하는 점-기반(point-based) 가치 업데이트를 이용한다. 실험 결과, 확률적 정책이 결정적(deterministic) 정책보다 더 나은 성능을 보이며, 근사 알고리즘을 통해 계산 시간을 줄일 수 있음을 보였다.

• Journal of the Korea Society of Computer and Information
• /
• v.14 no.4
• /
• pp.1-8
• /
• 2009

In this paper, we deal an implementation of the software that produces sub-optimal solution of cutting-ironbar planning problem using dynamic programming. Generally, it is required to design an optimization algorithm to accept the practical requirements of cutting ironbar manufacturing. But, this problem is a multiple-sized 1-dimensional cutting stock problem and Linear Programming approaches to get the optimal solution is difficult to be applied due to the problem of explosive computation and memory limitation. In order to overcome this problem, we reform the problem for applying Dynamic Programming and propose a cutting-ironbar planning algorithm searching the sub-optimal solution in the space of fixed amount of combinated columns by using heuristics. Then, we design a graphic user interfaces and screen displays to be operated conveniently in the industry workplace and implement the software using open-source GUI library toolkit, GTK+.

• Journal of Korea Spatial Information System Society
• /
• v.7 no.2 s.14
• /
• pp.57-66
• /
• 2005

The Interconnecting Highways problem is an abstract of many practical Layout Design problems in the areas of VLSI design, the optical and wired network design, and the planning for the road constructions. For the road constructions, the shortest-length road layouts that interconnect existing positions will provide many more economic benefits than others. That is, finding new road layouts to interconnect existing roads and cities over a wide area is an important issue. This paper addresses an approximation scheme that finds near optimal road layouts for the Interconnecting Highways problem which is NP-hard. As long as computational resources are provided, the near optimality can be acquired asymptotically. This implies that the result of the scheme can be regarded as the optimal solution for the problem in practice. While other approximation schemes can be made for the problem, this proposed scheme provides a big merit that the algorithm designed by this scheme fits well to given problem instances.

• Journal of radiological science and technology
• /
• v.33 no.2
• /
• pp.133-141
• /
• 2010

The evaluation of Varian enhanced dynamic wedges (EDW) were performed in terms of quality assurance in external radiotherapy. The seven (10, 15, 20, 25, 30, 45, 60 deg.) EDW angles were evaluated for 6 and 15 MV x-rays in Varian Linac. The STT (segmented treatment table) for a field were calculated and compared with actual movement of the jaw using Dynalog files in order to evaluate mechanical operation. Two dimensional array detector and an ionization chamber were used to measure dose distributions in phantom from Linac. The mechanical movement of jaw was agreed with its expectation and two dimensional dose distributions including beam profiles were in agreement with RTP data approximately. In comparison with RTP calculations the percentage difference of output dose values for 100 MU irradiation was less than 2.9% and measured wedge factor was less than 2.6%. These results are shown that there is no problem in clinical applications of EDW equipped on this linac.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.25 no.4
• /
• pp.319-326
• /
• 2015

Recently in the fields o f stochastic optimal control ( SOC) and reinforcemnet l earning (RL), there have been a great deal of research efforts for the problem of finding data-based sub-optimal control policies. The conventional theory for finding optimal controllers via the value-function-based dynamic programming was established for solving the stochastic optimal control problems with solid theoretical background. However, they can be successfully applied only to extremely simple cases. Hence, the data-based modern approach, which tries to find sub-optimal solutions utilizing relevant data such as the state-transition and reward signals instead of rigorous mathematical analyses, is particularly attractive to practical applications. In this paper, we consider a couple of methods combining the modern SOC strategies and approximate inference together with machine-learning-based data treatment methods. Also, we apply the resultant methods to a variety of application domains including financial engineering, and observe their performance.