• Title/Summary/Keyword: Markov decision processes

Search Result 19, Processing Time 0.022 seconds

Dynamic Supplier-Managed Inventory Control and the Beneficial Effect of Information Sharing (공급자 주도의 동적 재고 통제와 정보 공유의 수혜적 효과 분석에 대한 연구)

  • Kim Eun-gab;Park Chan-kwon;Shin Ki-tae
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.29 no.3
    • /
    • pp.63-78
    • /
    • 2004
  • This paper deals with a supplier-managed inventory(SMI) control for a two-echelon supply chain model with a service facility and a single supplier. The service facility is allocated to customers and provides a service using items of inventory that are purchased from the supplier, Assuming that the supplier knows the information of customer queue length as well as inventory position in the service facility at the time when it makes a replenishment decision, we identify an optimal replenishment policy which minimizes the total supply chain costs by reflecting these information into the replenishment decision. Numerical analysis demonstrates that the SMI strategy can be more cost-effective when the information of both customer queue length and inventory position is shared than when the information of inventory position only is shared.

Optimal LNG Procurement Policy in a Spot Market Using Dynamic Programming (동적 계획법을 이용한 LNG 현물시장에서의 포트폴리오 구성방법)

  • Ryu, Jong-Hyun
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.41 no.3
    • /
    • pp.259-266
    • /
    • 2015
  • Among many energy resources, natural gas has recently received a remarkable amount of attention, particularly from the electrical generation industry. This is in part due to increasing shale gas production, providing an environment-friendly fossil fuel, and high risk of nuclear power. Because South Korea, the world's second largest LNG importing nation after Japan, has no international natural gas pipelines and relies on imports in the form of LNG, the natural gas has been traditionally procured by long term LNG contracts at relatively high price. Thus, there is a need of developing an Asian LNG trading hub, where LNG can be traded at more competitive spot prices. In a natural gas spot market, the amount of natural gas to be bought should be carefully determined considering a limited storage capacity and future pricing dynamics. In this work, the problem to find the optimal amount of natural gas in a spot market is formulated as a Markov decision process (MDP) in risk neutral environment and the optimal base stock policy which depends on a stage and price is established. Taking into account price and demand uncertainties, the basestock target levels are simply approximated from dynamic programming. The simulation results show that the basestock policy can be one of effective ways for procurement of LNG in a spot market.

Topological measures for algorithm complexity of Markov decision processes (마르코프 결정 프로세스의 위상적 계산 복잡도 척도)

  • Yi, Seung-Joon;Zhang, Byoung-Tak
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2007.06c
    • /
    • pp.319-323
    • /
    • 2007
  • 실세계의 여러 문제들은 마르코프 결정 문제(Markov decision problem, MDP)로 표현될 수 있고, 이 MDP는 모델이 알려진 경우에는 평가치 반복(value iteration) 이나 모델이 알려지지 않은 경우에도 강화 학습(reinforcement learning) 알고리즘 등을 사용하여 풀 수 있다. 하지만 이들 알고리즘들은 시간 복잡도가 높아 크기가 큰 실세계 문제에 적용하기 쉽지 않아, MDP를 계층적으로 분할하거나, 여러 단계를 묶어서 수행하는 등의 시간적 추상화(temporal abstraction) 방법이 제안되어 왔다. 이러한 시간적 추상화 방법들의 문제점으로는 시간적 추상화의 디자인에 따라 MDP의 풀이 성능이 크게 달라질 수 있으며, 많은 경우 사용자가 이 디자인을 직접 제공해야 한다는 것들이 있다. 최근 사용자의 간섭이 필요 없이 자동적으로 시간적 추상화를 만드는 방법들이 제안된 바 있으나, 이들 방법들 역시 결과물에 대한 이론적인 성능 보장(performance guarantee)은 제공하지 못하고 있다. 본 연구에서는 이러한 문제점을 해결하기 위해 MDP의 구조와 그 풀이 성능을 연관짓는 복잡도 척도에 대해 살펴본다. 이를 위해 MDP로부터 얻은 상태 경로 그래프(state trajectory graph)의 위상적 성질들을 여러 네트워크 척도(network measurements) 들을 이용하여 측정하고, 이와 MDP의 풀이 성능과의 관계를 다양한 상황에 대해 실험적, 이론적으로 분석해 보았다.

  • PDF

Decision Feedback Detector for Space-Time Block Codes over Time-Varying Channels

  • Ahn, Kyung-Seung;Baik, Heung-Ki
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.28 no.5C
    • /
    • pp.506-513
    • /
    • 2003
  • Most existing space-time coding (STC) schemes have been developed for flat fading channels. To obtain antenna diversity gain, they rely on channel state information (CSI) required at the receiver through channel estimation techniques. This paper proposes a new decision feedback decoding scheme for Alamouti-based space-time block coding (STBC) transmission over time-selective fading channels. In wireless channels, time-selective fading effects arise mainly due to Doppler shift and carrier frequency offset, Modelling the time-selective fading channels as the first-order Gauss-Markov processes, we use recursive algorithms such as Kalman filtering, LMS and RLS algorithms for channel tracking. The proposed scheme consists of the symbol decoding stage and channel tracking algorithms. Computer simulations confirm that the proposed scheme shows the better performance and robustness to time-selectivity.

Approximate Dynamic Programming Based Interceptor Fire Control and Effectiveness Analysis for M-To-M Engagement (근사적 동적계획을 활용한 요격통제 및 동시교전 효과분석)

  • Lee, Changseok;Kim, Ju-Hyun;Choi, Bong Wan;Kim, Kyeongtaek
    • Journal of the Korean Society for Aeronautical & Space Sciences
    • /
    • v.50 no.4
    • /
    • pp.287-295
    • /
    • 2022
  • As low altitude long-range artillery threat has been strengthened, the development of anti-artillery interception system to protect assets against its attacks will be kicked off. We view the defense of long-range artillery attacks as a typical dynamic weapon target assignment (DWTA) problem. DWTA is a sequential decision process in which decision making under future uncertain attacks affects the subsequent decision processes and its results. These are typical characteristics of Markov decision process (MDP) model. We formulate the problem as a MDP model to examine the assignment policy for the defender. The proximity of the capital of South Korea to North Korea border limits the computation time for its solution to a few second. Within the allowed time interval, it is impossible to compute the exact optimal solution. We apply approximate dynamic programming (ADP) approach to check if ADP approach solve the MDP model within processing time limit. We employ Shoot-Shoot-Look policy as a baseline strategy and compare it with ADP approach for three scenarios. Simulation results show that ADP approach provide better solution than the baseline strategy.

Reinforcement Learning-based Dynamic Weapon Assignment to Multi-Caliber Long-Range Artillery Attacks (다종 장사정포 공격에 대한 강화학습 기반의 동적 무기할당)

  • Hyeonho Kim;Jung Hun Kim;Joohoe Kong;Ji Hoon Kyung
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.45 no.4
    • /
    • pp.42-52
    • /
    • 2022
  • North Korea continues to upgrade and display its long-range rocket launchers to emphasize its military strength. Recently Republic of Korea kicked off the development of anti-artillery interception system similar to Israel's "Iron Dome", designed to protect against North Korea's arsenal of long-range rockets. The system may not work smoothly without the function assigning interceptors to incoming various-caliber artillery rockets. We view the assignment task as a dynamic weapon target assignment (DWTA) problem. DWTA is a multistage decision process in which decision in a stage affects decision processes and its results in the subsequent stages. We represent the DWTA problem as a Markov decision process (MDP). Distance from Seoul to North Korea's multiple rocket launchers positioned near the border, limits the processing time of the model solver within only a few second. It is impossible to compute the exact optimal solution within the allowed time interval due to the curse of dimensionality inherently in MDP model of practical DWTA problem. We apply two reinforcement-based algorithms to get the approximate solution of the MDP model within the time limit. To check the quality of the approximate solution, we adopt Shoot-Shoot-Look(SSL) policy as a baseline. Simulation results showed that both algorithms provide better solution than the solution from the baseline strategy.

Integrated Inventory Allocation and Customer Order Admission Control in a Two-stage Supply Chain with Make-to-stock and Make-to-order Facilities (계획생산과 주문생산 시설들로 이루어진 두 단계 공급망에서 재고 할당과 고객주문 수용 통제의 통합적 관리)

  • Kim, Eun-Gab
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.35 no.1
    • /
    • pp.83-95
    • /
    • 2010
  • This paper considers a firm that operates make-to-stock and make-to-order facilities in successive stages. The make-to-stock facility produces components which are consumed by the external market demand as well as the internal make-to-order operation. The make-to-order facility processes customer orders with the option of acceptance or rejection. In this paper, we address the problem of coordinating how to allocate the capacity of the make-to-stock facility to internal and external demands and how to control incoming customer orders at the make-to-order facility so as to maximize the firm's profit subject to the system costs. To deal with this issue, we formulate the problem as a Markov decision process and characterize the structure of the optimal inventory allocation and customer order control. In a numerical experiment, we compare the performance of the optimal policy to the heuristic with static inventory allocation and admission control under different operating conditions of the system.

Exploring the Usage of the DEMATEL Method to Analyze the Causal Relations Between the Factors Facilitating Organizational Learning and Knowledge Creation in the Ministry of Education

  • Park, Sun Hyung;Kim, Il Soo;Lim, Seong Bum
    • International Journal of Contents
    • /
    • v.12 no.4
    • /
    • pp.31-44
    • /
    • 2016
  • Knowledge creation and management are regarded as critical success factors for an organization's survival in the knowledge era. As a process of knowledge acquisition and sharing, organizational learning mechanisms (OLMs) guide the learning function of organizations represented by its different learning activities. We examined a variety of learning processes that constitute OLMs. In this study, we aimed to capture the process and framework of OLMs and knowledge sharing and acquisition. Factors facilitating OLMs were investigated at three levels: individual, group, and organizational. The concept of an OLM has received some attention in the field of organizational learning, however, the relationship among the factors generating OLMs has not been empirically tested. As part of the ongoing discussion, we attempted a systemic approach for OLMs. OLMs can be represented by factors that are inherent to the organization's system; therefore, prior to empirically testing the OLM generating factor(s), evaluation of its organizational integration is required to determine effective treatment of each factor. Thus, we developed a framework to manage knowledge and proposed a method to numerically evaluate factors influencing the OLMs. Specifically, composite importance (CI) of the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method was applied to explore the interaction effect of these factors based on systemic approach. The augmented matrix thus generated is expected to serve as a stochastic matrix of an absorbing Markov chain.

Sound Model Generation using Most Frequent Model Search for Recognizing Animal Vocalization (최대 빈도모델 탐색을 이용한 동물소리 인식용 소리모델생성)

  • Ko, Youjung;Kim, Yoonjoong
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.10 no.1
    • /
    • pp.85-94
    • /
    • 2017
  • In this paper, I proposed a sound model generation and a most frequent model search algorithm for recognizing animal vocalization. The sound model generation algorithm generates a optimal set of models through repeating processes such as the training process, the Viterbi Search process, and the most frequent model search process while adjusting HMM(Hidden Markov Model) structure to improve global recognition rate. The most frequent model search algorithm searches the list of models produced by Viterbi Search Algorithm for the most frequent model and makes it be the final decision of recognition process. It is implemented using MFCC(Mel Frequency Cepstral Coefficient) for the sound feature, HMM for the model, and C# programming language. To evaluate the algorithm, a set of animal sounds for 27 species were prepared and the experiment showed that the sound model generation algorithm generates 27 HMM models with 97.29 percent of recognition rate.