• Title/Summary/Keyword: Policy Iteration

Search Result 18, Processing Time 0.022 seconds

Rental Resource Management Model with Capacity Expansion and Return (용량 확장과 반납을 갖는 렌탈 자원 관리모델)

  • Kim Eun-Gab;Byun Jin-Ho
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.31 no.3
    • /
    • pp.81-96
    • /
    • 2006
  • We consider a rental company that dynamically manages Its capacity level through capacity addition and return While serving customer with its own capacity, the company expands its capacity by renting items from an outside source so that it can avoid lost opportunities of rental which occur when stock is not sufficient. If stock becomes sufficiently large enough to cope with demands, the company returns expanded capacity to the outside source. Formulating the model into a Markov decision problem, we identify an optimal capacity management Policy which states when the company should expand its capacity and when it should return expanded capacity after capacity addition. Since it is intractable to analytically find the optimal capacity management policy and the optimal size of capacity expansion, we present a numerical procedure that finds these optimal values based on the value iteration method. Numerical analysis is implemented and we observe monotonic properties of the optimal performance measures by system parameters, which are meaningful in developing effective heuristic policies.

On The Mathematical Structure of Markov Process and Markovian Sequential Decision Process (Markov 과정(過程)의 수리적(數理的) 구조(構造)와 그 축차결정과정(逐次決定過程))

  • Kim, Yu-Song
    • Journal of Korean Society for Quality Management
    • /
    • v.11 no.2
    • /
    • pp.2-9
    • /
    • 1983
  • As will be seen, this paper is tries that the research on the mathematical structure of Markov process and Markovian sequential decision process (the policy improvement iteration method,) moreover, that it analyze the logic and the characteristic of behavior of mathematical model of Markov process. Therefore firstly, it classify, on research of mathematical structure of Markov process, the forward equation and backward equation of Chapman-kolmogorov equation and of kolmogorov differential equation, and then have survey on logic of equation systems or on the question of uniqueness and existence of solution of the equation. Secondly, it classify, at the Markovian sequential decision process, the case of discrete time parameter and the continuous time parameter, and then it explore the logic system of characteristic of the behavior, the value determination operation and the policy improvement routine.

  • PDF

An Application of Delphi Method to the Assessment of Current Status of Cancer Research (델파이법을 적용한 암연구수준의 평가)

  • Khang, Young-Ho;Yoon, Seok-Jun;Kang, Gil-Won;Kim, Chang-Yup;Yoo, Keun-Young;Shin, Young-Soo
    • Journal of Preventive Medicine and Public Health
    • /
    • v.31 no.4 s.63
    • /
    • pp.844-856
    • /
    • 1998
  • Globally, cancer research has been considered one of the most important field of biomedical researches. Recently, in Korea, there are increasing concerns about cancer research and the development of national cancer control programme. For the efficient investment in cancer research at the national level, strategic approach is needed based on the nationwide information about current status of research. However even the basic data on cancer research have not been systematically collected, and are not available when necessary. The aim of this study is to assess current status of cancer research. For this purpose, this study applied two round Delphi method in which fifteen experts in cancer research fields participated. They rated each items on the initial list at the first round, and modified their responses at the second round. Panels responded that pathogenesis of cancer, research & development of cancer drug, and oncogene, etc. are the most urgent and important research Holds. They assessed national level of cancer research as being 49.6% of the world highest level. Coefficient of variation tended to be lowered with the iteration. Predictive stability was evaluated to be lower in items of urgency than in items of importance and research level. Although this study shares the same limitations in the selection of the experts with many other Delphi studies, it provides a primary data that would be required to plan the national strategy of the cancer research.

  • PDF

Optimal solution search method by using modified local updating rule in ACS-subpath algorithm (부경로를 이용한 ACS 탐색에서 수정된 지역갱신규칙을 이용한 최적해 탐색 기법)

  • Hong, SeokMi;Lee, Seung-Gwan
    • Journal of Digital Convergence
    • /
    • v.11 no.11
    • /
    • pp.443-448
    • /
    • 2013
  • Ant Colony System(ACS) is a meta heuristic approach based on biology in order to solve combinatorial optimization problem. It is based on the tracing action of real ants which accumulate pheromone on the passed path and uses as communication medium. In order to search the optimal path, ACS requires to explore various edges. In existing ACS, the local updating rule assigns the same pheromone to visited edge. In this paper, our local updating rule gives the pheromone according to the total frequency of visits of the currently selected node in the previous iteration. I used the ACS algoritm using subpath for search. Our approach can have less local optima than existing ACS and find better solution by taking advantage of more informations during searching.

Advanced controller design for AUV based on adaptive dynamic programming

  • Chen, Tim;Khurram, Safiullahand;Zoungrana, Joelli;Pandey, Lallit;Chen, J.C.Y.
    • Advances in Computational Design
    • /
    • v.5 no.3
    • /
    • pp.233-260
    • /
    • 2020
  • The main purpose to introduce model based controller in proposed control technique is to provide better and fast learning of the floating dynamics by means of fuzzy logic controller and also cancelling effect of nonlinear terms of the system. An iterative adaptive dynamic programming algorithm is proposed to deal with the optimal trajectory-tracking control problems for autonomous underwater vehicle (AUV). The optimal tracking control problem is converted into an optimal regulation problem by system transformation. Then the optimal regulation problem is solved by the policy iteration adaptive dynamic programming algorithm. Finally, simulation example is given to show the performance of the iterative adaptive dynamic programming algorithm.

Forecasting KOSPI Return Using a Modified Stochastic AdaBoosting

  • Bae, Sangil;Jeong, Minsoo
    • East Asian Economic Review
    • /
    • v.25 no.4
    • /
    • pp.403-424
    • /
    • 2021
  • AdaBoost tweaks the sample weight for each training set used in the iterative process, however, it is demonstrated that it provides more correlated errors as the boosting iteration proceeds if models' accuracy is high enough. Therefore, in this study, we propose a novel way to improve the performance of the existing AdaBoost algorithm by employing heterogeneous models and a stochastic twist. By employing the heterogeneous ensemble, it ensures different models that have a different initial assumption about the data are used to improve on diversity. Also, by using a stochastic algorithm with a decaying convergence rate, the model is designed to balance out the trade-off between model prediction performance and model convergence. The result showed that the stochastic algorithm with decaying convergence rate's did have a improving effect and outperformed other existing boosting techniques.

Control of pH Neutralization Process using Simulation Based Dynamic Programming in Simulation and Experiment (ICCAS 2004)

  • Kim, Dong-Kyu;Lee, Kwang-Soon;Yang, Dae-Ryook
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2004.08a
    • /
    • pp.620-626
    • /
    • 2004
  • For general nonlinear processes, it is difficult to control with a linear model-based control method and nonlinear controls are considered. Among the numerous approaches suggested, the most rigorous approach is to use dynamic optimization. Many general engineering problems like control, scheduling, planning etc. are expressed by functional optimization problem and most of them can be changed into dynamic programming (DP) problems. However the DP problems are used in just few cases because as the size of the problem grows, the dynamic programming approach is suffered from the burden of calculation which is called as 'curse of dimensionality'. In order to avoid this problem, the Neuro-Dynamic Programming (NDP) approach is proposed by Bertsekas and Tsitsiklis (1996). To get the solution of seriously nonlinear process control, the interest in NDP approach is enlarged and NDP algorithm is applied to diverse areas such as retailing, finance, inventory management, communication networks, etc. and it has been extended to chemical engineering parts. In the NDP approach, we select the optimal control input policy to minimize the value of cost which is calculated by the sum of current stage cost and future stages cost starting from the next state. The cost value is related with a weight square sum of error and input movement. During the calculation of optimal input policy, if the approximate cost function by using simulation data is utilized with Bellman iteration, the burden of calculation can be relieved and the curse of dimensionality problem of DP can be overcome. It is very important issue how to construct the cost-to-go function which has a good approximate performance. The neural network is one of the eager learning methods and it works as a global approximator to cost-to-go function. In this algorithm, the training of neural network is important and difficult part, and it gives significant effect on the performance of control. To avoid the difficulty in neural network training, the lazy learning method like k-nearest neighbor method can be exploited. The training is unnecessary for this method but requires more computation time and greater data storage. The pH neutralization process has long been taken as a representative benchmark problem of nonlin ar chemical process control due to its nonlinearity and time-varying nature. In this study, the NDP algorithm was applied to pH neutralization process. At first, the pH neutralization process control to use NDP algorithm was performed through simulations with various approximators. The global and local approximators are used for NDP calculation. After that, the verification of NDP in real system was made by pH neutralization experiment. The control results by NDP algorithm was compared with those by the PI controller which is traditionally used, in both simulations and experiments. From the comparison of results, the control by NDP algorithm showed faster and better control performance than PI controller. In addition to that, the control by NDP algorithm showed the good results when it applied to the cases with disturbances and multiple set point changes.

  • PDF

Behavior-Structure-Evolution Evaluation Model(BSEM) for Open Source Software Service (공개소프트웨어 서비스 평가모델(BSEM)에 관한 개념적 연구)

  • Lee, Seung-Chang;Park, Hoon-Sung;Suh, Eung-Kyo
    • Journal of Distribution Science
    • /
    • v.13 no.1
    • /
    • pp.57-70
    • /
    • 2015
  • Purpose - Open source software has high utilization in most of the server market. The utilization of open source software is a global trend. Particularly, Internet infrastructure and platform software open source software development has increased rapidly. Since 2003, the Korean government has published open source software promotion policies and a supply promotion policy. The dynamism of the open source software market, the lack of relevant expertise, and the market transformation due to reasons such as changes in the relevant technology occur slowly in relation to adoption. Therefore, this study proposes an assessment model of services provided in an open source software service company. In this study, the service level of open source software companies is classified into an enterprise-level assessment area, the service level assessment area, and service area. The assessment model is developed from an on-site driven evaluation index and proposed evaluation framework; the evaluation procedures and evaluation methods are used to achieve the research objective, involving an impartial evaluation model implemented after pilot testing and validation. Research Design, data, and methodology - This study adopted an iteration development model to accommodate various requirements, and presented and validated the assessment model to address the situation of the open source software service company. Phase 1 - Theoretical background and literature review Phase 2 - Research on an evaluation index based on the open source software service company Phase 3 - Index improvement through expert validation Phase 4 - Finalizing an evaluation model reflecting additional requirements Based on the open source software adoption case study and latest technology trends, we developed an open source software service concept definition and classification of public service activities for open source software service companies. We also presented open source software service company service level measures by developing a service level factor analysis assessment. The Behavior-Structure-Evolution Evaluation Model (BSEM) proposed in this study consisted of a rating methodology for calculating the level that can be granted through the assessment and evaluation of an enterprise-level data model. An open source software service company's service comprises the service area and service domain, while the technology acceptance model comprises the service area, technical domain, technical sub-domain, and open source software name. Finally, the evaluation index comprises the evaluation group, category, and items. Results - Utilization of an open source software service level evaluation model For the development of an open source software service level evaluation model, common service providers need to standardize the quality of the service, so that surveys and expert workshops performed in open source software service companies can establish the evaluation criteria according to their qualitative differences. Conclusion - Based on this evaluation model's systematic evaluation process and monitoring, an open source software service adoption company can acquire reliable information for open source software adoption. Inducing the growth of open source software service companies will facilitate the development of the open source software industry.