• Title/Summary/Keyword: Policy Optimization

Search Result 303, Processing Time 0.025 seconds

Optimization for Inventory Level of Spare Parts Considering System Availability (시스템 가용도를 고려한 수리부품의 재고수준 최적화)

  • Kim, Heung-Seob;Kim, Pansoo
    • Korean Management Science Review
    • /
    • v.31 no.2
    • /
    • pp.1-13
    • /
    • 2014
  • In almost all of the organizations, the cost for acquiring and maintaining the inventory takes a considerable portion of the management budget, and thus a certain constraint is set upon the budget itself. The previous studies on inventory control for each item that aimed to improve the fill rate, backorder, and the expenditure on inventory are fitting for the commercially-operated SCM, but show some discrepancies when they are applied to the spare parts for repairing disabled systems. Therefore, many studies on systematic approach concept considering spare parts of various kinds simultaneously have been conducted to achieve effective performance for the inventory control at a lower cost, and primarily, METRIC series models can be named. However, the past studies were limited when dealing with the probability distributions for representing the situation on demand and transportation of the parts, with the (S-1, S) inventory control policy, and so on. To address these shortcomings, the Continuous Time Markov Chain (CTMC) model, which considers the phase-type distributions and the (s, Q) inventory control policies to best describe the real-world situations inclusively, is presented in this study. Additionally, by considering the cost versus the system availability, the optimization of the inventory level, based on this model, is also covered.

Electricity Cost Minimization for Delay-tolerant Basestation Powered by Heterogeneous Energy Source

  • Deng, Qingyong;Li, Xueming;Li, Zhetao;Liu, Anfeng;Choi, Young-june
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.12
    • /
    • pp.5712-5728
    • /
    • 2017
  • Recently, there are many studies, that considering green wireless cellular networks, have taken the energy consumption of the base station (BS) into consideration. In this work, we first introduce an energy consumption model of multi-mode sharing BS powered by multiple energy sources including renewable energy, local storage and power grid. Then communication load requests of the BS are transformed to energy demand queues, and battery energy level and worst-case delay constraints are considered into the virtual queue to ensure the network QoS when our objective is to minimize the long term electricity cost of BSs. Lyapunov optimization method is applied to work out the optimization objective without knowing the future information of the communication load, real-time electricity market price and renewable energy availability. Finally, linear programming is used, and the corresponding energy efficient scheduling policy is obtained. The performance analysis of our proposed online algorithm based on real-world traces demonstrates that it can greatly reduce one day's electricity cost of individual BS.

Gaussian Model Optimization using Configuration Thread Control In CHMM Vocabulary Recognition (CHMM 어휘 인식에서 형상 형성 제어를 이용한 가우시안 모델 최적화)

  • Ahn, Chan-Shik;Oh, Sang-Yeob
    • Journal of Digital Convergence
    • /
    • v.10 no.7
    • /
    • pp.167-172
    • /
    • 2012
  • In vocabulary recognition using HMM(Hidden Markov Model) by model for the observation of a discrete probability distribution indicates the advantages of low computational complexity, but relatively low recognition rate has the disadvantage that require sophisticated smoothing process. Gaussian mixtures in order to improve them with a continuous probability density CHMM (Continuous Hidden Markov Model) model is proposed for the optimization of the library system. In this paper is system configuration thread control in recognition Gaussian mixtures model provides a model to optimize of the CHMM vocabulary recognition. The result of applying the proposed system, the recognition rate of 98.1% in vocabulary recognition, respectively.

Joint Optimization Algorithm Based on DCA for Three-tier Caching in Heterogeneous Cellular Networks

  • Zhang, Jun;Zhu, Qi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.15 no.7
    • /
    • pp.2650-2667
    • /
    • 2021
  • In this paper, we derive the expression of the cache hitting probability with random caching policy and propose the joint optimization algorithm based on difference of convex algorithm (DCA) in the three-tier caching heterogeneous cellular network assisted by macro base stations, helpers and users. Under the constraint of the caching capacity of caching devices, we establish the optimization problem to maximize the cache hitting probability of the network. In order to solve this problem, a convex function is introduced to convert the nonconvex problem to a difference of convex (DC) problem and then we utilize DCA to obtain the optimal caching probability of macro base stations, helpers and users for each content respectively. Simulation results show that when the density of caching devices is relatively low, popular contents should be cached to achieve a good performance. However, when the density of caching devices is relatively high, each content ought to be cached evenly. The algorithm proposed in this paper can achieve the higher cache hitting probability with the same density.

Flexible operation and maintenance optimization of aging cyber-physical energy systems by deep reinforcement learning

  • Zhaojun Hao;Francesco Di Maio;Enrico Zio
    • Nuclear Engineering and Technology
    • /
    • v.56 no.4
    • /
    • pp.1472-1479
    • /
    • 2024
  • Cyber-Physical Energy Systems (CPESs) integrate cyber and hardware components to ensure a reliable and safe physical power production and supply. Renewable Energy Sources (RESs) add uncertainty to energy demand that can be dealt with flexible operation (e.g., load-following) of CPES; at the same time, scenarios that could result in severe consequences due to both component stochastic failures and aging of the cyber system of CPES (commonly overlooked) must be accounted for Operation & Maintenance (O&M) planning. In this paper, we make use of Deep Reinforcement Learning (DRL) to search for the optimal O&M strategy that, not only considers the actual system hardware components health conditions and their Remaining Useful Life (RUL), but also the possible accident scenarios caused by the failures and the aging of the hardware and the cyber components, respectively. The novelty of the work lies in embedding the cyber aging model into the CPES model of production planning and failure process; this model is used to help the RL agent, trained with Proximal Policy Optimization (PPO) and Imitation Learning (IL), finding the proper rejuvenation timing for the cyber system accounting for the uncertainty of the cyber system aging process. An application is provided, with regards to the Advanced Lead-cooled Fast Reactor European Demonstrator (ALFRED).

Optimal Electric Energy Subscription Policy for Multiple Plants with Uncertain Demand

  • Nilrangsee, Puvarin;Bohez, Erik L.J.
    • Industrial Engineering and Management Systems
    • /
    • v.6 no.2
    • /
    • pp.106-118
    • /
    • 2007
  • This paper present a new optimization model to generate aggregate production planning by considering electric cost. The new Time Of Switching (TOS) electric type is introduced by switching over Time Of Day (TOD) and Time Of Use (TOU) electric types to minimize the electric cost. The fuzzy demand and Dynamic inventory tracking with multiple plant capacity are modeled to cover the uncertain demand of customer. The constraint for minimum hour limitation of plant running per one start up event is introduced to minimize plants idle time. Furthermore; the Optimal Weight Moving Average Factor for customer demand forecasting is introduced by monthly factors to reduce forecasting error. Application is illustrated for multiple cement mill plants. The mathematical model was formulated in spreadsheet format. Then the spreadsheet-solver technique was used as a tool to solve the model. A simulation running on part of the system in a test for six months shows the optimal solution could save 60% of the actual cost.

Approximate Solution to Optimal Packing Problem by Renewal Process (재생확률과정에 의한 최적 포장계획 수립에 관한 연구)

  • Lee, Ho-Chang
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.22 no.2
    • /
    • pp.125-137
    • /
    • 1997
  • We are concerned with the packing policy determines the optimal packing of products with variable sizes to minimize the penalty costs for idle space and product spliting. Optimal packing problem is closely related to the optimal packet/record sizing problem in that randomly generated data stream with variable bytes are divided into a unit of packet/record for transmitting or storing. Assuming the product size and the production period are independently determined by renewal process, we can approximate the renewal process and formulate the optimization problem that minimize the expected packing cost for a production period. The problem is divided into two cases according to whether a product is allowed to split or not. Computational results for various distributions will be given to verify the approximation procedure and the resulting optimization problem.

  • PDF

Intelligent Decision Support Algorithm for Uncertain Inventory Management

  • Le Ngoc Bao Long;Sam-Sang You;Truong Ngoc Cuong;Hwan-Seong Kim
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2023.05a
    • /
    • pp.254-255
    • /
    • 2023
  • This paper discovers a robust managerial strategy for a stochastic inventory of perishable products, where the model experiences changing factors including inner parameters and an external disturbance with unknown form. An analytical solution for the optimization problem can be obtained by applying the Hamilton-Bellman-Jacobi equation, however the policy result cannot completely suppress the oscillation from the external disturbance. Therefore, an intelligent approach named Radial Basis Function Neural Networks is applied to estimate the unknown disturbance and provide a robust controller to manipulate the inventory level more effective. The final results show the outstanding performance of RBFNN controller, where both the estimation error and control error are guaranteed in the predefined limit.

  • PDF

Reinforcement learning portfolio optimization based on portfolio theory (강화학습을 이용한 포트폴리오 투자 프로세스 최적화에 대한 연구)

  • Hyeong-Jin Son;Lim Donhui;Young-Woo Han
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.11a
    • /
    • pp.961-962
    • /
    • 2023
  • 포트폴리오 구성문제는 과거부터 현재까지 많은 연구가 이루어지고 있다. 현재는 강화학습을 통해 포트폴리오를 구성하는 연구가 많이 진행되고있다. 포트폴리오를 구성함에 있어 종목선택과 각 종목을 얼만큼 투자할 것인지는 둘 다 중요한 문제이다. 본 연구에서는 과거부터 많이 사용해오던 방식을 차용하여 강화학습 방법과 접목시켰고 이를 통해 설명력이 높은 모델을 만들려고 노력하였다. 강화학습에 사용한 모델은 PPO(Proximal Policy Optimization)을 기본으로 하였고 인공신경망은 LSTM을 활용하였다. 실험결과 실험 기간 동안(2023년 3월 30일 부터 108 영업일 까지)의 코스피 수익률은 5%인데 반해 본 연구에서 제시한 모델의 수익률은 평균 약 9%를 기록했다.

Optimal deep machine learning framework for vibration mitigation of seismically-excited uncertain building structures

  • Afshin Bahrami Rad;Javad Katebi;Saman Yaghmaei-Sabegh
    • Structural Engineering and Mechanics
    • /
    • v.88 no.6
    • /
    • pp.535-549
    • /
    • 2023
  • Deep extreme learning machine (DELM) and multi-verse optimization algorithms (MVO) are hybridized for designing an optimal and adaptive control framework for uncertain buildings. In this approach, first, a robust model predictive control (RMPC) scheme is developed to handle the problem uncertainty. The optimality and adaptivity of the proposed controller are provided by the optimal determination of the tunning weights of the linear programming (LP) cost function for clustered external loads using the MVO. The final control policy is achieved by collecting the clustered data and training them by DELM. The efficiency of the introduced control scheme is demonstrated by the numerical simulation of a ten-story benchmark building subjected to earthquake excitations. The results represent the capability of the proposed framework compared to robust MPC (RMPC), conventional MPC (CMPC), and conventional DELM algorithms in structural motion control.