• International Journal of Control, Automation, and Systems
• /
• 제1권3호
• /
• pp.358-367
• /
• 2003

We consider discrete-time factorial Markov Decision Processes (MDPs) in multiple decision-makers environment for infinite horizon average reward criterion with a general joint reward structure but a factorial joint state transition structure. We introduce the "localization" concept that a global MDP is localized for each agent such that each agent needs to consider a local MDP defined only with its own state and action spaces. Based on that, we present a gradient-ascent like iterative distributed algorithm that converges to a local optimal solution of the global MDP. The solution is an autonomous joint policy in that each agent's decision is based on only its local state.cal state.

• 산업공학
• /
• 제24권4호
• /
• pp.274-280
• /
• 2011

Delayed access to surgery may lead to deterioration in the patient condition, poor clinical outcomes, increase in the probability of emergency admission, or even death. The purpose of this work is to decide the number of patients selected from a waiting list and to schedule them in accordance with the operating room capacity in the next period. We formulate the problem as an infinite horizon Markov Decision Process (MDP), which attempts to strike a balance between the patient waiting times and overtime works. Structural properties of the proposed model are investigated to facilitate the solution procedure. The proposed procedure modifies the conventional value iteration method along with the binary search technique. An example of the optimal policy is provided, and computational results are given to show that the proposed procedure improves computational efficiency.

• 품질경영학회지
• /
• 제16권2호
• /
• pp.1-9
• /
• 1988

This paper considers a machine maintenance problem. The machine's condition is partially known by observing the machine's output products. This problem is formulated as an infinite horizon partially observable Markov decison process to find an optimal maintenance policy. However, even though the optimal policy of the model exists, finding the optimal policy is very time consuming. Thus, the intends of this study is to find ${\varepsilon}-optimal$ stationary policy minimizing the expected discounted total cost of the system, ${\varepsilon}-optimal$ policy is found by using a modified version of the well-known policy iteration algorithm. A numerical example is also shown.

• 품질경영학회지
• /
• 제22권1호
• /
• pp.214-218
• /
• 1994

An inventory supplies stock continuously at a constant rate. A deliveryman arrives according to a Poisson process. If the level of the inventory, when he arrives, exceeds a threshold, no action is taken, otherwise a delivery is made by a random amount. Costs are assigned to each visit of the deliveryman, to each delivery, to the inventory being empty and to the stock being kept. It is shown that there exists a unique arrival rate of the deliveryman which minimizes the average cost per unit time over an infinite horizon.

• Industrial Engineering and Management Systems
• /
• 제3권2호
• /
• pp.134-139
• /
• 2004

During the last three decades, a few attentions have been paid for investigating the cost distribution for the optimal maintenance problems. In this article, we derive the moment of the discounted cost distribution over an infinite time horizon for the basic age replacement problem. With first two moments of the discounted cost distribution, we approximate the underlying distribution function by three theoretical distributions. Through a Monte Carlo simulation, we conclude that the log-normal distribution is the best fitted one to approximate the discounted cost distribution.

• 한국경영과학회지
• /
• 제5권1호
• /
• pp.39-51
• /
• 1980

This paper deals with the problem of determining the capacity expansion timing and sizes of conduits and feeder cables for a given cable network configuration of a single exchange ares, which minimizes the present worth of total costs. The planning horizon is infinite and the demand of line pairs at each cabinet is assumed to be determininstically growing. As a solution method, the heuristic branch-and-bound algorithm of Freidenfelds and Mclaughlin is elaborated by adding details and some minor modifications, which generates a good near-optimal solution with far less computation than would otherwise be possible. We also develop a computer program, which is shown to be effective and efficient through the test run of an illustrative example.

• Journal of the Korean Data and Information Science Society
• /
• 제25권4호
• /
• pp.915-920
• /
• 2014

We consider a continuous time surplus process with investments the sizes of which are independent and identically distributed. It is assumed that an investment of the surplus to other business is made, if and only if the surplus reaches a given sufficient level. We establish an integro-differential equation for the distribution function of the surplus and solve the equation to obtain the moment generating function for the stationary distribution of the surplus. As a consequence, we obtain the first and second moments of the level of the surplus in an infinite horizon.

• Journal of applied mathematics & informatics
• /
• 제13권1_2호
• /
• pp.85-97
• /
• 2003

An indefinite stochastic linear-quadratic(LQ) optimal control problem with cross term over an infinite time horizon is studied, allowing the weighting matrices to be indefinite. A systematic approach to the problem based on semidefinite programming (SDP) and .elated duality analysis is developed. Several implication relations among the SDP complementary duality, the existence of the solution to the generalized Riccati equation and the optimality of LQ problem are discussed. Based on these relations, a numerical procedure that provides a thorough treatment of the LQ problem via primal-dual SDP is given: it identifies a stabilizing optimal feedback control or determines the problem has no optimal solution. An example is provided to illustrate the results obtained.

• Management Science and Financial Engineering
• /
• 제14권1호
• /
• pp.35-63
• /
• 2008

We discuss the problem of selecting profitable customer orders arriving at a company providing two classes of services. For an arriving customer order for the first class, the company 1) makes a decision whether to accept or reject it (admission control), or 2) decides a price of the order to offer to an arriving customer (pricing control). The customer order undergoing processing in the system may be cancelled owing to unavoidable circumstances with the customer. The second class of service is provided as a sideline, which prevents the server from being idle when all the customer orders for the first class are completed and delivered. This yields the sideline profit. We discuss both admission control and pricing control problems in an identical framework as well as examine the structure of the optimal policies maximizing the total expected present discounted net profit gained over an infinite planning horizon.

• Communications for Statistical Applications and Methods
• /
• 제20권6호
• /
• pp.475-480
• /
• 2013

The surplus process in a risk model is stochastically analyzed. We obtain the characteristic function of the level of the surplus at a finite time, by establishing and solving an integro-differential equation for the distribution function of the surplus. The characteristic function of the stationary distribution of the surplus is also obtained by assuming that an investment of the surplus is made to other business when the surplus reaches a sufficient level. As a consequence, we obtain the first and second moments of the surplus both at a finite time and in an infinite horizon (in the long-run).