• International Journal of Control, Automation, and Systems
• /
• v.2 no.3
• /
• pp.263-278
• /
• 2004

This paper reviews dynamic programming (DP), surveys approximate solution methods for it, and considers their applicability to process control problems. Reinforcement Learning (RL) and Neuro-Dynamic Programming (NDP), which can be viewed as approximate DP techniques, are already established techniques for solving difficult multi-stage decision problems in the fields of operations research, computer science, and robotics. Owing to the significant disparity of problem formulations and objective, however, the algorithms and techniques available from these fields are not directly applicable to process control problems, and reformulations based on accurate understanding of these techniques are needed. We categorize the currently available approximate solution techniques fur dynamic programming and identify those most suitable for process control problems. Several open issues are also identified and discussed.

• 제어로봇시스템학회:학술대회논문집
• /
• 2003.10a
• /
• pp.2617-2622
• /
• 2003

The pH neutralization process has long been taken as a representative benchmark problem of nonlinear chemical process control due to its nonlinearity and time-varying nature. For general nonlinear processes, it is difficult to control with a linear model-based control method so nonlinear controls must be considered. Among the numerous approaches suggested, the most rigorous approach is the dynamic optimization. However, as the size of the problem grows, the dynamic programming approach is suffered from the curse of dimensionality. In order to avoid this problem, the Neuro-Dynamic Programming (NDP) approach was proposed by Bertsekas and Tsitsiklis (1996). The NDP approach is to utilize all the data collected to generate an approximation of optimal cost-to-go function which was used to find the optimal input movement in real time control. The approximation could be any type of function such as polynomials, neural networks and etc. In this study, an algorithm using NDP approach was applied to a pH neutralization process to investigate the feasibility of the NDP algorithm and to deepen the understanding of the basic characteristics of this algorithm. As the global approximator, the neural network which requires training and k-nearest neighbor method which requires querying instead of training are investigated. The global approximator requires optimal control strategy. If the optimal control strategy is not available, suboptimal control strategy can be used even though the laborious Bellman iterations are necessary. For pH neutralization process it is rather easy to devise an optimal control strategy. Thus, we used an optimal control strategy and did not perform the Bellman iteration. Also, the effects of constraints on control moves are studied. From the simulations, the NDP method outperforms the conventional PID control.

• 제어로봇시스템학회:학술대회논문집
• /
• 2004.08a
• /
• pp.620-626
• /
• 2004

For general nonlinear processes, it is difficult to control with a linear model-based control method and nonlinear controls are considered. Among the numerous approaches suggested, the most rigorous approach is to use dynamic optimization. Many general engineering problems like control, scheduling, planning etc. are expressed by functional optimization problem and most of them can be changed into dynamic programming (DP) problems. However the DP problems are used in just few cases because as the size of the problem grows, the dynamic programming approach is suffered from the burden of calculation which is called as 'curse of dimensionality'. In order to avoid this problem, the Neuro-Dynamic Programming (NDP) approach is proposed by Bertsekas and Tsitsiklis (1996). To get the solution of seriously nonlinear process control, the interest in NDP approach is enlarged and NDP algorithm is applied to diverse areas such as retailing, finance, inventory management, communication networks, etc. and it has been extended to chemical engineering parts. In the NDP approach, we select the optimal control input policy to minimize the value of cost which is calculated by the sum of current stage cost and future stages cost starting from the next state. The cost value is related with a weight square sum of error and input movement. During the calculation of optimal input policy, if the approximate cost function by using simulation data is utilized with Bellman iteration, the burden of calculation can be relieved and the curse of dimensionality problem of DP can be overcome. It is very important issue how to construct the cost-to-go function which has a good approximate performance. The neural network is one of the eager learning methods and it works as a global approximator to cost-to-go function. In this algorithm, the training of neural network is important and difficult part, and it gives significant effect on the performance of control. To avoid the difficulty in neural network training, the lazy learning method like k-nearest neighbor method can be exploited. The training is unnecessary for this method but requires more computation time and greater data storage. The pH neutralization process has long been taken as a representative benchmark problem of nonlin ar chemical process control due to its nonlinearity and time-varying nature. In this study, the NDP algorithm was applied to pH neutralization process. At first, the pH neutralization process control to use NDP algorithm was performed through simulations with various approximators. The global and local approximators are used for NDP calculation. After that, the verification of NDP in real system was made by pH neutralization experiment. The control results by NDP algorithm was compared with those by the PI controller which is traditionally used, in both simulations and experiments. From the comparison of results, the control by NDP algorithm showed faster and better control performance than PI controller. In addition to that, the control by NDP algorithm showed the good results when it applied to the cases with disturbances and multiple set point changes.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2010.05a
• /
• pp.779-783
• /
• 2010

최근 들어 그 심각성을 더하고 있는 이상기후 현상으로 가용 수자원의 변동이 커지고 있으며, 이에 따라 수자원의 효율적인 운영이 요구되고 있다. 그러나 효율적인 운영을 위해서는 미래 유입량의 불확실성의 고려하고, 홍수 조절용량의 확보하면서도, 용수공급을 위한 저수량을 확보하고, 수력 발전을 해야 하는 복잡한 상황을 모두 고려하여야한다. 이러한 복잡한 시스템에서 하나의 최적화 기법으로는 모든 고려사항들을 만족시키는 최적해를 찾는 것은 사실상 불가능에 가깝다. 그러므로 저수지 운영의 최적화를 위한 연구에서 한 가지 이상의 기법을 조합하는 기법을 사용하게 되었다. 이러한 기법은 각 기법의 장점을 취하고 각각의 한계를 극복하기 위해 주로 사용되었다. 본 연구에서는 저수지 운영 최적화를 모의하기 위하여 대청댐에서의 저수위, 유입량, 용수이용량 등을 고려하여 방류량의 예측을 동적 계획법(Dynamic Programming Model)으로부터 동적 신경망(Dynamic Neural Network Model)과 적응 퍼지 제어기법(Adaptive Neuro-Fuzzy Inference System)을 개발하여 실제 방류량과 세 가지 최적화 방법에 의한 결과를 비교 검정하였다. 본 연구의 수행으로 인해 얻어진 결과를 요약하면 다음과 같다. 첫째, 동적 신경망과 적응 퍼지 제어기법에 의한 최적화 모의가 동적 계획법에 비해 시스템의 구축이 쉽고 유연하다. 둘째, 퍼지추론의 Membership 함수의 구축에 따라 단시간에 많은 양의 강우가 발생하는 국지성 강우에 대해서도 최적 방류량을 예측할 수 있다. 셋째, 저수지 운영 과거자료의 부족과 불확실성을 해결하면, 보다 용이하고 양호한 예측결과를 얻을 수 있을 것이다.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2011.05a
• /
• pp.18-18
• /
• 2011

Climate change is impacting and will increasingly impact both the quantity and quality of the world's water resources in a variety of ways. In some areas warming climate results in increased rainfall, surface runoff, and groundwater recharge while in others there may be declines in all of these. Water quality is described by a number of variables. Some are directly impacted by climate change. Temperature is an obvious example. Notably, increased atmospheric concentrations of $CO_2$ triggering climate change increase the $CO_2$ dissolving into water. This has manifold consequences including decreased pH and increased alkalinity, with resultant increases in dissolved concentrations of the minerals in geologic materials contacted by such water. Climate change is also expected to increase the number and intensity of extreme climate events, with related hydrologic changes. A simple framework has been developed in New Zealand for assessing and predicting climate change impacts on water resources. Assessment is largely based on trend analysis of historic data using the non-parametric Mann-Kendall method. Trend analysis requires long-term, regular monitoring data for both climate and hydrologic variables. Data quality is of primary importance and data gaps must be avoided. Quantitative prediction of climate change impacts on the quantity of water resources can be accomplished by computer modelling. This requires the serial coupling of various models. For example, regional downscaling of results from a world-wide general circulation model (GCM) can be used to forecast temperatures and precipitation for various emissions scenarios in specific catchments. Mechanistic or artificial intelligence modelling can then be used with these inputs to simulate climate change impacts over time, such as changes in streamflow, groundwater-surface water interactions, and changes in groundwater levels. The Waimea Plains catchment in New Zealand was selected for a test application of these assessment and prediction methods. This catchment is predicted to undergo relatively minor impacts due to climate change. All available climate and hydrologic databases were obtained and analyzed. These included climate (temperature, precipitation, solar radiation and sunshine hours, evapotranspiration, humidity, and cloud cover) and hydrologic (streamflow and quality and groundwater levels and quality) records. Results varied but there were indications of atmospheric temperature increasing, rainfall decreasing, streamflow decreasing, and groundwater level decreasing trends. Artificial intelligence modelling was applied to predict water usage, rainfall recharge of groundwater, and upstream flow for two regionally downscaled climate change scenarios (A1B and A2). The AI methods used were multi-layer perceptron (MLP) with extended Kalman filtering (EKF), genetic programming (GP), and a dynamic neuro-fuzzy local modelling system (DNFLMS), respectively. These were then used as inputs to a mechanistic groundwater flow-surface water interaction model (MODFLOW). A DNFLMS was also used to simulate downstream flow and groundwater levels for comparison with MODFLOW outputs. MODFLOW and DNFLMS outputs were consistent. They indicated declines in streamflow on the order of 21 to 23% for MODFLOW and DNFLMS (A1B scenario), respectively, and 27% in both cases for the A2 scenario under severe drought conditions by 2058-2059, with little if any change in groundwater levels.