• Proceedings of the Korea Water Resources Association Conference
• /
• 2006.05a
• /
• pp.354-359
• /
• 2006

The analysis of large-scale water resources systems is often complicated by the presence of multiple reservoirs and diversions, the uncertainty of unregulated inflows and demands, and conflicting objectives. Reinforcement learning is presented herein as a new approach to solving the challenging problem of stochastic optimization of multi-reservoir systems. The Q-Learning method, one of the reinforcement learning algorithms, is used for generating integrated monthly operation rules for the Keum River basin in Korea. The Q-Learning model is evaluated by comparing with implicit stochastic dynamic programming and sampling stochastic dynamic programming approaches. Evaluation of the stochastic basin-wide operational models considered several options relating to the choice of hydrologic state and discount factors as well as various stochastic dynamic programming models. The performance of Q-Learning model outperforms the other models in handling of uncertainty of inflows.

• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• v.25 no.4
• /
• pp.162-172
• /
• 2021

This paper proposes stochastic methods to find an approximate solution for the L²-Wasserstein least squares problem of Gaussian measures. The variable for the problem is in a set of positive definite matrices. The first proposed stochastic method is a type of classical stochastic gradient methods combined with projection and the second one is a type of variance reduced methods with projection. Their global convergence are analyzed by using the framework of proximal stochastic gradient methods. The convergence of the classical stochastic gradient method combined with projection is established by using diminishing learning rate rule in which the learning rate decreases as the epoch increases but that of the variance reduced method with projection can be established by using constant learning rate. The numerical results show that the present algorithms with a proper learning rate outperforms a gradient projection method.

• Journal of Korean Institute of Industrial Engineers
• /
• v.32 no.1
• /
• pp.61-73
• /
• 2006

This paper deals with traveling salesman problem(TSP) with the stochastic travel time. Practically, the travel time between demand points changes according to day and time zone because of traffic interference and jam. Since the almost pervious studies focus on TSP with the deterministic travel time, it is difficult to apply those results to logistics problem directly. But many logistics problems are strongly related with stochastic situation such as stochastic travel time. We need to develop the efficient solution method for the TSP with stochastic travel time. From the previous researches, we know that Q-learning technique gives us to deal with stochastic environment and neural network also enables us to calculate the Q-value of Q-learning algorithm. In this paper, we suggest an algorithm for TSP with the stochastic travel time integrating Q-learning and neural network. And we evaluate the validity of the algorithm through computational experiments. From the simulation results, we conclude that a new route obtained from the suggested algorithm gives relatively more reliable travel time in the logistics situation with stochastic travel time.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 1998.10a
• /
• pp.346-352
• /
• 1998

In this paper, we give some geometric condition for a stochastic nonlinear system and we propose a control method for a stochastic nonlinear system using neural networks. Since a competitive learning neural networks has been developed based on the stochastic approximation method, it is regarded as a stochastic recursive filter algorithm. In addition, we provide a filtering and control condition for a stochastic nonlinear system, called perfect filtering condition, in a viewpoint of stochastic geometry. The stochastic nonlinear system satisfying the perfect filtering condition is decoupled with a deterministic part and purely semi martingale part. Hence, the above system can be controlled by conventional control laws and various intelligent control laws. Computer simulation shows that the stochastic nonlinear system satisfying the perfect filtering condition is controllable. and the proposed neural controller is more efficient than the conventional LQG controller and the canoni al LQ-Neural controller.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.23 no.4
• /
• pp.348-353
• /
• 2013

Recently, control theory including stochastic optimal control and various machine-learning-based artificial intelligence methods have become major tools in the field of financial engineering. In this paper, we briefly review some recent papers utilizing stochastic optimal control theory in the fields of the pair trading for mean-reverting markets and the trend-following strategy, and consider a couple of strategies utilizing both stochastic optimal control theory and machine learning methods to acquire more flexible and accessible tools. Illustrative simulations show that the considered strategies can yield encouraging results when applied to a set of real financial market data.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.10 no.3
• /
• pp.270-279
• /
• 2000

This paper describes the on-chip learning algorithm of neural networks using the stochastic pulse arithmetic. Stochastic pulse arithmetic is the computation using the numbers represented by the probability of 1' and 0's occurrences in a random pulse stream. This stochastic arithmetic has the merits when applied to neural network ; reduction of the area of the implemented hardware and getting a global solution escaping from local minima by virtue of the stochastic characteristics. And in this study, the on-chip learning algorithm is derived from the backpropagation algorithm for effective hardware implementation. We simulate the nonlinear separation problem of the some character patterns to verify the proposed learning algorithm. We also had good results after applying this algorithm to recognize printed and handwritten numbers.

• 제어로봇시스템학회:학술대회논문집
• /
• 2001.10a
• /
• pp.96.3-96
• /
• 2001

In this paper the self-organized and fuzzy inference used stochastic gradient ascent method is proposed. Fuzzy rule and fuzzy set increase as occasion demands autonomously according to the observation information. And two rules(or two fuzzy sets)becoming to be similar each other as progress of learning are unified. This unification causes the reduction of a number of parameters and learning time. Using fuzzy inference and making a rule with an appropriate state division, our proposed method makes it possible to construct a robust reinforcement learning system.

• Journal of Communications and Networks
• /
• v.18 no.3
• /
• pp.411-419
• /
• 2016

This article investigates the problem of distributed channel selection in opportunistic spectrum access networks from a perspective of interference minimization. The traditional physical (PHY)-layer interference model is for information theoretic analysis. When practical multiple access mechanisms are considered, the recently developed binary medium access control (MAC)-layer interference model in the previous work is more useful, in which the experienced interference of a user is defined as the number of competing users. However, the binary model is not accurate in mathematics analysis with poor achievable performance. Therefore, we propose a real-valued one called stochastic MAC-layer interference model, where the utility of a player is defined as a function of the aggregate weight of the stochastic interference of competing neighbors. Then, the distributed channel selection problem in the stochastic MAC-layer interference model is formulated as a weighted stochastic MAC-layer interference minimization game and we proved that the game is an exact potential game which exists one pure strategy Nash equilibrium point at least. By using the proposed stochastic learning-automata based uncoupled algorithm with heterogeneous learning parameter (SLA-H), we can achieve suboptimal convergence averagely and this result can be verified in the simulation. Moreover, the simulated results also prove that the proposed stochastic model can achieve higher throughput performance and faster convergence behavior than the binary one.

• 제어로봇시스템학회:학술대회논문집
• /
• 1996.10b
• /
• pp.1173-1176
• /
• 1996

Multilayer feedforward networks may be applied to identify the deterministic relationship between input and output data. When the results from the network require a high level of assurance, considering of the stochastic relationship between the data may be very important. The variance is one of the useful parameters to represent the stochastic relationship. This paper presents a new algorithm for a multilayer feedforward network to learn the variance of dispersed data without preliminary calculation of variance. In this paper, the network with this learning algorithm is named as a variance learning neural network(VALEAN). Computer simulation examples are utilized for the demonstration and the evaluation of VALEAN.

• Journal of the Korean Society for Precision Engineering
• /
• v.14 no.6
• /
• pp.121-127
• /
• 1997

Multilayer feedforward networks may be applied to identify the deterministic relationship between input and output data. When the results from the network require a high level of assurance, consideration of the stochastic relationship between the input and output data may be very important. Variance is one of the effective parameters to deal with the stochastic relationship. This paper presents a new algroithm for a multilayer feedforward network to learn the variance of dispersed data without preliminary calculation of variance. In this paper, the network with this learning algorithm is named as a variance learning neural network(VALEAN). Computer simulation examples are utilized for the demonstration and the evaluation of VALEAN.