• Proceedings of the Korea Water Resources Association Conference
• /
• 2006.05a
• /
• pp.354-359
• /
• 2006

The analysis of large-scale water resources systems is often complicated by the presence of multiple reservoirs and diversions, the uncertainty of unregulated inflows and demands, and conflicting objectives. Reinforcement learning is presented herein as a new approach to solving the challenging problem of stochastic optimization of multi-reservoir systems. The Q-Learning method, one of the reinforcement learning algorithms, is used for generating integrated monthly operation rules for the Keum River basin in Korea. The Q-Learning model is evaluated by comparing with implicit stochastic dynamic programming and sampling stochastic dynamic programming approaches. Evaluation of the stochastic basin-wide operational models considered several options relating to the choice of hydrologic state and discount factors as well as various stochastic dynamic programming models. The performance of Q-Learning model outperforms the other models in handling of uncertainty of inflows.

• Journal of Platform Technology
• /
• v.9 no.1
• /
• pp.3-14
• /
• 2021

Cognitive Radio network (CR) is a promising paradigm that helps the unlicensed user (Secondary User) to analyse the spectrum and coordinate the spectrum access to support the creation of common control channel (CCC). The cooperation of secondary users and broadcasting between them is done through transmitting messages in CCC. In case, if the control channels may get jammed and it may directly degrade the network's performance and under such scenario jammers will devastate the control channels. Hopping sequences may be one of the predominant approaches and it may be used to fight against this problem to confront jammer. The jamming attack can be alleviated using one of the game modelling approach and in this proposed scheme stochastic games has been analysed with more single users to provide the flexible control channels against intrusive attacks by mentioning the states of each player, strategies ,actions and players reward. The proposed work uses a modern player action and better strategic view on game theoretic modelling is stochastic game theory has been taken in to consideration and applied to prevent the jamming attack in CR network. The selection of decision is based on Q learning approach to mitigate the jamming nodes using the optimal MDP decision process

• Journal of Electrical Engineering and Technology
• /
• v.9 no.1
• /
• pp.80-89
• /
• 2014

A stochastic optimal power flow (S-OPF) model considering uncertainties of load and wind power is developed based on chance constrained programming (CCP). The difficulties in solving the model are the nonlinearity and probabilistic constraints. In this paper, a limit relaxation approach and an iterative learning control (ILC) method are implemented to solve the S-OPF model indirectly. The limit relaxation approach narrows the solution space by introducing regulatory factors, according to the relationship between the constraint equations and the optimization variables. The regulatory factors are designed by ILC method to ensure the optimality of final solution under a predefined confidence level. The optimization algorithm for S-OPF is completed based on the combination of limit relaxation and ILC and tested on the IEEE 14-bus system.

• Journal of Communications and Networks
• /
• v.18 no.3
• /
• pp.411-419
• /
• 2016

This article investigates the problem of distributed channel selection in opportunistic spectrum access networks from a perspective of interference minimization. The traditional physical (PHY)-layer interference model is for information theoretic analysis. When practical multiple access mechanisms are considered, the recently developed binary medium access control (MAC)-layer interference model in the previous work is more useful, in which the experienced interference of a user is defined as the number of competing users. However, the binary model is not accurate in mathematics analysis with poor achievable performance. Therefore, we propose a real-valued one called stochastic MAC-layer interference model, where the utility of a player is defined as a function of the aggregate weight of the stochastic interference of competing neighbors. Then, the distributed channel selection problem in the stochastic MAC-layer interference model is formulated as a weighted stochastic MAC-layer interference minimization game and we proved that the game is an exact potential game which exists one pure strategy Nash equilibrium point at least. By using the proposed stochastic learning-automata based uncoupled algorithm with heterogeneous learning parameter (SLA-H), we can achieve suboptimal convergence averagely and this result can be verified in the simulation. Moreover, the simulated results also prove that the proposed stochastic model can achieve higher throughput performance and faster convergence behavior than the binary one.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.11 no.12
• /
• pp.5820-5834
• /
• 2017

This paper investigates the channel selection problem with dynamic users and the asymmetric interference relation in distributed opportunistic spectrum access systems. Since users transmitting data are based on their traffic demands, they dynamically compete for the channel occupation. Moreover, the heterogeneous interference range leads to asymmetric interference relation. The dynamic users and asymmetric interference relation bring about new challenges such as dynamic random systems and poor fairness. In this article, we will focus on maximizing the tradeoff between the achievable utility and access cost of each user, formulate the channel selection problem as a directional graphical game and prove it as an exact potential game presenting at least one pure Nash equilibrium point. We show that the best NE point maximizes both the personal and system utility, and employ the stochastic learning approach algorithm for achieving the best NE point. Simulation results show that the algorithm converges, presents near-optimal performance and good fairness, and the directional graphical model improves the systems throughput performance in different asymmetric level systems.

• Journal of Institute of Control, Robotics and Systems
• /
• v.17 no.8
• /
• pp.777-782
• /
• 2011

Neural network technique is widely employed in the fields of signal processing, control systems, pattern recognition, etc. Learning of neural networks is an important procedure to accomplish dynamic system modeling. This paper presents a novel learning approach for differential neural network models based on the Kalman-Bucy filter theory. We construct an augmented state vector including original neural state and parameter vectors and derive a state estimation rule avoiding gradient function terms which involve to the conventional neural learning methods such as a back-propagation approach. We carry out numerical simulation to evaluate the proposed learning approach in nonlinear system modeling. By comparing to the well-known back-propagation approach and Kalman-Bucy filtering, its superiority is additionally proved under stochastic system environments.

• Journal of Communications and Networks
• /
• v.12 no.6
• /
• pp.600-604
• /
• 2010

A method of width adaptation in the radial basis function network (RBFN) using stochastic gradient (SG) algorithm is introduced. Using Taylor's expansion of error signal and differentiating the error with respect to the step-size, the optimal time-varying step-size of the width in RBFN is derived. The proposed approach to adjusting widths in RBFN achieves superior learning speed and the steady-state mean square error (MSE) performance in nonlinear channel environment. The proposed method has shown enhanced steady-state MSE performance by more than 3 dB in both nonlinear channel environments. The results confirm that controlling over step-size of the width in RBFN by the proposed algorithm can be an effective approach to enhancement of convergence speed and the steady-state value of MSE.

• International Journal of Contents
• /
• v.12 no.4
• /
• pp.31-44
• /
• 2016

Knowledge creation and management are regarded as critical success factors for an organization's survival in the knowledge era. As a process of knowledge acquisition and sharing, organizational learning mechanisms (OLMs) guide the learning function of organizations represented by its different learning activities. We examined a variety of learning processes that constitute OLMs. In this study, we aimed to capture the process and framework of OLMs and knowledge sharing and acquisition. Factors facilitating OLMs were investigated at three levels: individual, group, and organizational. The concept of an OLM has received some attention in the field of organizational learning, however, the relationship among the factors generating OLMs has not been empirically tested. As part of the ongoing discussion, we attempted a systemic approach for OLMs. OLMs can be represented by factors that are inherent to the organization's system; therefore, prior to empirically testing the OLM generating factor(s), evaluation of its organizational integration is required to determine effective treatment of each factor. Thus, we developed a framework to manage knowledge and proposed a method to numerically evaluate factors influencing the OLMs. Specifically, composite importance (CI) of the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method was applied to explore the interaction effect of these factors based on systemic approach. The augmented matrix thus generated is expected to serve as a stochastic matrix of an absorbing Markov chain.

• Journal of the Institute of Convergence Signal Processing
• /
• v.12 no.3
• /
• pp.212-216
• /
• 2011

Fault detection technique for photovoltaic power systems is significant to dramatically reduce economic damage in industrial fields. This paper presents a novel fault detection approach using Fourier neural networks and stochastic decision making strategy for photovoltaic systems. We achieve neural modeling to represent its nonlinear dynamic behaviors through a gradient descent based learning algorithm. Next, a general likelihood ratio test (GLRT) is derived for constructing a decision malling mechanism in stochastic fault detection. A testbed of photovoltaic power systems is established to conduct real-time experiments in which the DC power line communication (DPLC) technique is employed to transfer data sets measured from the photovoltaic panels to PC systems. We demonstrate our proposed fault detection methodology is reliable and practicable over this real-time experiment.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.25 no.4
• /
• pp.319-326
• /
• 2015

Recently in the fields o f stochastic optimal control ( SOC) and reinforcemnet l earning (RL), there have been a great deal of research efforts for the problem of finding data-based sub-optimal control policies. The conventional theory for finding optimal controllers via the value-function-based dynamic programming was established for solving the stochastic optimal control problems with solid theoretical background. However, they can be successfully applied only to extremely simple cases. Hence, the data-based modern approach, which tries to find sub-optimal solutions utilizing relevant data such as the state-transition and reward signals instead of rigorous mathematical analyses, is particularly attractive to practical applications. In this paper, we consider a couple of methods combining the modern SOC strategies and approximate inference together with machine-learning-based data treatment methods. Also, we apply the resultant methods to a variety of application domains including financial engineering, and observe their performance.