• Journal of KIISE:Computing Practices and Letters
• /
• v.14 no.5
• /
• pp.502-506
• /
• 2008

The recently proposed "Potential-based" reinforcement learning (RL) method made it possible to combine multiple learnings and expert advices as supervised knowledge within an RL framework. The effectiveness of the approach has been established by a theoretical convergence guarantee to an optimal policy. In this paper, the potential-based RL method is applied to a dynamic channel assignment (DCA) problem in a cellular networks. It is empirically shown that the potential-based RL assigns channels more efficiently than fixed channel assignment, Maxavail, and Q-learning-based DCA, and it converges to an optimal policy more rapidly than other RL algorithms, SARSA(0) and PRQ-learning.

• Journal of Information Processing Systems
• /
• v.16 no.5
• /
• pp.1048-1063
• /
• 2020

In this study, we propose a reinforcement learning agent to control the data transmission rates of nodes in carrier sensing multiple access with collision avoidance (CSMA/CA)-based wireless networks. We design a reinforcement learning (RL) agent, based on Q-learning. The agent learns the environment using the timeout events of packets, which are locally available in data sending nodes. The agent selects actions to control the data transmission rates of nodes that adjust the modulation and coding scheme (MCS) levels of the data packets to utilize the available bandwidth in dynamically changing channel conditions effectively. We use the ns3-gym framework to simulate RL and investigate the effects of the parameters of Q-learning on the performance of the RL agent. The simulation results indicate that the proposed RL agent adequately adjusts the MCS levels according to the changes in the network, and achieves a high throughput comparable to those of the existing data transmission rate adaptation schemes such as Minstrel.

• Journal of KIISE:Computing Practices and Letters
• /
• v.15 no.12
• /
• pp.978-982
• /
• 2009

This paper proposes a potential-based reinforcement learning, called "RLs-CBDT", which combines multiple RL agents and case-base decision theory designed for decision making in uncertain environment as an expert knowledge in RL. We empirically show that RLs-CBDT converges to an optimal policy faster than pre-existing RL algorithms through a Tetris experiment.

• International Journal of Internet, Broadcasting and Communication
• /
• v.16 no.1
• /
• pp.63-71
• /
• 2024

Deep reinforcement learning (RL) significantly influences autonomous vehicle development by optimizing decision-making and adaptation to complex driving environments through simulation-based training. In deep RL, an activation function is used, and various activation functions have been proposed, but their performance varies greatly depending on the application environment. Therefore, finding the optimal activation function according to the environment is important for effective learning. In this paper, we analyzed nine commonly used activation functions for RL to compare and evaluate which activation function is most effective when using deep RL for autonomous vehicles to learn highway merging. To do this, we built a performance evaluation environment and compared the average reward of each activation function. The results showed that the highest reward was achieved using Mish, and the lowest using SELU. The difference in reward between the two activation functions was 10.3%.

• Nuclear Engineering and Technology
• /
• v.56 no.6
• /
• pp.1959-1974
• /
• 2024

Nuclear energy plays a crucial role in energy supply in the 21st century, and more and more Nuclear Power Plants (NPPs) will be in operation to contribute to the development of human society. However, as a typical complex system engineering, the operation and development of NPPs require efficient and stable control methods to ensure the safety and efficiency of nuclear power generation. Reinforcement learning (RL) aims at learning optimal control policies via maximizing discounted long-term rewards. The reward-oriented learning paradigm has witnessed remarkable success in many complex systems, such as wind power systems, electric power systems, coal fire power plants, robotics, etc. In this work, we try to present a systematic review of the applications of RL on these complex systems, from which we believe NPPs can borrow experience and insights. We then conduct a block-by-block investigation on the application scenarios of specific tasks in NPPs and carried out algorithmic research for different situations such as power startup, collaborative control, and emergency handling. Moreover, we discuss the possibilities of further application of RL methods on NPPs and detail the challenges when applying RL methods on NPPs. We hope this work can boost the realization of intelligent NPPs, and contribute to more and more research on how to better integrate RL algorithms into NPPs.

• International Journal of Computer Science & Network Security
• /
• v.21 no.9
• /
• pp.125-131
• /
• 2021

Detecting fraudulent insurance claims is difficult due to small and unbalanced data. Some research has been carried out to better cope with various types of fraudulent claims. Nowadays, technology for detecting fraudulent insurance claims has been increasingly utilized in insurance and technology fields, thanks to the use of artificial intelligence (AI) methods in addition to traditional statistical detection and rule-based methods. This study obtained meaningful results for a fraudulent insurance claim detection model based on machine learning (ML) and deep learning (DL) technologies, using fraudulent insurance claim data from previous research. In our search for a method to enhance the detection of fraudulent insurance claims, we investigated the reinforcement learning (RL) method. We examined how we could apply the RL method to the detection of fraudulent insurance claims. There are limited previous cases of applying the RL method. Thus, we first had to define the RL essential elements based on previous research on detecting anomalies. We applied the deep Q-network (DQN) and double deep Q-network (DDQN) in the learning fraudulent insurance claim detection model. By doing so, we confirmed that our model demonstrated better performance than previous machine learning models.

• International Journal of Computer Science & Network Security
• /
• v.24 no.4
• /
• pp.87-106
• /
• 2024

Vehicle-to-Home (V2H) and Home Centralized Photovoltaic (HCPV) systems can address various energy storage issues and enhance demand response programs. Renewable energy, such as solar energy and wind turbines, address the energy gap. However, no energy management system is currently available to regulate the uncertainty of renewable energy sources, electric vehicles, and appliance consumption within a smart microgrid. Therefore, this study investigated the impact of solar photovoltaic (PV) panels, electric vehicles, and Micro-Grid (MG) storage on maximum solar radiation hours. Several Deep Learning (DL) algorithms were applied to account for the uncertainty. Moreover, a Reinforcement Learning HCPV (RL-HCPV) algorithm was created for efficient real-time energy scheduling decisions. The proposed algorithm managed the energy demand between PV solar energy generation and vehicle energy storage. RL-HCPV was modeled according to several constraints to meet household electricity demands in sunny and cloudy weather. Simulations demonstrated how the proposed RL-HCPV system could efficiently handle the demand response and how V2H can help to smooth the appliance load profile and reduce power consumption costs with sustainable power generation. The results demonstrated the advantages of utilizing RL and V2H as potential storage technology for smart buildings.

• Journal of Internet Computing and Services
• /
• v.22 no.4
• /
• pp.13-28
• /
• 2021

Reinforcement Learning (RL) is a machine learning algorithm that repeat the closed-loop process that agents perform actions specified by the policy, the action is evaluated with a reward function, and the policy gets updated accordingly. The key benefit of RL is the ability to optimze the policy with action evaluation. Hence, it can effectively be applied to developing advanced intelligent systems and autonomous systems. Conventional RL incoporates a single policy, a reward function, and relatively simple policy update, and hence its utilization was limited. In this paper, we propose an extended RL model that considers multiple instances of RL elements. We define a formal model of the key elements and their computing model of the extended RL. Then, we propose design methods for applying to system development. As a case stud of applying the proposed formal model and the design methods, we present the design and implementation of an advanced car navigator system that guides multiple cars to reaching their destinations efficiently.

• KIPS Transactions on Software and Data Engineering
• /
• v.10 no.7
• /
• pp.263-270
• /
• 2021

Reinforcement learning(RL) is the method to find an optimal policy through training. and it is one of popular methods for solving lifesaving and disaster response problems effectively. However, the conventional reinforcement learning method for disaster response utilizes either simple environment such as. grid and graph or a self-developed environment that are hard to verify the practical effectiveness. In this paper, we propose the design of a disaster response RL environment which utilizes the detailed property information of the disaster simulation in order to utilize the reinforcement learning method in the real world. For the RL environment, we design and build the reinforcement learning communication as well as the interface between the RL agent and the disaster simulation. Also, we apply the dimension reduction method for converting non-image feature vectors into image format which is effectively utilized with convolution layer to utilize the high-dimensional and detailed property of the disaster simulation. To verify the effectiveness of our proposed method, we conducted empirical evaluations and it shows that our proposed method outperformed conventional methods in the building fire damage.

• Journal of Advanced Information Technology and Convergence
• /
• v.9 no.1
• /
• pp.127-134
• /
• 2019

In terms of deep Reinforcement Learning (RL), exploration can be worked stochastically in the action of a state space. On the other hands, exploitation can be done the proportion of well generalization behaviors. The balance of exploration and exploitation is extremely important for better results. The randomly selected action with ε-greedy for exploration has been regarded as a de facto method. There is an alternative method to add noise parameters into a neural network for richer exploration. However, it is not easy to predict or detect over-fitting with the stochastically exploration in the perturbed neural network. Moreover, the well-trained agents in RL do not necessarily prevent or detect over-fitting in the neural network. Therefore, we suggest a novel design of a deep RL by the balance of the exploration with drop-out to reduce over-fitting in the perturbed neural networks.