• The Transactions of The Korean Institute of Electrical Engineers
• /
• v.58 no.7
• /
• pp.1411-1417
• /
• 2009

In this paper, we consider the design of $H_{\infty}$ control of linear discrete-time systems having no mathematical model. The basic approach is to use Q-learning which is a reinforcement learning method based on actor-critic structure. The model-free control design is to use not the mathematical model of the system but the informations on states and inputs. As a result, the derived iterative algorithm is expressed as linear matrix inequalities(LMI) of measured data from system states and inputs. It is shown that, for a sufficiently rich enough disturbance, this algorithm converges to the standard $H_{\infty}$ control solution obtained using the exact system model. A simple numerical example is given to show the usefulness of our result on practical application.

• Smart Structures and Systems
• /
• v.32 no.4
• /
• pp.219-233
• /
• 2023

Crack width is an important indicator to evaluate the health condition of the concrete structure. The crack width is measured by manual using crack width gauge commonly, which is time-consuming and laborious. In this paper, we have proposed a fast and simplified crack width quantification method via deep Q learning and geometric calculation. Firstly, the crack edge is extracted by using U-Net network and edge detection operator. Then, the intelligent decision of is made by the deep Q learning model. Further, the geometric calculation method based on endpoint and curvature extreme point detection is proposed. Finally, a case study is carried out to demonstrate the effectiveness of the proposed method, achieving high precision in the real crack width quantification.

• Proceedings of the IEEK Conference
• /
• 2009.05a
• /
• pp.118-120
• /
• 2009

In this paper, we present the Q-learning method for adaptive traffic signal control on the basis of multi-agent technology. The structure is composed of sixphase agents and one intersection agent. Wireless communication network provides the possibility of the cooperation of agents. As one kind of reinforcement learning, Q-learning is adopted as the algorithm of the control mechanism, which can acquire optical control strategies from delayed reward; furthermore, we adopt dynamic learning method instead of static method, which is more practical. Simulation result indicates that it is more effective than traditional signal system.

• 제어로봇시스템학회:학술대회논문집
• /
• 2001.10a
• /
• pp.173.5-173
• /
• 2001

Q-Learning is a most widely used reinforcement learning, which addresses the question of how an autonomous agent can learn to choose optimal actions to achieve its goal about any one problem. Q-Learning can acquire optimal control strategies from delayed rewards, even when the agent has no prior knowledge of the effects of its action in the environment. If agent has an ability using previous knowledge, then it is expected that the agent can speed up learning by interacting with environment. We present a novel reinforcement learning method using domain knowledge, which is represented by problem-independent features and their classifiers. Here neural network are implied as knowledge classifiers. To show that an agent using domain knowledge can have better performance than the agent with standard Q-Learner. Computer simulations are ...

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.3
• /
• pp.853-873
• /
• 2021

In recent years, deep reinforcement learning (DRL) models are enjoying great interest as their success in a variety of challenging tasks. Deep Q-Network (DQN) is a widely used deep reinforcement learning model, which trains an intelligent agent that executes optimal actions while interacting with an environment. This model is well known for its ability to surpass skilled human players across many Atari 2600 games. Although DQN has achieved excellent performance in practice, there lacks a clear understanding of why the model works. In this paper, we present a visual analytics system for understanding deep Q-network in a non-blind matter. Based on the stored data generated from the training and testing process, four coordinated views are designed to expose the internal execution mechanism of DQN from different perspectives. We report the system performance and demonstrate its effectiveness through two case studies. By using our system, users can learn the relationship between states and Q-values, the function of convolutional layers, the strategies learned by DQN and the rationality of decisions made by the agent.

• The Transactions of the Korea Information Processing Society
• /
• v.6 no.5
• /
• pp.1303-1311
• /
• 1999

In order to learn in dynamic environments, reinforcement learning algorithms like Q-learning, TD(0)-learning, TD(λ)-learning have been proposed. however, most of them have a drawback of very slow learning because the reinforcement value is given when they reach their goal state. In this thesis, we have proposed a reinforcement learning method that can approximate fast to the goal state in maze environments. The proposed reinforcement learning method is separated into global learning and local learning, and then it executes learning. Global learning is a learning that uses the replacing eligibility trace method to search the goal state. In local learning, it propagates the goal state value that has been searched through global learning to neighboring sates, and then searches goal state in neighboring states. we can show through experiments that the reinforcement learning method proposed in this thesis can find out an optimal solution faster than other reinforcement learning methods like Q-learning, TD(o)learning and TD(λ)-learning.

• The Journal of Korea Robotics Society
• /
• v.17 no.2
• /
• pp.221-229
• /
• 2022

Sliding mode control (SMC) is a robust control method to control a robot arm with nonlinear properties. A high switching gain of SMC causes chattering problems, although the SMC allows the adequate control performance by giving high switching gain, without the exact robot model containing nonlinear and uncertainty terms. In order to solve this problem, SMC with sliding perturbation observer (SMCSPO) has been researched, where the method can reduce the chattering by compensating the perturbation, which is estimated by the observer, and then choosing a lower switching control gain of SMC. However, optimal gain tuning is necessary to get a better tracking performance and reducing a chattering. This paper proposes a method that the Q-learning automatically tunes the control gains of SMCSPO with an iterative operation. In this tuning method, the rewards of reinforcement learning (RL) are set minus tracking errors of states, and the action of RL is a change of control gain to maximize rewards whenever the iteration number of movements increases. The simple motion test for a 7-DOF robot arm was simulated in MATLAB program to prove this RL tuning algorithm. The simulation showed that this method can automatically tune the control gains for SMCSPO.

• Proceedings of the IEEK Conference
• /
• 2003.07d
• /
• pp.1299-1302
• /
• 2003

In this paper, we propose new method which can provide user the path to the target place efficiently. It stores the state of roads to target place as the form of Q-table and finds the proper path using Q-table.0-table is updated by the information about real traffic which is reported by users. This method can provides the proper path, using less storage and less computation time than the conventional method which stores entire road traffic information and finds the path by graph search algorithm.

• Journal of Internet of Things and Convergence
• /
• v.8 no.4
• /
• pp.1-7
• /
• 2022

As the number of IoT devices increases, power management of the cluster head, which acts as a gateway between the cluster and sink nodes in the IoT network, becomes crucial. Particularly when the cluster head is a mobile wireless terminal, the power consumption of the IoT network must be minimized over its lifetime. In addition, the delay of information transmission in the IoT network is one of the primary metrics for rapid information collecting in the IoT network. In this paper, we propose a low-power buffer management algorithm that takes into account the information transmission delay in an IoT network. By forwarding or skipping received packets utilizing deep Q learning employed in deep reinforcement learning methods, the suggested method is able to reduce power consumption while decreasing transmission delay level. The proposed approach is demonstrated to reduce power consumption and to improve delay relative to the existing buffer management technique used as a comparison in slotted ALOHA protocol.

• 제어로봇시스템학회:학술대회논문집
• /
• 2001.10a
• /
• pp.24.2-24
• /
• 2001

In this paper, the authors present a row dynamic route planning by Q-learning. The proposed algorithm is executed in a cellular automation based traffic simulator, which is also newly created. In Vehicle Information and Communication System(VICS), which is an active field of Intelligent Transport System(ITS), information of traffic congestion is sent to each vehicle at real time. However, a centralized navigation system is not realistic to guide millions of vehicles in a megalopolis. Autonomous distributed systems should be more flexible and scalable, and also have a chance to focus on each vehicles demand. In such systems, each vehicle can search an own optimal route. We employ Q-learning of the reinforcement learning method to search an optimal or sub-optimal route, in which route drivers can avoid traffic congestions. We find some applications of the reinforcement learning in the "static" environment, but there are ...