• Proceedings of the IEEK Conference
• /
• 2009.05a
• /
• pp.94-96
• /
• 2009

In this paper, we present the Q-learning method for adaptive traffic signal control on the basis of multi-agent technology. The structure is composed of sixphase agents and one intersection agent. Wireless communication network provides the possibility of the cooperation of agents. As one kind of reinforcement learning, Q-learning is adopted as the algorithm of the control mechanism, which can acquire optical control strategies from delayed reward; furthermore, we adopt dynamic learning method instead of static method, which is more practical. Simulation result indicates that it is more effective than traditional signal system.

• Proceedings of the IEEK Conference
• /
• 2003.07a
• /
• pp.69-72
• /
• 2003

This paper presents a traffic control agent that can perform the dynamic resource allocation by controlling traffic flows on a DiffServ network. In addition, this paper presents a router that can support DiffServ on Linux to support selective QoS in IP network environment. To implement a method for selective traffic transmission based on priority on a DiffServ router, this paper changes the queuing discipline in Linux, and presents the traffic control agent so that it can efficiently control routers, efficiently allocates network resources according to service requests, and relocate resources in response to state changes of the network.

• Journal of Information Processing Systems
• /
• v.16 no.5
• /
• pp.1048-1063
• /
• 2020

In this study, we propose a reinforcement learning agent to control the data transmission rates of nodes in carrier sensing multiple access with collision avoidance (CSMA/CA)-based wireless networks. We design a reinforcement learning (RL) agent, based on Q-learning. The agent learns the environment using the timeout events of packets, which are locally available in data sending nodes. The agent selects actions to control the data transmission rates of nodes that adjust the modulation and coding scheme (MCS) levels of the data packets to utilize the available bandwidth in dynamically changing channel conditions effectively. We use the ns3-gym framework to simulate RL and investigate the effects of the parameters of Q-learning on the performance of the RL agent. The simulation results indicate that the proposed RL agent adequately adjusts the MCS levels according to the changes in the network, and achieves a high throughput comparable to those of the existing data transmission rate adaptation schemes such as Minstrel.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.7 no.10
• /
• pp.2376-2394
• /
• 2013

In order to ensure both of the whole system capacity and users QoS requirements in heterogeneous wireless networks, admission control mechanism should be well designed. In this paper, Multi-agent Q-learning based Admission Control Mechanism (MQACM) is proposed to handle new and handoff call access problems appropriately. MQACM obtains the optimal decision policy by using an improved form of single-agent Q-learning method, Multi-agent Q-learning (MQ) method. MQ method is creatively introduced to solve the admission control problem in heterogeneous wireless networks in this paper. In addition, different priorities are allocated to multiple services aiming to make MQACM perform even well in congested network scenarios. It can be observed from both analysis and simulation results that our proposed method not only outperforms existing schemes with enhanced call blocking probability and handoff dropping probability performance, but also has better network universality and stability than other schemes.

• Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
• /
• v.25 no.8
• /
• pp.70-81
• /
• 2011

In this paper, a multi-agent control system for DC-coupled photovoltaic (PV), fuel cell (FC), ultracapacitor(UC) and battery hybrid power system is studied for commercial buildings & apartment buildings microgrid. In this proposed system, the PV system provides electric energy to the electrolyzer to produce hydrogen for future use and transfer to the load side, if possible. Whenever the PV system cannot completely meet load demands, the FC system provides power to meet the remaining load. A multi-agent system based-power management and control algorithm is proposed for the hybrid power system by taking into account the characteristics of each power source. The main works of this paper are hybridization of alternate energy sources with FC systems using long and short storage strategies to build the multi-agent control system with pragmatic design, and a dynamic model proposed for a PV/FC/UC/battery bank hybrid power generation system. A dynamic simulation model for the hybrid power system has been developed using Matlab/Simulink, SimPowerSystems and Stateflow. Simulation results are also presented to demonstrate the effectiveness of the proposed multi-agent control and management system for building microgrid.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.14 no.4
• /
• pp.457-467
• /
• 2004

In this paper, a robot is considered as an agent, a structure of robot is presented which consisted by multi-subagents and they have diverse capacity such as perception, intelligence, action etc., required for robot. Also, subagents are consisted by micro-agent($\mu$agent) charged for elementary action required. The structure of robot control have two sub-agents, the one is behavior based reactive controller and action selection sub agent, and action selection sub-agent select a action based on the high label action and high performance, and which have a learning mechanism based on the reinforcement learning. For presented robot structure, it is easy to give intelligence to each element of action and a new approach of multi robot control. Presented robot is simulated for two goals: chaotic exploration and obstacle avoidance, and fabricated by using 8bit microcontroller, and experimented.

• 제어로봇시스템학회:학술대회논문집
• /
• 2005.06a
• /
• pp.433-437
• /
• 2005

The goal of our research is to develop a formal modeling methodology for supervisory and controlling systems that have artificially intelligent features. This approach is agent-based and central to the development of the model of mobility agent considering reactivity for real-time purpose and deliberation for optimal realization and safe-fail problems for critical systems like Intelligent Transportation Systems by high-level Petri net. By using nets within nets we investigate the concurrency of the system and the agent in one model without losing the needed abstraction, and synchronous channels are introduced to denote the coordination and communication. Finally an example is demonstrated.

• Journal of Korea Game Society
• /
• v.2 no.2
• /
• pp.37-45
• /
• 2002

Computer games could be regarded as simulation of the real world. Control problems of software agents have long been studied in the field of Artificial Intelligence (AI), resulting in giving a birth to the behavior-based approach. three main approaches might be categorized out of the history of AI study. First, Cognitivists propose that intelligence could be represented and manipulated in terms of symbols. Second, Connectionists claim that symbols could not be isolated but they are embedded in the body structure. Third, the behavior-based approach is an approach to AI which suggests that intelligence is dynamic property that exists nowhere but emerges in the relationship of an agent and the world including observers while the agent performs behavior. This paper explains and compares the three approaches to AI, then discusses the plausibility of the behavior-based approach and problems. Finally, this paper proposes application of behavior-based approach to computer games in terms of agent control.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.18 no.1
• /
• pp.78-85
• /
• 2008

In this paper, we propose the extension of cooperation protocol which realizes the autonomous control of QoS and show its effectiveness with implementation and experiments of our agent-based videoconference system. The autonomous QoS control in the agent-based videoconference system means the framework of adjusting its functions/abilities by itself to cope with the changes of user requirements and system resource situations. But, the original cooperation protocol used by the agent-based videoconference system is not enough to perform cooperation closely to use the limited resources effectively. In our proposal, the cooperation protocol is offended in the following points: (1) introducing a cooperation states, (2) extension of performatives, (3) introducing compromise levels. We embedded the proposed protocol to the agent-based videoconference system and experimented the behavior of the system. According to the experiments on this prototype system, we confirmed that the flexibility of the system is improved.

• Journal of Institute of Control, Robotics and Systems
• /
• v.11 no.12
• /
• pp.1039-1044
• /
• 2005

This paper presents a robotic agent design and application in the ubiquitous intelligent space. We set up an experimental environment with Bluetooth host, Bluetooth client, furniture and home appliance, and robotic agents. First, the agents basically performed patrol guard to detect unexpected penetration, and to keep home safely from gas-leakage, electric leakage, and so on. They were out to patrol fur a robbery while navigating in a living room and a private room. In this task, we used an area-based action making and a hexagon-based Q-learning to control the agents. Second, the agents communicate with Bluetooth host device to access and control a home appliance. The Bluetooth host offers a manual control to person by inquiring a client robot when one would like to check some place especially. In this exercise, we organize asynchronous connection less (ACL) between the host and the client robots and control the robot maneuver by Bluetooth host controller interface (HCI).