• The KIPS Transactions:PartB
• /
• v.9B no.2
• /
• pp.139-146
• /
• 2002

The robot soccer simulation game is a dynamic multi-agent environment. In this paper we suggest a new reinforcement learning approach to each agent's dynamic positioning in such dynamic environment. Reinforcement learning is the machine learning in which an agent learns from indirect, delayed reward an optimal policy to choose sequences of actions that produce the greatest cumulative reward. Therefore the reinforcement learning is different from supervised learning in the sense that there is no presentation of input-output pairs as training examples. Furthermore, model-free reinforcement learning algorithms like Q-learning do not require defining or learning any models of the surrounding environment. Nevertheless these algorithms can learn the optimal policy if the agent can visit every state-action pair infinitely. However, the biggest problem of monolithic reinforcement learning is that its straightforward applications do not successfully scale up to more complex environments due to the intractable large space of states. In order to address this problem, we suggest Adaptive Mediation-based Modular Q-Learning (AMMQL) as an improvement of the existing Modular Q-Learning (MQL). While simple modular Q-learning combines the results from each learning module in a fixed way, AMMQL combines them in a more flexible way by assigning different weight to each module according to its contribution to rewards. Therefore in addition to resolving the problem of large state space effectively, AMMQL can show higher adaptability to environmental changes than pure MQL. In this paper we use the AMMQL algorithn as a learning method for dynamic positioning of the robot soccer agent, and implement a robot soccer agent system called Cogitoniks.

• Journal of the Korea Society of Computer and Information
• /
• v.26 no.3
• /
• pp.9-17
• /
• 2021

In this paper, an autonomous multi-agent path planning using reinforcement learning for monitoring of infrastructures and resources in a computationally distributed system was proposed. Reinforcement-learning-based multi-agent exploratory system in a distributed node enable to evaluate a cumulative reward every action and to provide the optimized knowledge for next available action repeatedly by learning process according to a learning policy. Here, the proposed methods were presented by (a) approach of dynamics-based motion constraints multi-agent path-planning to reduce smaller agent steps toward the given destination(goal), where these agents are able to geographically explore on the environment with initial random-trials versus optimal-trials, (b) approach using agent sub-goal selection to provide more efficient agent exploration(path-planning) to reach the final destination(goal), and (c) approach of reinforcement learning schemes by using the proposed autonomous and asynchronous triggering of agent exploratory phases.

• Proceedings of the Korea Inteligent Information System Society Conference
• /
• 2001.01a
• /
• pp.65-70
• /
• 2001

We consider a hyper-redundant system that consists of many uniform units. The hyper-redundant system has many degrees of freedom and it can accomplish various tasks. Applysing the reinforcement learning to the hyper-redundant system is very attractive because it is possible to acquire various behaviors for various tasks automatically. In this paper we present a new reinforcement learning algorithm "Q-learning with propagation of motion". The algorithm is designed for the multi-agent systems that have strong connections. The proposed algorithm needs only one small Q-table even for a large scale system. So using the proposed algorithm, it is possible for the hyper-redundant system to learn the effective behavior. In this algorithm, only one leader agent learns the own behavior using its local information and the motion of the leader is propagated to another agents with time delay. The reward of the leader agent is given by using the whole system information. And the effective behavior of the leader is learned and the effective behavior of the system is acquired. We apply the proposed algorithm to a snake-like hyper-redundant robot. The necessary condition of the system to be Markov decision process is discussed. And the computer simulation of learning the locomotion is demonstrated. From the simulation results we find that the task of the locomotion of the robot to the desired point is learned and the winding motion is acquired. We can conclude that our proposed system and our analysis of the condition, that the system is Markov decision process, is valid.

• The Journal of Korea Robotics Society
• /
• v.12 no.3
• /
• pp.339-349
• /
• 2017

In this paper, we present a finite-time sliding mode control (FSMC) with an integral finite-time sliding surface for applying the concept of graph theory to a distributed wheeled mobile robot (WMR) system. The kinematic and dynamic property of the WMR system are considered simultaneously to design a finite-time sliding mode controller. Next, consensus and formation control laws for distributed WMR systems are derived by using the graph theory. The kinematic and dynamic controllers are applied simultaneously to compensate the dynamic effect of the WMR system. Compared to the conventional sliding mode control (SMC), fast convergence is assured and the finite-time performance index is derived using extended Lyapunov function with adaptive law to describe the uncertainty. Numerical simulation results of formation control for WMR systems shows the efficacy of the proposed controller.

• Proceedings of the KIEE Conference
• /
• 2005.07d
• /
• pp.2780-2782
• /
• 2005

In this paper, we present a mobile robot localization solution by using a Ubiquitous Vision System (UVS). The collective information gathered by multiple cameras that are strategically placed has many advantages. For example, aggregation of information from multiple viewpoints reduces the uncertainty about the robots' positions. We construct UVS as a multi-agent system by regarding each vision sensor as one vision agent (VA). Each VA performs target segmentation by color and motion information as well as visual tracking for multiple objects. Our modified identified contractnet (ICN) protocol is used for communication between VAs to coordinate multitask. This protocol raises scalability and modularity of thesystem because of independent number of VAs and needless calibration. Furthermore, the handover between VAs by using ICN is seamless. Experimental results show the robustness of the solution with respect to a widespread area. The performance in indoor environments shows the feasibility of the proposed solution in real-time.

• The Transactions of the Korea Information Processing Society
• /
• v.13 no.8
• /
• pp.356-366
• /
• 2024

Users of intelligent robots require access to the status data of the robots for various reasons. The status data of intelligent robots can be generated by combining the status data of the functional agents that constitute the intelligent robot. However, existing intelligent robot systems do not generate the necessary agent status data for creating the status data of intelligent service robots, or they generate it in different ways, making it impossible to collect this information in a uniform manner. Furthermore, these systems have limitations such as collecting the same information redundantly if multiple users request it and only using a single method of communication to deliver robot information, thereby failing to offer the communication methods desired by users. This paper proposes a multi-agent monitoring system for intelligent service robots designed to overcome these limitations. This monitoring system generates status data in response to the actions performed by functional agents, thereby allowing for the unified generation and collection of agent status data. Additionally, the monitoring system resolves data redundancy issues by collecting the necessary data just once, in accordance with user monitoring demands, and delivers status data through a proxy that supports the preferred communication methods of users, thereby providing compatibility with various communication methods. Through experiments, we have verified that this monitoring system can deliver the status data of intelligent robots to multiple users using various communication methods.

• Journal of Korean Society for Quality Management
• /
• v.31 no.3
• /
• pp.114-125
• /
• 2003

In this paper, we deal with the problem of modeling of agent-based robot manufacturing cell. Its role is becoming increasingly important in automated manufacturing systems. For Object Oriented & Queueing Petri Nets (OO&QPNs), an extended formalism for the combined quantitative and qualitative analysis of different systems is used for structure and performance analysis of mobile robotic cell. In the case study, the OO&QPN model of a mobile robotic cell is represented and analyzed, considering multi-class parts, non-preemptive priority and alternative routing. Finally, the comparison of performance values between Shortest Process Time (SPT) rule and First Come First Serve (FCFS) rule is suggested. In general, SPT rule is most suitable for parts that have shorter processing time than others.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.16 no.2
• /
• pp.95-103
• /
• 2016

In a dynamic environment autonomous robots often encounter unexpected situations that the robots have to deal with in order to continue proceeding their mission. We propose an adaptive goal-based model that allows cyber-physical systems (CPS) to update their environmental model and helps them analyze for attainment of their goals from current state using the updated environmental model and its capabilities. Information exchange approach utilizes Human-Agent-Robot-Machine-Sensor (HARMS) model to exchange messages between CPS. Model validation method uses NuSMV, which is one of Model Checking tools, to check whether the system can continue its mission toward the goal in the given environment. We explain a practical set up of the model in a situation in which homogeneous robots that has the same capability work in the same environment.

• 제어로봇시스템학회:학술대회논문집
• /
• 1997.10a
• /
• pp.621-624
• /
• 1997

This paper presents the design procedure for soccer-playing rovots based on the centralized approach. Using a fast vision system, we obtain the configuration of each robot and then the host computer computes the desired motion and commands each robot directly via RF communication. The robot soccer game has a lot of problems such as obstacle avoidance, coordination between robots, dribbling the ball, and so on. To implement such motions, we think that the centralized approach seems to be more powerful than the distributed approach. We describe the technical tips for developing the robots in detail here and explain our strategy for getting the scores.

• Proceedings of the KIEE Conference
• /
• 2008.04a
• /
• pp.246-247
• /
• 2008

본 논문에서는 군집간 제어를 위한 이동용 로봇제작과 로봇의 좌표인식, 방향을 찾기 위하여 GPS(Global Positioning System)수신 모듈과 Bluetooth송 수신기를 사용하였다. 실험에 쓰인 모든 이동용 로봇에 GPS수신기와 Bluetooth 송 수신기를 장착하고, GPS 수신기로부터 받은 blaster-이동로봇의 위치좌표를 Bluetooth통신 영역 내에 있는 모든 이동로봇에게 보내면 각 Slave-이동로봇은 Master이동로봇으로부터 받은 위치 좌표를 이용하여 blaster-이동로봇을 중심으로 상대적인 위치, 거리 유지, 진행방향을 계산한다. Master-이동로봇과 Slave-이동로봇 간의 실시간 통신과 일정거리를 유지함으로써 군집간 제어를 할 수 있다.