Journal of the Korean Institute of Intelligent Systems (한국지능시스템학회논문지)

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Reinforcement Learning Based Evolution and Learning Algorithm for Cooperative Behavior of Swarm Robot System

군집 로봇의 협조 행동을 위한 강화 학습 기반의 진화 및 학습 알고리즘

  • 서상욱 (중앙대학교 전자전기공학부) ;
  • 김호덕 (중앙대학교 전자전기공학부) ;
  • 심귀보 (중앙대학교 전자전기공학부)
  • Published : 2007.10.25
Download PDF
⟨ Previous Next ⟩

Abstract

In swarm robot systems, each robot must behaves by itself according to the its states and environments, and if necessary, must cooperates with other robots in order to carry out a given task. Therefore it is essential that each robot has both learning and evolution ability to adapt the dynamic environments. In this paper, the new polygon based Q-learning algorithm and distributed genetic algorithms are proposed for behavior learning and evolution of collective autonomous mobile robots. And by distributed genetic algorithm exchanging the chromosome acquired under different environments by communication each robot can improve its behavior ability Specially, in order to improve the performance of evolution, selective crossover using the characteristic of reinforcement learning is adopted in this paper. we verify the effectiveness of the proposed method by applying it to cooperative search problem.

군집 로봇시스템에서 개개의 로봇은 스스로 주위의 환경과 자신의 상태를 스스로 판단하여 행동하고, 필요에 따라서는 다른 로봇과 협조를 통하여 어떤 주어진 일을 수행할 수 있어야 한다. 따라서 개개의 로봇은 동적으로 변화하는 환경에 잘 적응할 수 있는 학습과 진화능력을 갖는 것이 필수적이다 이를 위하여 본 논문에서는 새로운 Polygon 기반의 Q-learning 알고리즘과 분산유전알고리즘을 이용한 새로운 자율이동로봇의 행동학습 및 진화방법을 제안한다. 또한 개개의 로봇이 통신을 통하여 염색체를 교환하는 분산유전알고리즘은 각기 다른 환경에서 학습한 우수한 염색체로부터 자신의 능력을 향상시킨다. 특히 본 논문에서는 진화의 성능을 향상시키기 위하여 강화학습의 특성을 이용한 선택 교배방법을 채택하였다. 제안된 방법은 협조탐색 문제에 적용하여 컴퓨터 모의실험을 통하여 그 유효성을 검증한다.

Keywords

References

  1. 윤한얼, 심귀보, '다수 로봇의 목표물 탐색을 위한 Area-Based Q-leaming 알고리즘', 한국퍼지 및 지능 시스템학회 논문지, 제15권, 제4호, pp. 406-411, 2005 https://doi.org/10.5391/JKIIS.2005.15.4.406
  2. Jindong Tan, Ning Xi, Weihua Sheng and Jizhong Xiao, 'Modeling Multiple Robot Systems for Area Coverage and Cooperation', Proc. of the 2004 IEEE International Conference on Robotics & Automation, Vol. 3, pp. 2568-2573, New Orleans, LA, April, 2004
  3. Vaithilingam Kumarathasan, Thrishantha Nanayakkara, 'Intelligent Collaboration among Robotic Agents for Landmine Detection', Proc. of the Annual Sessions of the Sri Lanka Association for Artificial Intelligence, 2005
  4. Alessabdro de Luna Almeida, Samir Aknine, Jean-Pierre Briot, Jacques Malenfant, 'Plan-Based Replication for Fault-Tolerant Multi - Agent Systems', Proc. of the 20th International Parallel and Distributed Processing Symposium, 2006
  5. Izzet Can Envarli, Julie A. Adams, 'Task Lists for Human-Multiple Robot Interaction', Proc. of the IEEE International Workshop on Robots and Human Interaction Communication, pp. 119-124, Aug. 2005
  6. Alain Cardon, Thierry Galinho, Ieari-Philippe Vacher, 'Genetic algorithm using multi-objectives in a multi-agent system', Proc. of Robotics and Autonomous System, pp. 179-190, 2000
  7. Xiaojiang Zhang, 'Fuzzy control system for a mobile robot collision avoidance', Pro. of the IEEE International Conference on Industrial Technology, pp. 125-128, 1994
  8. Ding Yingying, He Yan, Jiang jing-Ping, 'Self-Organizing Multi-robot System Based on Personality Evolution', Proc. of the IEEE International Conference on Systems, Man and Cybernetics, 2002
  9. Prasanna Sridhar, Shahab Sheikh-Bahaei, Shan Xia, Mo Jamshidi, 'Multi agent Simulation using Discrete Event and Soft-computing Methodologies', Proc. of the IEEE International Conference on Systems, Man and Cybernetics, Vol. 1, pp 1004-1012, Dec, 2003
  10. Thomas W. Dunbar, Joel M. Esposito, 'Artificial Potential Field Controllers for Robust Communications in a Network of Swarm Robots', Proc. of the Thirty-Seventh Southeastern Symposium, pp. 401-405, March. 2005
  11. Mary Berna-Koes, Illah Nourbakhsh, Katia Sycara, 'Communication Efficiency in Multi-Agent Systems', Proc. of the IEEE International Conference on Robotics & Automation, pp. 2129-2134, April. 2004
  12. Chris A. C. Parker, Hong Zhang, 'A Practical Implementation of Random Peer-to-Peer Communication for a Multiple-Robot System', Proc. of the IEEE International Conference on Robotics and Automation, pp. 3730- 3735, April. 2007
  13. Mohd Ridzuan Ahmad, Shamsudin H.M. Amin, Rosbi Mamat, 'Development of Decentralized Based Reactive Control Strategy for Intelligent Multi-Agent Mobile Robotics System', Proc. of the Seventh International Conference on Control, Automation, Robotics And Vision, pp. 220-227, Dec. 2002
  14. Ou Haitao, Zhang Weidong, Zhang Wenyuan, Xu Xiaoming, 'A novel multi-agent Q-leaming algorithm in cooperative multi-agent system', Proc. of the 3rd World Congress on Intelligent Control and Automation, pp. 272-276, 2000
  15. Jing Huang, Bo Yang, Da-You Liu, 'A Distributed Q-leaming Algorithm for Multi-Agent Team Coordination', Proc. of the Fourth International Conference on Machine Learning and Cybernetics, pp. 108-113, August. 2005
  16. Tong Zhou, Bing-Rong Hong, Chao-Xia Shi, Hong-Yu Zhou, 'Cooperative Behavior Acquisition Based Modular Q-learning m Multi-Agent System', Proc. of the Fourth International Conference on Machine Learning and Cybernetics, pp 205-210, August. 2005