Browse > Article
http://dx.doi.org/10.5762/KAIS.2011.12.11.5135

Traffic Control using Q-Learning Algorithm  

Zheng, Zhang (Mechanical Engineering Xian Jiaotong University)
Seung, Ji-Hoon (Electronics and Information Department, Chonbuk National University)
Kim, Tae-Yeong (Electronics and Information Department, Chonbuk National University)
Chong, Kil-To (Electronics and Information Department, Chonbuk National University)
Publication Information
Journal of the Korea Academia-Industrial cooperation Society / v.12, no.11, 2011 , pp. 5135-5142 More about this Journal
Abstract
A flexible mechanism is proposed in this paper to improve the dynamic response performance of a traffic flow control system in an urban area. The roads, vehicles, and traffic control systems are all modeled as intelligent systems, wherein a wireless communication network is used as the medium of communication between the vehicles and the roads. The necessary sensor networks are installed in the roads and on the roadside upon which reinforcement learning is adopted as the core algorithm for this mechanism. A traffic policy can be planned online according to the updated situations on the roads, based on all the information from the vehicles and the roads. This improves the flexibility of traffic flow and offers a much more efficient use of the roads over a traditional traffic control system. The optimum intersection signals can be learned automatically online. An intersection control system is studied as an example of the mechanism using Q-learning based algorithm, and simulation results showed that the proposed mechanism can improve the traffic efficiency and the waiting time at the signal light by more than 30% in various conditions compare to the traditional signaling system.
Keywords
Intelligent Transportation System; Cooperative Vehicle-Highway Systems; Reinforcement Learning; Traffic Control Mechanism; Intersection Signal Control;
Citations & Related Records
연도 인용수 순위
  • Reference
1 D. Srinivasan and M.C. Choy. "Cooperative multi-agent system for coordinated traffic signal control." IEE Proc. Intell. Transp. Syst. Vol. 153, No. 1, pp. 41-50, March 2006.   DOI
2 Lee, J.H., and Lee-Kwang, H. "Distributed and cooperative fuzzy controllers for traffic intersections group." IEEE Trans. Syst. Man Cybern. C, Appl. Rev. 29, (2), 1999.
3 Michael Wooldridge. "An Introduction to Multi agent Systems." JOHN WILEY & SONS, LTD. Baffins Lane, Chichester, West Sussex PO191UD, England, 2002.
4 Watkins, C., & Dayan, P. "Q-learning. Machine Learning," 8, pp. 279-292, 1997.
5 A. D'Ambrogio et al., "Simulation model building of traffic intersections, Simulat. Modell. Pract. Theory", 2008.
6 Min Chee Choy, Dipti Srinivasan, Ruey Long Cheu. "Cooperative, Hybrid agent Architecture for Real-Time traffic Signal Control." IEEE Transaction on Systems, Man, and Cybernetics - Part A: Systems and Humans, Vol. 33, No. 5, pp. 597-607, Sep. 2003.   DOI
7 Kouvelas, A., Aboudolas, K., Papageorgiou, M., Kosmatopoulos, E. B., "A Hybrid Strategy for Real Time Traffic Signal Control of Urban Road Networks, Intelligent Transportation Systems," IEEE Transactions on, pp. 1-11, 2011.
8 Srinivasan, D., Min Chee Choy, Cheu, R.L., "Neural Networks for Real-Time Traffic Signal Control, Intelligent Transportation Systems," IEEE Transactions on, pp. 261-272, 2006.
9 Gokulan, B.P., Srinivasan, D., "Distributed Geometric Fuzzy Multiagent Urban Traffic Signal Control, Intelligent Transportation Systems," IEEE Transactions on, pp. 714-727, 2010.
10 Balaji, P.G., German, X., Srinivasan, D.,"Urban Traffic Signal Control Using Reinforcement Learning Agents, Intelligent Transport Systems", IET, pp. 177-188, 2010.
11 Robertson, D.I., Bretherton, R.D., "Optimizing Networks of Traffic Signals in Real Time-The SCOOT Method," Vehicular Technology, IEEE Transactions on, pp. 11-15, 1991.
12 LIU Guang-ping, ZHAI Run-ping and PEI Yu-long. "A Calculating Method of Intersection Delay under Signal Control." Proceedings of the 2007 IEEE Intelligent Transportation Systems Conference, Seattle, WA, USA, Sept. 30 - Oct. 3, pp. 1114-1119, 2007.
13 Weihua Bao, Quanlin Chen and Xiaoxia Xu. "An Adaptive Traffic Signal Timing Scheme for Bus Priority at Isolated Intersection." Proceedings of the 6th World Congress on Intelligent Control and Automation, June 21-23, Dalian, China, pp. 8712- 8716, 2006,
14 Liao Yongquan, Cheng Xiangjun, "Study on Traffic Signal Control Based on Q-Learning," Fuzzy Systems and Knowledge Discovery, 2009. FSKD '09. Sixth International Conference on, pp. 581-585, 2009.
15 Wei Wu, Gong Shufeng, Liu Hongxiu, "A Coordinated Urban Traffic Signal Control Approach based on Multi Agent," Intelligent Engineering Systems, 2009. INES 2009. International Conference on, pp. 263-267, 2009
16 http://www.ewh.ieee.org/tc/its/.
17 Bong Keun Lee, Jae Du Chung, Keun Ho Ryu, "Multi-Agent Reinforcement Learning Model based on Fuzzy Inference", International Journal of Contents Vol. 9 No. 10, pp. 51-58, 2009.