Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Cooperative Traffic Signal Control at multi-intersection

다중 교차로에서 협력적 교통신호제어에 대한 연구

  • Received : 2019.12.05
  • Accepted : 2019.12.29
  • Published : 2019.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

As traffic congestion in cities becomes more serious, intelligent traffic control is actively being researched. Reinforcement learning is the most actively used algorithm for traffic signal control, and recently Deep reinforcement learning has attracted attention of researchers. Extended versions of deep reinforcement learning have been emerged as deep reinforcement learning algorithm showed high performance in various fields. However, most of the existing traffic signal control were studied in a single intersection environment, and there is a limitation that the method at a single intersection does not consider the traffic conditions of the entire city. In this paper, we propose a cooperative traffic control at multi-intersection environment. The traffic signal control algorithm is based on a combination of extended versions of deep reinforcement learning and we considers traffic conditions of adjacent intersections. In the experiment, we compare the proposed algorithm with the existing deep reinforcement learning algorithm, and further demonstrate the high performance of our model with and without cooperative method.

도시의 교통 혼잡 문제가 심각해지면서 지능형 교통신호제어가 활발하게 연구되고 있다. 강화학습은 교통신호제어에 가장 활발하게 사용되고 있는 알고리즘으로 최근에는 심층 강화학습 알고리즘이 관심을 끌고 있다. 또한 심층 강화학습 알고리즘이 다양한 분야에서 높은 성능을 보이면서 심층 강화학습의 확장 버전들이 빠른 속도로 등장했다. 하지만 기존 교통신호제어 연구들은 대부분 단일 교차로 환경에서 진행되었으며, 단일 교차로의 교통 혼잡만 완화하는 방법은 도시 전체의 교통 상황을 고려하지 못한다는 한계가 있다. 본 논문에서는 다중 교차로 환경에서 협력적 교통신호제어를 제안한다. 신호제어 알고리즘에는 심층 강화학습의 확장 버전들이 결합된 알고리즘을 적용했으며 다중 교차로를 효율적으로 제어하기 위해 인접한 교차로의 교통 상황을 고려하였다. 실험에서는 제안하는 알고리즘과 기존 심층 강화학습 알고리즘을 비교하였으며, 더 나아가 협력적 방법이 적용된 모델과 적용되지 않은 모델의 실험 결과를 보여줌으로써 높은 성능을 증명한다.

Keywords

References

  1. J. H. Youn, Y. G. Kang, "Simulation of Traffic Signal Control with Adaptive Priority Order through Object Extraction in Images," Journal of Korea Multimedia Society, Vol.11, No.8 pp.1043-1193, 2008.
  2. Cai, Chen, Chi Kwong Wong, and Benjamin G. Heydecker. "Adaptive traffic signal control using approximate dynamic programming," Transportation Research Part C: Emerging Technologies, Vol.17, No.5 pp.456-474, 2009. DOI: 10.1016/j.trc.2009.04.005
  3. Pandit, Kartik, et al. "Adaptive traffic signal control with vehicular ad hoc network," IEEE Transactions on Vehicular Technolog, Vol.62, No.4 pp.1459-1471, 2013. DOI: 10.1109/TVT.2013.2241460
  4. Maslekar, Nitin, et al. "VANET based adaptive traffic signal control," 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring). IEEE, 2011. DOI: 10.1109/VETECS.2011.5956305
  5. Mannion, Patrick, Jim Duggan, and Enda Howley. "An experimental review of reinforcement learning algorithms for adaptive traffic signal control," Autonomic Road Transport Support Systems. Birkhauser, Cham, pp.47-66, 2016. DOI: 10.1007/978-3-319-25808-9_4
  6. Sutton, Richard S., and Andrew G. Barto. Introduction to reinforcement learning. Vol.2. No.4. Cambridge: MIT press, 1998.
  7. Kaelbling, Leslie Pack, Michael L. Littman, and Andrew W. Moore. "Reinforcement learning: A survey," Journal of artificial intelligence research 4, pp.237-285, 1996. https://doi.org/10.1613/jair.301
  8. Li, Li, Yisheng Lv, and Fei-Yue Wang. "Traffic signal timing via deep reinforcement learning," IEEE/CAA Journal of Automatica Sinica, Vol.3, No.3 247-254, 2016. DOI: 10.1109/JAS.2016.7508798
  9. Genders, Wade, and Saiedeh Razavi. "Using a deep reinforcement learning agent for traffic signal control," arXiv preprint arXiv:1611.01142, 2016.
  10. Gao, Juntao, et al. "Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network," arXiv preprint arXiv:1705.02755, 2017.
  11. Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning," Nature 518, 7540 pp.529-533, 2015. DOI: 10.1038/nature14236
  12. YoungTae Jo, JinSup Choi, and InBum Jung, "Intersection Traffic Signal Control based on Traffic Pattern Learning for Repetitive Traffic Congestion," Journal of Computing Science and Engineering, Vol.20, No.8, pp.450-465, 2014.
  13. Kwang-baek Kim, "Intelligent Traffic Light Control using Fuzzy Method," Journal of The Korea Society of Computer and Information, Vol.15, No.9, pp.19-24. 2010. DOI: 10.6109/jkiice.2012.16.8.1593
  14. Chen, Ruey-Shun, Duen-Kai Chen, and Szu-Yin Lin. "ACTAM: Cooperative multi-agent system architecture for urban traffic signal control," IEICE transactions on Information and Systems Vol.88, No.1, pp.119-126, 2005.
  15. Van Hasselt, Hado, Arthur Guez, and David Silver. "Deep reinforcement learning with double q-learning," Thirtieth AAAI conference on artificial intelligence. 2016.
  16. Wang, Ziyu, et al. "Dueling network architectures for deep reinforcement learning," arXiv preprint arXiv:1511.06581, 2015.
  17. Schaul, Tom, et al. "Prioritized experience replay," arXiv preprint arXiv:1511.05952 (2015).
  18. Sutton, Richard S. "Learning to predict by the methods of temporal differences," Machine learning Vol.3, No.1, pp.9-44, 1988. DOI: 10.1007/BF00115009
  19. Bellemare, Marc G., Will Dabney, and Remi Munos. "A distributional perspective on reinforcement learning," Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.
  20. Fortunato, Meire, et al. "Noisy networks for exploration." arXiv preprint arXiv:1706.10295, 2017.
  21. Krajzewicz, Daniel, et al. "SUMO (Simulation of Urban MObility)-an open-source traffic simulation," Proceedings of the 4th middle East Symposium on Simulation and Modelling (MESM20002), 2002.