ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Two tales of platoon intelligence for autonomous mobility control: Enabling deep learning recipes

  • Soohyun Park (School of Electrical Engineering, Korea University) ;
  • Haemin Lee (School of Electrical Engineering, Korea University) ;
  • Chanyoung Park (School of Electrical Engineering, Korea University) ;
  • Soyi Jung (Department of Electrical and Computer Engineering, Ajou University) ;
  • Minseok Choi (Department of Electronics Engineering, Kyung Hee University) ;
  • Joongheon Kim (School of Electrical Engineering, Korea University)
  • Received : 2023.03.31
  • Accepted : 2023.08.08
  • Published : 2023.10.20
Download PDF
⟨ Previous Next ⟩

Abstract

This paper surveys recent multiagent reinforcement learning and neural Myerson auction deep learning efforts to improve mobility control and resource management in autonomous ground and aerial vehicles. The multiagent reinforcement learning communication network (CommNet) was introduced to enable multiple agents to perform actions in a distributed manner to achieve shared goals by training all agents' states and actions in a single neural network. Additionally, the Myerson auction method guarantees trustworthiness among multiple agents to optimize rewards in highly dynamic systems. Our findings suggest that the integration of MARL CommNet and Myerson techniques is very much needed for improved efficiency and trustworthiness.

Keywords

Acknowledgement

IITP funded by the Korea government (MSIT) (No. 2022-0-00907) and also by the National Research Foundation of Korea (2022R1C1C1010766).

References

  1. S. Jung and J. Kim, Adaptive and stabilized real-time superresolution control for uav-assisted smart harbor surveillance platforms, J. Real-Time Image Process. 18 (2021), 1815-1825. https://doi.org/10.1007/s11554-021-01163-2
  2. J. Wang, J. Liu, and N. Kato, Networking and communications in autonomous driving: a survey, IEEE Commun. Surv. Tutorials 21 (2019), no. 2, 1243-1274. https://doi.org/10.1109/COMST.2018.2888904
  3. Y. Zheng, S. Eben Li, J. Wang, D. Cao, and K. Li, Stability and scalability of homogeneous vehicular platoon: study on the influence of information flow topologies, IEEE Trans. Intell. Transp. Syst. 17 (2016), no. 1, 14-26. https://doi.org/10.1109/TITS.2015.2402153
  4. S. Aradi, Survey of deep reinforcement learning for motion planning of autonomous vehicles, IEEE Trans. Intell. Transp. Syst. 23 (2022), no. 2, 740-759. https://doi.org/10.1109/TITS.2020.3024655
  5. S. Jung, W. J. Yun, J. Kim, and J.-H. Kim, Infrastructureassisted cooperative multi-UAV deep reinforcement energy trading learning for big-data processing, (International Conference on Information Networking (ICOIN), Jeju, Republic of Korea), 2021, pp. 159-162.
  6. L. Lei, Y. Tan, K. Zheng, S. Liu, K. Zhang, and X. Shen, Deep reinforcement learning for autonomous internet of things: model, applications and challenges, IEEE Commun. Surv. Tutorials 22 (2020), no. 3, 1722-1760. https://doi.org/10.1109/COMST.2020.2988367
  7. X. Pan, B. Chen, S. Timotheou, and S. A. Evangelou, A convex optimal control framework for autonomous vehicle intersection crossing, IEEE Trans. Intell. Transp. Syst. 24 (2023), no. 1, 163-177.
  8. M. Choi, J. Kim, and J. Moon, Wireless video caching and dynamic streaming under differentiated quality requirements, IEEE J. Sel. Areas Commun. 36 (2018), no. 6, 1245-1257. https://doi.org/10.1109/JSAC.2018.2844980
  9. J. Kim, G. Caire, and A. F. Molisch, Quality-aware streaming and scheduling for device-to-device video delivery, IEEE/ACM Trans. Netw. 24 (2016), no. 4, 2319-2331.
  10. J. Koo, J. Yi, J. Kim, M. A. Hoque, and S. Choi, Seamless dynamic adaptive streaming in LTE/Wi-Fi integrated network under smartphone resource constraints, IEEE Trans. Mobile Comput. 18 (2019), no. 7, 1647-1660.
  11. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, Playing atari with deep reinforcement learning, arXiv preprint, 2013, arXiv: 1312.5602.
  12. S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, A survey of deep learning techniques for autonomous driving, J. Field Robot. 37 (2020), no. 3, 362-386.
  13. X. Tan, L. Zhou, H. Wang, Y. Sun, H. Zhao, B.-C. Seet, J. Wei, and V. C. M. Leung, Cooperative multi-agent reinforcementlearning-based distributed dynamic spectrum access in cognitive radio networks, IEEE Internet Things J. 9 (2022), no. 19, 19477-19488. https://doi.org/10.1109/JIOT.2022.3168296
  14. S. Sukhbaatar, A. Szlam, and R. Fergus, Learning multiagent communication with backpropagation, (Proc. of Advances in Neural Information Processing Systems (NEURIPS), Barcelona, Spain), 2016, pp. 2244-2252.
  15. M. Shin, D.-H. Choi, and J. Kim, Cooperative management for PV/ESS-enabled electric vehicle charging stations: a multiagent deep reinforcement learning approach, IEEE Trans. Ind. Inform. 16 (2020), no. 5, 3493-3503. https://doi.org/10.1109/TII.2019.2944183
  16. A. P. Cohen, S. A. Shaheen, and E. M. Farrar, Urban air mobility: history, ecosystem, market potential, and challenges, IEEE Trans. Intell. Transp. Syst. 22 (2021), no. 9, 6074-6087. https://doi.org/10.1109/TITS.2021.3082767
  17. S. Jung, W. J. Yun, M. Shin, J. Kim, and J.-H. Kim, Orchestrated scheduling and multi-agent deep reinforcement learning for cloud-assisted multi-UAV charging systems, IEEE Trans. Veh. Technol. 70 (2021), no. 6, 5362-5377. https://doi.org/10.1109/TVT.2021.3062418
  18. Z. Zhang, Y. Xiao, Z. Ma, M. Xiao, Z. Ding, X. Lei, G. K. Karagiannidis, and P. Fan, 6G wireless networks: vision, requirements, architecture, and key technologies, IEEE Veh. Technol. Mag. 14 (2019), no. 3, 28-41.
  19. W. J. Yun, S. Park, J. Kim, M. Shin, S. Jung, D. A. Mohaisen, and J.-H. Kim, Cooperative multiagent deep reinforcement learning for reliable surveillance via autonomous multi-UAV control, IEEE Trans. Ind. Inform. 18 (2022), no. 10, 7086-7096. https://doi.org/10.1109/TII.2022.3143175
  20. X. Cao, J. Zhang, and H. V. Poor, A virtual-queue-based algorithm for constrained online convex optimization with applications to data center resource allocation, IEEE J. Sel. Topics Signal Process. 12 (2018), no. 4, 703-716. https://doi.org/10.1109/JSTSP.2018.2827302
  21. T. Chen, Q. Ling, and G. B. Giannakis, An online convex optimization approach to proactive network resource allocation, IEEE Trans. Signal Process 65 (2017), no. 24, 6350-6364. https://doi.org/10.1109/TSP.2017.2750109
  22. K. Tan, L. Feng, G. Dan, and M. Torngren, Decentralized convex optimization for joint task offloading and resource allocation of vehicular edge computing systems, IEEE Trans. Veh. Technol. 71 (2022), no. 12, 13226-13241. https://doi.org/10.1109/TVT.2022.3197627
  23. P. S. Pillai and S. Rao, Resource allocation in cloud computing using the uncertainty principle of game theory, IEEE Syst. J. 10 (2016), no. 2, 637-648. https://doi.org/10.1109/JSYST.2014.2314861
  24. W. Yuan, P. Wang, W. Liu, and W. Cheng, Variable-width channel allocation for access points: a game-theoretic perspective, IEEE Trans. Mobile Comput. 12 (2013), no. 7, 1428-1442. https://doi.org/10.1109/TMC.2012.109
  25. E. Maskin, Auctions and efficiency, School of Social Science, Institute for Advanced Study, 2001.
  26. M. Shin, J. Kim, and M. Levorato, Auction-based charging scheduling with deep learning framework for multi-drone networks, IEEE Trans. Veh. Technol. 68 (2019), no. 5, 4235-4248. https://doi.org/10.1109/TVT.2019.2903144
  27. X. Wang, X. Chen, and W. Wu, Towards truthful auction mechanisms for task assignment in mobile device clouds, (IEEE Infocom 2017-IEEE Conference on Computer Communications, Atlanta, GA, USA), 2017, pp. 1-9.
  28. L. Park, S. Jeong, J. Kim, and S. Cho, Joint geometric unsupervised learning and truthful auction for local energy market, IEEE Trans. Ind. Electron. 66 (2018), no. 2, 1499-1508.
  29. L. Park, S. Jeong, D. S. Lakew, J. Kim, and S. Cho, New challenges of wireless power transfer and secured billing for internet of electric vehicles, IEEE Commun. Mag. 57 (2018), no. 3, 118-124.
  30. W. Sun, J. Liu, Y. Yue, and H. Zhang, Double auction-based resource allocation for mobile edge computing in industrial internet of things, IEEE Trans. Indust. Inform. 14 (2018), no. 10, 4692-4701. https://doi.org/10.1109/TII.2018.2855746
  31. N. C. Luong, Z. Xiong, P. Wang, and D. Niyato, Optimal auction for edge computing resource management in mobile blockchain networks: a deep learning approach, (IEEE International Conference on Communications, Kansas City, MO, USA), 2018, pp. 1-6.
  32. K. Zhu, Y. Xu, Q. Jun, and D. Niyato, Revenue-optimal auction for resource allocation in wireless virtualization: a deep learning approach, IEEE Trans. Mobile Comput. 21 (2020), no. 4, 1374-1387.
  33. R. B. Myerson, Optimal auction design, Math. Oper. Res. 6 (1981), no. 1, 58-73. https://doi.org/10.1287/moor.6.1.58
  34. H. Daniels and M. Velikova, Monotone and partially monotone neural networks, IEEE Trans. Neural Netw. 21 (2010), no. 6, 906-917. https://doi.org/10.1109/TNN.2010.2044803
  35. S. Say, H. Inata, J. Liu, and S. Shimamoto, Priority-based data gathering framework in uav-assisted wireless sensor networks, IEEE Sensors J. 16 (2016), no. 14, 5785-5794. https://doi.org/10.1109/JSEN.2016.2568260
  36. D. Van Huynh, T. Do-Duy, L. D. Nguyen, M.-T. Le, N.-S. Vo, and T. Q. Duong, Real-time optimized path planning and energy consumption for data collection in unmanned ariel vehicles-aided intelligent wireless sensing, IEEE Trans. Ind. Inform. 18 (2021), no. 4, 2753-2761.
  37. C. Zhan, Y. Zeng, and R. Zhang, Energy-efficient data collection in UAV enabled wireless sensor network, IEEE Wirel. Commun. Lett. 7 (2017), no. 3, 328-331.
  38. J. Gong, T.-H. Chang, C. Shen, and X. Chen, Flight time minimization of uav for data collection over wireless sensor networks, IEEE J. Sel. Areas Commun. 36 (2018), no. 9, 1942-1954. https://doi.org/10.1109/JSAC.2018.2864420
  39. C. Singhal and B. N. Chandana, Aerial-SON: UAV-based selforganizing network for video streaming in dense urban scenario, (International Conference on Communication Systems & Networks (COMSNETS), Bangalore, India), 2021, pp. 7-12.
  40. V. Krishna, Auction theory, Academic press, 2009.
  41. P. Klemperer, Auction theory: a guide to the literature, J. Econ. Surv. 13 (1999), no. 3, 227-286. https://doi.org/10.1111/1467-6419.00083
  42. B. Dai, H. Chen, and G. Yang, Price-setting based combinatorial auction approach for carrier collaboration with pickup and delivery requests, Oper. Res. 14 (2014), 361-386.
  43. D. C. Marinescu, A. Paya, J. P. Morrison, and P. Healy, An auction-driven self-organizing cloud delivery model, arXiv preprint, 2013, arXiv:1312.2998.
  44. B. Coltin and M. Veloso, Online pickup and delivery planning with transfers for mobile robots, (IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China) 2014, pp. 5786-5791.
  45. J. He, D. Zhang, Y. Zhou, and Y. Zhang, A truthful online mechanism for collaborative computation offloading in mobile edge computing, IEEE Trans. Ind. Inform. 16 (2019), no. 7, 4832-4841.
  46. H. Lee, S. Jung, and J. Kim, Truthful electric vehicle charging via neural-architectural myerson auction, ICT Express 7 (2021), no. 2, 196-199. https://doi.org/10.1016/j.icte.2021.03.009
  47. K. Kuo, A. Ostuni, E. Horishny, M. J. Curry, S. Dooley, P. Chiang, T. Goldstein, and J. P. Dickerson, ProportionNet: balancing fairness and revenue for auction design with deep learning, arXiv preprint, 2020, arXiv:2010.06398.