期刊
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
卷 23, 期 12, 页码 24904-24917出版社
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TITS.2022.3208236
关键词
Turning; Training; Throughput; Reinforcement learning; Analytical models; Observers; Vehicle dynamics; Traffic signal control; machine learning; deep reinforcement learning
资金
- NSF Research Project [1562536, 1932451]
- TOMNET University Transportation Center at Georgia Tech
This paper highlights the impact of several properties of large urban networks on machine learning methods for traffic signal control. It shows that the average network flow is independent of signal control policy as density increases, making random policy ineffective for a large family of networks. The paper also demonstrates the ineffectiveness of deep reinforcement learning methods when trained under congested conditions, and suggests discarding congested data during training to improve performance. The findings emphasize the significant impact of turning probability on network symmetry and the potential of supervised learning methods.
This paper highlights several properties of large urban networks that can have an impact on machine learning methods applied to traffic signal control. In particular, we note that the average network flow tends to be independent of the signal control policy as density increases past the critical density. We show that this property, which so far has remained under the radar, implies that no control (i.e. a random policy) can be an effective control strategy for a surprisingly large family of networks, especially for networks with short blocks. We also show that this property makes deep reinforcement learning (DRL) methods ineffective when trained under congested conditions, independently of the particular algorithm used. Accordingly, in contrast to the conventional wisdom around learning-based methods promoting the exploration of all states, we find that for urban networks it is advisable to discard any congested data when training, and that doing so will improve performance under all traffic conditions. Our results apply to all possible grid networks thanks to a parametrization introduced here. The impact of the turning probability was found to be very significant, in particular to explain the loss of symmetry observed in the macroscopic fundamental diagram of the networks, which is not captured by existing theories that rely on corridor approximations without turns. Our findings also suggest that supervised learning methods have enormous potential as they require very little examples to produce excellent policies.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据