Congested Urban Networks Tend to Be Insensitive to Signal Settings: Implications for Learning-Based Control

This paper highlights several properties of large urban networks that can have an impact on machine learning methods applied to traffic signal control. In particular, we note that the average network flow tends to be independent of the signal control policy as density increases past the critical density. We show that this property, which so far has remained under the radar, implies that no control (i.e. a random policy) can be an effective control strategy for a surprisingly large family of networks, especially for networks with short blocks. We also show that this property makes deep reinforcement learning (DRL) methods ineffective when trained under congested conditions, independently of the particular algorithm used. Accordingly, in contrast to the conventional wisdom around learning-based methods promoting the exploration of all states, we find that for urban networks it is advisable to discard any congested data when training, and that doing so will improve performance under all traffic conditions. Our results apply to all possible grid networks thanks to a parametrization introduced here. The impact of the turning probability was found to be very significant, in particular to explain the loss of symmetry observed in the macroscopic fundamental diagram of the networks, which is not captured by existing theories that rely on corridor approximations without turns. Our findings also suggest that supervised learning methods have enormous potential as they require very little examples to produce excellent policies.

Automated summary

This paper highlights the impact of several properties of large urban networks on machine learning methods for traffic signal control. It shows that the average network flow is independent of signal control policy as density increases, making random policy ineffective for a large family of networks. The paper also demonstrates the ineffectiveness of deep reinforcement learning methods when trained under congested conditions, and suggests discarding congested data during training to improve performance. The findings emphasize the significant impact of turning probability on network symmetry and the potential of supervised learning methods.