Deep Reinforcement Learning-Based Holding Control for Bus Bunching under Stochastic Travel Time and Demand

Due to the inherent uncertainties of the bus system, bus bunching remains a challenging problem that degrades bus service reliability and causes passenger dissatisfaction. This paper introduces a novel deep reinforcement learning framework specifically designed to address the bus bunching problem by implementing dynamic holding control in a multi-agent system. We formulate the bus holding problem as a decentralized, partially observable Markov decision process and develop an event-driven simulator to emulate real-world bus operations. An approach based on deep Q-learning with parameter sharing is proposed to train the agents. We conducted extensive experiments to evaluate the proposed framework against multiple baseline strategies. The proposed approach has proven to be adaptable to the uncertainties in bus operations. The results highlight the significant advantages of the deep reinforcement learning framework across various performance metrics, including reduced passenger waiting time, more balanced bus load distribution, decreased occupancy variability, and shorter travel time. The findings demonstrate the potential of the proposed method for practical application in real-world bus systems, offering promising solutions to mitigate bus bunching and enhance overall service quality.

Automated summary

This paper presents a novel deep reinforcement learning framework specifically designed to address the issue of bus bunching. The framework implements dynamic holding control in a multi-agent system, formulating the bus holding problem as a decentralized, partially observable Markov decision process. An event-driven simulator is developed to emulate real-world bus operations, and deep Q-learning with parameter sharing is proposed to train the agents. Extensive experiments demonstrate the significant advantages of the framework in reducing passenger waiting time, balancing bus load distribution, decreasing occupancy variability, and shortening travel time. The findings highlight the potential of the proposed method for practical application in real-world bus systems, offering promising solutions to mitigate bus bunching and enhance overall service quality.