基于深度加强学习的CBTC系统性能优化研究

Communication Based Train Control (CBTC) systems are automated train control systems based on continuous and bidirectional train-ground communications. CBTC is the direction of future train control systems. Complex channel condition and frequent handoff can severely affect urban rail CBTC train-ground communication performance, and consequently affect CBTC operation efficiency. Current research regarding CBTC does not consider the impact of train-ground communications on train control performance. This project studies the joint optimization of communication and control in CBTC systems. With the objective to minimize the optimal operation profile tracking error and energy consumption, Linear Quadratic Cost is defined as the control performance parameter. In order to ensure train operation safety, the optimization model puts constraints on train control actions related to safety. Based on the real field test data from urban rail operations, this project will first construct a finite state Markov channel model considering train positions. Next, according to the characteristics of train control in CBTC systems, a train control model considering communication latency will be established. Finally, based on the channel stochastic characteristics and the real time train position information, handoff decision and train control policy are jointly optimized using Deep Reinforcement Learning. The research in this project will provide strong theoretical basis for design and optimization of urban rail CBTC systems.

基于通信的列车控制(CBTC)系统是以连续车地通信为基础的列车运行控制系统，是列控系统的发展方向。复杂的信道条件，频繁的越区切换，将严重影响CBTC系统运行效率。当前对CBTC系统的研究很少考虑车地通信系统性能对列车控制的影响。 本项目研究CBTC系统中通信与控制的联合优化问题。项目以列车最优运行曲线跟踪误差和能耗最小为优化目标，定义列车的控制性能参数线性二次型成本。为了保证列车运行的安全，优化模型将增加安全相关的列车控制行为限制条件。项目首先利用城轨运营场景下的测试数据建立基于位置的有限状态马尔科夫信道模型；随后，根据列车运行控制的特点，研究考虑通信延迟的列车控制模型；最后，利用深度加强学习方法，根据系统中信道的随机变化特性以及列车的实时位置信息，将车地通信系统切换的决策，和列车控制行为决策进行联合优化设计。该项目的研究将为城轨CBTC系统的优化设计提供强有力的理论基础。

基于通信的列车控制(CBTC)系统是采用连续和双向车地通信的列车自动控制系统。CBTC是未来列车控制系统的发展方向。复杂的信道环境和频繁的切换会严重影响城市轨道交通CBTC系统中列车与地面的通信性能，进而影响CBTC的运行效率。目前关于CBTC的研究没有考虑列车与地面通信对列车控制性能的影响。本项目研究了CBTC系统中通信与控制的联合优化问题。为了最小化运行跟踪误差和能耗，项目将线性二次型成本定义为控制性能指标。为了保证列车运行的安全性，优化模型对安全相关的列车控制行为进行了约束。此外，项目根据随机信道特性和实时列车位置信息，利用深度强化学习方法对切换决策策略和列车控制策略进行了联合优化。基于实际信道测量结果和仿真结果表明，项目所提出的优化方法可以显著提高CBTC系统的列车控制性能，而CBTC系统需要牺牲部分性能来保证系统安全。