随机系统的时间不一致最优控制

Bellman optimality principle is a keystone of dynamic programming theory, by which we have the time-consistency of optimal control: the local optimality can be inferred from the global optimality. However, in reality the time-consistency of optimal control fails quite often. The reasons lie in the following three points: the non-exponential discounting in the cost functional, the nonlinear term of conditional expectation in the cost functional, and the fact that time-state initial pair enters explicitly into the controlled system. The time-inconsistency pulls down the Bellman dynamic programming principle, and thus its study is very meaningful. This project aims to the study of optimal control problems with time-inconsistency, and the content is as follows. For the linear-quadratic case, some necessary and sufficient conditions will be, respectively, presented for the existence of open-loop equilibrium control and closed-loop equilibrium strategy, and the robustness of these two time-consistent solutions will be investigated. Moreover, for the general nonlinear case, necessary conditions on the existence of open-loop equilibrium control and closed-loop equilibrium strategy will be established. Policy iteration algorithm and an adaptive-dynamic-programming-type algorithm will also be presented to construct the equilibrium strategy/control. Furthermore, apply the derived time-inconsistent stochastic control theory to practical problems (such as the economic investment decision), which will promote the solvability of some corresponding problems. The implement of this project will deeply push existing optimal control theory, which has scientific significance and practical values.

Bellman最优性原理是动态规划理论的基石，由它可得最优控制的时间一致性：总体最优可推出局部最优。然而在实践中，该时间一致性却经常丢失，且导致时间不一致性的因素有三：性能指标中的非指数折扣、性能指标中条件数学期望的非线性项、时间状态初始对显示地进入到受控系统中。时间不一致性破坏了动态规划原理，因而对它的研究具有重大意义。本项目旨在研究时间不一致随机最优控制的如下内容：(1) 对线性二次问题，分别从开环和闭环的角度建立平衡策略/控制存在的充要条件，并探讨平衡策略/控制的鲁棒性；(2) 对一般非线性问题，研究平衡策略/控制存在的必要条件，并建立构造平衡策略/控制的迭代优化算法和自适应动态规划型算法；(3) 将所得时间不一致随机控制理论应用到经济投资决策等实际问题中，以期推进相关问题的解决。本项目的实施将深度推进现有最优控制理论，具有重大的科学价值和实用价值。

Bellman最优性原理是动态规划理论的基石，由它可得最优控制的时间一致性：总体最优可推出局部最优。然而在实践中，该时间一致性却经常丢失，且导致时间不一致性的因素有三：性能指标中的非指数折扣、性能指标中条件数学期望的非线性项、时间状态初始对显示地进入到受控系统中。时间不一致性破坏了动态规划原理，因而对它的研究具有重大意义。本项目的主要研究内容如下：对几大类线性二次问题，完全刻画了时间一致的开环均衡控制，引入了混合均衡解这一新概念，提出了处理时间不一致最优控制问题的Nash-型虚拟博弈框架，研究了鲁棒时间不一致的最优控制问题和马氏跳变参数下的问题，对Stackelberg动态博弈引入了时间一致的开环均衡解概念等。项目负责人发表文章11篇，会议论文5篇，其中SIAM Journal on Control and Optimization上3篇（其中一篇接收待见刊），IEEE Transactions on Automatic Control上2篇，International Journal of Robust and Nonlinear Control上2篇，IEEE Transactions on Cybernetics上1篇等。且项目组成员倪元华、孙明玮、张建磊分别获批新的国家自然科学基金面上项目。