连续时间马氏决策过程受约束问题的研究

Markov decision processes have wide applications to many areas, such as finance, insurance, communication network, inventory management and reliability theory. Since the time interval in many real-world applications is always finite, the study on the finite horizon problems of Markov decision processes is of important significance on both theoretical and applied sides. We will carry out an research on continuous-time Markov decision processes and study the following constrained optimization problems in this project. . (1) We will consider the finite horizon expected criteria with multiple constraints in which the objective function and the constraint functions are all the finite horizon expected total payoffs. . (2) We will investigate the mean-variance problem of the finite horizon expected total payoffs. That is, we aim to find a policy which minimizes the variance of the finite horizon expected total payoffs over the set of all policies satisfying the constraint that the mean of the finite horizon expected total payoffs is equal to a given constant. . We will study the existence and computational methods of the optimal policies for the above problems. The research of this project not only enriches the optimization theory of the stochastic dynamic systems, but also lays a theoretical basis for the applied research of Markov decision models.

马氏决策过程在众多领域有着广泛的应用，如金融保险、通信网络、库存管理、可靠性理论等。现实世界中的许多应用问题涉及到的时间总是有限的，研究马氏决策过程的有限阶段问题在理论上和应用上都具有重要的意义。本项目拟研究连续时间马氏决策过程的受约束问题：(1)有限阶段期望准则的多约束问题，其中目标函数和约束函数均为有限阶段期望总效益；(2)有限阶段总效益的均值-方差问题，即在有限阶段总效益的均值等于某一常数的策略类中寻找使得有限阶段总效益的方差最小的策略。我们将研究上述问题最优策略的存在性和计算方法这两个核心的理论问题。本项目的研究不仅能丰富随机动态系统的最优化理论，而且为马氏决策模型的应用研究奠定理论基础。

本项目主要研究连续时间马氏决策过程的有限阶段问题，具体如下：. （1）有限阶段期望准则最优值函数和最优策略的计算方法。关于该准则，给出了计算最优值函数和最优策略的有限逼近算法，得到了相应的误差估计式，证明了该算法的收敛性。相应结果发表在国际学术期刊4OR-A Quarterly Journal of Operations Research上。. （2）风险灵敏性有限阶段准则最优策略的存在性条件和计算方法。关于该准则，给出了最优策略的存在性条件，得到了转移率无界情形的连续时间马氏决策过程的Feynman-Kac公式，建立了最优方程解的存在唯一性，证明了最优策略的存在性，给出了近似计算最优值函数和最优策略的值迭代算法和相应的误差估计式。相应结果发表在国际学术期刊Mathematical Methods of Operations Research上。. 本项目给出了近似计算最优值函数和最优策略的新算法，为马氏决策模型的应用研究奠定了算法基础。