马氏决策过程理论及其在基因调控网络中的应用

First passage models for discrete-time Markov decision processes have been studied by many authors. However, the state and action spaces are assumed to be denumerable and finite respectively, the rewards may be bounded in the existing work on first passage models. On the other hand, Markov decision processes appear in many classes of applicitions. However, few studies have dealt with the applications of Markov decision processses in genetic regulatory networks. In this project, it mainly concludes two parts, theoretical and applications studies on Markov decision processes.The first part deals with the first passage problems for discrete-time Markov decision processes. More precisely, we focus on the existence of optimal policies under this optimality criterion, and we also consider the efficient algorithm for computing optimal policies.The second part is on applications of Markov decision processes in genetic regulatory networks.The main contributions of applications of Markov decision processes are concluded in the following aspects. (1)We make use of the theory of discrete-time Markov decision processes to solve optimal control problems for synchronous probabilistic Boolean networks.(2)We formulate some control models for a generalized asynchronous as some models for semi-Markov decision processes and solve the corresponding optimal control problems.The aims of this project are to promote further development of stochastic optimal control, better understand the mechanism of gene regulatory network and increase the potential applications in designing drugs targets and gene therapy.

过去对离散时间马氏过程的首达目标准则的理论研究仅局限在状态空间可数、行动空间有限且报酬有界的情形。另一方面，马氏决策过程在实际中的应用研究已经深入到很多领域，但其在基因调控网方面的应用研究很少有人探讨。本项目拟在已有工作的基础上，对马氏决策过程理论和应用两大方面展开研究。理论研究集中于探索离散时间马氏决策过程中状态空间和行动空间均一般，报酬无界的首达目标准则，试图探讨在这个准则下最优策略存在的条件及算法；应用研究主要是在概率布尔型的基因调控网方面，包括：（1）构建相应的离散时间马氏决策过程模型，寻求同步概率布尔网络情形的最优控制策略；(2) 构建相应的半马氏决策过程模型，寻求非同步概率布尔网络情形的最优控制策略。本项目中的研究内容不仅能推动随机最优控制理论的新进展，而且有助于理解基因调控网的本质机制，并对药物设计、基因治疗等方面具有潜在的应用前景。

本项目对马氏决策过程理论和应用两大方面展开研究。理论研究集中于探索半马氏决策过程中状态空间和行动空间均一般，报酬无界的风险概率准则，探讨在这个准则下最优策略存在的条件及算法；应用研究主要是在概率布尔型的基因调控网方面，包括：（1）构建相应的离散时间马氏决策过程模型，寻求同步概率布尔网络情形的最优控制策略；(2) 构建相应的半马氏决策过程模型，寻求非同步概率布尔网络情形的最优控制策略。