并行与分布式稀疏正则化学习

Bounding the estimation errors and developing fast numerical algorithms are two main tasks of sparse statistical learning. Due to the huge number of the sample and the dimensionality of features, designing and analyzing of parallel algorithm which is can be implemented at multi-core platform, and developing estimation theory and algorithms that ate suitable for distributed stored data become hot topics in statistical learning based on sparse regularization. The main effort of this proposal aimed at addressing the following two issues. The first major research of this project is to propose a parallel framework for multi - core platforms with shared memory to solve common sparse regularization learning models. We study the numerical convergence of the parallel algorithm (the convergence of the generated sequence to the optimal solution set) and the statistical property of the output the algorithm (the estimation error between the generated sequence and the true solution established with high probability). The second research contents of this project is proposing a distributed PDAS method when the data are distributed storage. We will bound the number of rounds of communications of distributed PDAS and prove the high probability estimation error and qualifying the uncertainty of the output solution.

研究稀疏正则化模型的估计误差和发展相应数值算法是稀疏统计学习的两个主要任务。由于数据的样本量和特征的维数都很大, 设计和分析适合共享内存的多核计算平台的并行算法以及发展适合数据分布式存储在多机情景下的估计理论和算法成为了基于稀疏正则化的统计学习的研究热点。本项目的第一个主要研究内容是提出一个适用于共享内存的多核平台的并行框架求解常见稀疏正则化学习模型。结合连续化研究所提并行算法的数值收敛性（迭代点列到模型最优解集的收敛性分析其并行效率）和算法输出解的统计性质（以高概率成立的迭代点列和真解之间的估计误差）。本项目的第二研究内容是提出在数据分布式存储情景下的分布式PDAS方法并分析分布式PDAS 的通讯次数、证明输出解和真解之间以高概率成立的估计误差、并基于分布式PDAS输出解对真解构建置信区间进行不确定性量化。

稀疏学习的模型与算法是近10年统计、应用数学、计算机、工程应用等众多领域地交叉点。相关的模型、算法对国防、医疗、生产生活产生了巨大的影响。研究稀疏学习中有理论保证的高效算法, 给众多的应用场景提供理论基石有着重要的意义。按照项目的规划我们设计求解稀疏学习模型的高效算法并分析了其统计性质，弥补了统计和计算之间的间隙。具体如下：. 统计COPSS总统奖获得者、ICM45钟报告人、加州大学伯克利分校的M. Wainwright 教授在2014年发表在Annual Review of Statistics and Its Applications 的综述性论文中指出“填补高维统计和计算之间的间隙是一个公开问题”。候选人和合作者设计了求解高维稀疏模型PDAS算法和其分布式版本, 并证明了算法输出结果与真解的统计误差, 解决了此公开问题, 相关论文发表在Journal of Mechine Learning Research 、Statistical Science、SIAM Jouranl on Scientific Computing、IEEE Transaction on Signal Processing, Neurocomputing 等杂志。 相关工作收到 AMS和ISA Fellow、滴滴出行首席统计学家Zhu Hongtu 教授等国内外同行的关注应用和推广。PDAS算法得到了工业界的广泛关注被集成到华为自主研发的的机器学习库中。