大规模数据的统计建模与异常检测及其应用研究

The high dimensionality and complex correlation of large-scale data increase the chance of outlying points occurrence inevitably. The presence of outliers has adverse effects on statistical analysis results such as parameter estimation, statistical inference, statistical prediction and model selection. This project will explore reasonable and effective statistical modeling and outlier detection methods for large-scale complex data. It mainly includes: (1) detecting multiple change points and outlying points of high-dimensional data, estimating the number and location of change points by truncation loss function, and then detecting outlying data between adjacent change points by robust covariance matrix estimation; (2) Detecting multiple outlying points of complex profile data, and detecting outlying profiles by robust non-parametric statistics methods. The statistical method proposed in this project will be used to detect and analyze the abnormal changes of large-scale complex data in the fields of industrial production, economy and finance.

大规模数据的高维度和复杂相关性不可避免的增加异常点出现的机会，异常点的存在对参数估计、统计推断、统计预测以及模型选择等统计分析结果造成不利的影响。本项目将针对大规模复杂数据，探索合理、有效的统计建模与异常检测方法。主要包括：(1)检测高维数据的多个变点和异常点，利用截断损失函数估计变点的数目和位置，再利用稳健的协方差矩阵估计的方法检测相邻变点之间异常数据; (2)检测复杂profile数据的多个异常点，利用稳健的非参数统计方法检测异常的profile数据。本项目提出的统计方法将用来检测并分析工业生产、经济金融等领域的大规模复杂数据的异常变化。