大规模数据的统计建模与异常检测及其应用研究

With the rapid development of modern science and technology, the wide application of information, sensing and the data storage technology. Large-scale data emerged in many fields can be effectively collected. The high dimensionality and complex correlation of large-scale data increase the chance of outlying points occurrence. The presence of outliers inevitably has adverse effects on statistical analysis results such as parameter estimation, statistical inference, statistical prediction and model selection. This project will explore reasonable and effective statistical modeling and outlier detection methods for large-scale complex data. It mainly includes: (1) detecting multiple change points and outlying points of high-dimensional data, estimating the location of change points by truncation loss function, and then detecting outlying data between adjacent change points by robust covariance matrix estimation; (2) Detecting multiple abnormal points of complex profile data, and detecting outlying profiles by robust non-parametric statistics methods. The statistical method proposed in this project will be used to detect and analyze the abnormal changes of large-scale complex data in the fields of industrial production, economy and finance.

随着现代科技的迅猛发展以及信息、传感和数据存储技术的广泛应用，众多领域涌现的大规模数据可以被有效地采集。大规模数据的高维度和复杂相关性增加了其出现异常的机会，异常点的存在不可避免的对参数估计、统计推断、统计预测以及模型选择等统计分析结果造成不利的影响。本项目将针对大规模复杂数据，探索合理、有效的统计建模与异常检测方法。主要包括：(1)检测高维数据的多个变点和异常点，利用截断损失函数估计变点位置，再利用稳健的协方差矩阵估计的方法检测相邻变点之间异常数据; (2)检测复杂profile数据的多个异常点，利用稳健的非参数统计方法检测异常的profile数据。本项目提出的统计方法将用来检测并分析工业生产、经济金融等领域的大规模复杂数据的异常变化。