基于图的半监督学习算法研究

Nowadays, the semi-supervised learning has become a hot topic in the area of machine learning, pattern recognition and signal processing. The graph-based semi-supervised algorithm is an important method in semi-supervised learning. Recent research has revealed several drawbacks of these algorithms such as the high computation cost in the optimization procedure and the sensitivity in dealing with problems involving heavy tailed non-Gaussian noise. If these problems are not solved properly, they will decrease the effectiveness of the graph-based methods, which could even destroy the benefits of semi-supervised learning. In this project, we will first try to establish proper assumption models. Under these models, the sparse graph-based regularization algorithms will be proposed, which can effectively reduce the computational complexity. Then the entropy in information theory will be applied to the learning of graph-based regularization algorithms. We will employ entropy to substitute the traditional square loss function in graph-based regularization methods, and construct novel algorithms which can effectively deal with the non-Gaussian distribution noise. Next, in the framework of statistical learning theory, we provide a comprehensive analysis on the sparsity, stability and convergence for the proposed formulations under the operator assumptions. Finally, the efficiency of the proposed methods will be verified by experiments on both artificial data sets and real data sets. The aim of the research project is to improve the sparsity, stability and convergence of graph based regularization algorithms. Also, we wish to promote the basic theoretical achievements to the level of application. We believe that the expected results of the project could provide effective graph based regularization algorithms, and enrich the development of the theory and application for graph-based semi-supervised learning.

半监督学习是近年来机器学习、模式识别及信号处理等领域的热点问题。而基于图正则化的半监督算法是半监督学习中的一类重要方法。近来的研究揭示了此类算法计算量庞大及对非高斯噪声敏感等不足。如不妥善解决，将使算法的学习效率大打折扣，阻碍半监督学习优势的发挥。本项目首先针对基于图正则化的半监督算法计算量大的弊端建立合理的模型假设，提出稀疏的基于图的半监督算法。然后，将信息论中熵的概念引入到基于图的半监督学习中。利用熵替代传统图正则化算法中的平方损失，提出能够有效处理非高斯分布噪声的算法，接下来从统计学习理论的角度出发，利用算子逼近技术，全面地分析图正则化算法的稀疏性、对非高斯分布噪声的稳定性及收敛性。并且，将算法应用于模式识别、信号处理等实际问题。项目以提高算法的稀疏性、稳定性及收敛性为目标，并将部分基础理论成果推广至应用技术层面，促进图正则化的半监督学习理论和应用的进一步深化和发展。

半监督学习是近年来机器学习、模式识别等领域的热点问题。基于图的半监督算法是半监督学习中的一类重要方法。传统的基于图的半监督算法具有计算量庞大、对非高斯噪声敏感与匮乏理论分析等弊端。其严重阻碍了半监督算法学习性能的提高。针对这些不足，本项目考虑设计新的基于图的半监督学习算法，提高算法的稀疏性、对非高斯分布噪声的稳定性与收敛性。.首先，针对图正则化半监督算法计算量庞大的弊端，考虑设计稀疏的图正则化半监督算法。目前大量的基于图的半监督算法模型利用图正则化因子挖掘海量未标记训练样本的数据结构，以此提高算法的推广性能。本项目考虑在传统的基于图的半监督算法框架中引入L1-正则化因子，构造新的图正则化半监督算法。通过L1-正则化因子使构造的半监督算法对训练样本进行选择，进而提高算法的稀疏性。.其次，考虑将信息理论中的相关熵函数与传统的基于图的半监督算法相结合，设计对非高斯分布噪声稳定的图正则化半监督算法。目前大量的基于图的半监督算法采用平方损失函数构造正则化体系。由于平方损失对高斯分布的依赖性，此类算法在处理非高斯分布的噪声问题时表现并不好。针对这一不足，本项目分三个方面来构造基于相关熵的图正则化半监督算法。首先采用信息理论中的相关熵函数构造图正则化的半监督算法模型。利用相关熵函数对非高斯分布噪声不敏感的特性，设计出对非高斯分布噪声稳定的图正则化半监督算法。接下来，利用统计学习理论中的误差分析技术对所设计算法进行理论分析，证明算法的稳定性。并且，建立算法的学习速率，证明算法的收敛性。.最后，通过实验验证所设计的图正则化半监督学习算法具有较好的稀疏性、能够有效处理非高斯分布的噪声问题。并且，将此类应用于模式识别及图像处理等实际问题。