半监督深度学习算法及其在时序数据预测上的应用研究

Deep learning has achieved great successes in many areas, especially for image recognition and natural language understanding, but it also faces a big barrier while going further to general applications for the reason of requiring a huge amount of labeled data. Because for many real applications, it is quite difficult even impossible to collect such a big amount of well labeled samples for a particular task. This project will study semi-supervised learning, by combining traditional semi-supervised learning with recently developed deep learning, to train a deep network with high accuracy but just using a small amount of labeled samples with abundant easily available unlabeled samples. More specifically, the project will focus on algorithmic theories and application extensions. For theory part, it will study a new semi-supervised learning model by combining the whole data distribution information with an individual sample's content information; it will investigate the joint problem of samples similarity measure and the hidden feature representation by designing two simultaneously optimized semi-supervised learning models; it will develop sample similarity based new efficient stochastic gradient descending (SGD) algorithm to make full use of semi-supervised learning characteristics. For the application part, the project will study semi-supervised deep learning based multi-type vessels' trajectory modeling and long term path prediction, which has a key role in modern maritime intelligence; it will improve pulse networks' structure analysis and visualization, which will provide a basis for future deeper researches on predicting time series by spiking neural network.

深度学习在以图像识别和自然语言理解为代表的诸多领域取代了巨大成功，但深度学习对海量标记样本的依赖成为其走向普及应用的一大障碍，因为在许多实际应用中收集如此大量的标记样本很困难甚至无法实现。本项目拟结合半监督学习思想研究半监督深度学习算法，以实现利用较少标记样本和大量容易获得的无标记样本就能训练出具有较高精度的深度神经网络，具体包括算法理论与应用拓展两部分：1. 算法理论部分，本项目将研究利用数据整体信息和样本个体信息的新型半监督深度学习模型；研究样本相似性度量和隐特征表示的同步学习问题；结合半监督学习特点，开发新型基于样本相似性关系的随机梯度下降算法。2. 应用拓展部分，本项目将研究基于半监督深度学习的多类型船只轨迹（一种时序数据）混合建模与远期预测，其在现代智能海洋交通中具有关键应用；研究半监督学习用于脉冲网络的结构分析和可视化，这为以后更深入研究利用脉冲神经网络做时序数据预测奠定基础。

本项目主要研究了半监督深度学习算法，即如何利用少量的标记样本和大量相对容易获取的无标记样本训练深度学习模型，从而降低深度学习在很多应用中的门槛。本项目的主要研究内容和取得的成果如下：（1）针对图论半监督学习算法中相似性度量对最终结果影响较大问题，研究了如何对复杂数据（如自然图像）通过深度网络自适应的学习来更准确地衡量样本之间的相似性，从而提高无标记样本分布信息提取的准确性，并在此基础上构建了隐特征学习和相似性学习联合的网络架构，提出基于扩展拉普拉斯正则化的端到端半监督深度学习算法；（2）针对无标记样本的伪标记存在噪声且在算法训练过程中可能造成误导的问题，设计了置信度模块学习真实类别概率(TCP)监督的置信度信息，并提出了基于TCP感知的局部一致性优化策略半监督深度学习算法，降低了伪标记中不可靠信息对于网络学习的影响；（3）针对半监督深度学习算法对输入图数据结构的严重依赖以及图边缘广泛存在的连接噪声对算法结果影响较大问题，研究了如何在隐空间中动态学习锚点图结构来指导网络特征学习，提出了利用样本整体特征和个体特征相结合的半监督学习算法，有效提升了算法的稳定性、精确度和训练速度。