低阶特征组合效应对深度学习模型性能影响的研究

As a special machine learning method, deep learning can obtain high-level features directly from the data. Since there are extractions of high-level features, can we ignore the traditional processing of low-level features? This issue is the current research difficulty of machine learning. Therefore, this proposal starts with the evaluation of the combined effect of low-level features and studies its influence on deep learning models. The research mainly includes: 1) the evaluation of combined effect in low-level features; 2) the optimization of deep learning models based on feature combined effect; 3) deep edge calculation framework based on the model decomposition. Firstly, it is proposed to solve the problem that the feature combined effect lacks the overall evaluation. The calculation of the neighborhood correlated information entropy is proposed, which can measure the information of the feature subset as an entire part. This calculation can be used to evaluate low-level feature combined effects. Secondly, it is proposed to enhance the performance of some models of deep learning. The optimization of two types of deep learning models (DBN and CNN) are designed, and the selection of low-level features (relative to higher layer) is integrated into deep learning models. This design simplifies the network structure. Finally, it is proposed to overcome the limitations of computing and transmission capabilities at the edge of the Internet of Things. A complex deep learning model is deployed in the edge computing framework which forms a deep edge computing framework.

深度学习作为一种特殊的机器学习方法，可直接从数据中获取高阶特征。那么既然存在高阶特征的抽取，是否可以忽略传统的低阶特征处理方式？这一问题是当前机器学习领域的研究难点。因此，本项目从低阶特征的组合效应评价入手，研究其对深度学习模型的影响。研究主要包括：1）低阶特征组合效应的评价方法；2）基于特征组合效应的深度学习模型优化；3）基于模型分解的深度边缘计算框架。首先，拟解决特征组合效应缺乏整体评价的问题，提出邻域关联信息熵的计算，该计算可度量特征子集整体的信息量，用以评价低阶特征组合效应。其次，拟提升深度学习的部分模型的性能，设计两类深度学习模型（DBN和CNN）的优化过程，将低阶特征（相对于更高层）的筛选融入深度学习模型，简化网络结构。最后，拟克服物联网边缘端计算和传输能力的限制，在边缘计算框架中部署深度学习模型，形成深度边缘计算框架。

目前，深度学习在国家很多重点领域中提供了优秀的解决方案，如对物联网入侵检测、雷达组网干扰识别、声呐事件检测、城市能源预测等。 在深度学习模型中，数据质量和数据量是决定任务性能的两个因素。 本研究致力于利用高质量的低阶特征组合优化深度学习模型，同时提高深度网络的效率和精度。 本项目第一部分研究使用关联信息熵度量输入端不同特征子集的组合效果，将选择的特征子集作为深度学习模型的输入。提出一种基于特征组合效应的两阶段学习模型。 本项目第二部分研究将注意力机制的特征强化效果作为中间层低阶特征组合的依据，提出了图关联注意力机制和稀疏注意力机制。基于这两种注意力机制对中间层低阶特征的强化，构造了两种多变量时间序列深度预测模型。本项目第三部分研究主要针对低阶特征组合的应用，将其应用到边缘检测领域、声呐时间检测领域和服务组合领域。通过上述三部分的研究，本项目分析了在学习过程中对于特征的不同处理方式，寻找低阶特征与高阶特征的关联性。验证了精简后具有组合效应的数据集有助于深度模型的学习；验证了中间层低阶特征的权重对深度预测模型的影响。有助于学者继续研究深度学习模型学习过程中低阶特征到高阶特征的可解释性。