无线感知网络分布式协同学习稀疏核学习机的理论和算法研究

Wireless sensor networks are characterized by constrains on energy and bandwidth, which limits the communication overhead. Hence distributed learning in wireless sensor networks to achieve the global optimal classifier or regression estimator by interchanging information and coordination among neighboring nodes, has attracted much attention from scholars and practitioners in different fields recently. In this project, we study the energy-efficient distributed algorithms for learning the L1 regularization sparse kernel methods in wireless sensor networks based on distributed optimization theory and statistical machine learning theory. According to the properties of wireless sensor networks and the L1 regularization loss functions of kernel methods, we first investigate predicting the active set during learning and the mechanism of information transmission and coordination among neighboring nodes in wireless sensor networks to reduce communication overhead and aim to propose the energy-efficient local message-passing algorithms to obtain an optimal or close-to-optimal classifier or regression estimator while minimizing the required amount of information exchange among neighboring sensors. Then on the basis of the above research and filed theory, dynamic energy balancing algorithms are investigated, which choose sensors with more energy to perform more tasks during learning to prolong the lifetime of wireless sensor networks. In addition, we analyze the proposed local message-passing algorithms in terms of energy cost, bandwidth cost, convergence, convergence rate and validate them in wireless sensor networks. And then compare the learned classifiers or regression estimators in terms of sparsity, prediction accuracy and test time on real data sets. The study of this project is meaningful and significant for efficient learning kernel methods in wireless sensor networks and will promote the flouring of the distributed learning in wireless sensor networks and the practical applications of wireless sensor networks.

在无线感知网络中,能源和带宽资源都非常宝贵,因此通过相邻节点间的传输共享数据和相互协作在网内分布式协同学习分类器、回归机的方法已成为无线感知网络领域新的研究热点.本课题在分布式优化理论的框架下,基于统计机器学习理论,根据无线感知网络和L1正则化稀疏核方法的特点,研究有效集的预测、数据传输和共享方式、相邻感知节点间的协作对学习算法能量消耗、带宽占用、收敛性、收敛速度等的影响，旨在提出能充分挖掘 L1正则化稀疏核方法的特性及节点间协作关系的低功耗、低通信代价的分布式协同学习算法,并在此基础上基于场理论,研究在学习过程中网内各感知节点的能量损耗情况,提出能动态平衡能量损耗的学习算法.在无线感知网络平台上，用实测数据分析所提算法在能源消耗、带宽占用、收敛速度等方面的表现，并测试学习到的L1正则化稀疏核学习机的稀疏性、预测精度及预测时间.本课题的实施对无线感知网络分布式协同学习的研究具有积极促进作用

针对无线传感器网络中节点能量非常宝贵、带宽资源有限、网络拓扑动态变化等特点，基于分布式优化和统计机器学习理论研究了通过节点间的传输共享和相互协作来低功耗分布式学习核学习机的理论和算法。具体包括：研究了经典核方法SVM（Support Vector Machine）基于线性核函数的分布式学习问题的分解策略、平均一致性的协作方式对分布式算法收敛速度、收敛性及传输数据量的影响，提出了两种基于平均一致性的分布式SVM学习算法AC-DSVM和1-AC-DSVM，与现有算法相比AC-DSVM能收敛到更优结果，而1-AC-DSVM具有较好收敛性的同时，在收敛速度和数据传输量上也表现出了显著优势；建模了L1正则的核最小平方误差（Kernel Minimum Square Error, KMSE）学习问题，研究了三种启发式协作方式对算法收敛速度、数据传输量、模型稀疏度及模型精度等的影响；研究了基于增广拉格朗日法和并行投影法的L1正则KMSE分布式学习算法，并研究了这两种算法在同步更新方式和异步更新方式下的迭代速度、数据传输量、模型稀疏度及模型精度，所提算法在数据传输量、模型精度上都有明显优势；研究了L1正则KMSE问题的增量式训练方法，基于交替乘子法提出一种增量式训练核学习机的算法L1-KMSE-Increm，与现有算法相比，L1-KMSE-Increm算法具有预测时计算代价和模型传递代价低的优点；研究了基于Markov链随机传输的协作方式，提出了一种L1正则化核学习机的分布式增量训练算法DI-L1KMSE，与现有算法相比，DI-L1KMSE在数据传输量上具有明显优势；研究了基于节点能量动态平衡的L1正则化学习机分布式训练策略；为验证上述研究中提出的方法和策略，自主设计和研制了实验平台，并验证和对比了相关算法的性能。