移动端深度学习系统中的隐私保护研究

With the compelling advancement of deep learning driven by big data, billions of mobile devices have become the entries of all sorts of data whereas the cloud serves as the powerful computational engine. However, in such a mobile cloud computing infrastructure, the problem of user privacy leak has raised wide attention. Conventional privacy-preserving mechanisms, unaware of the application performance or the properties of neural networks, are high in energy cost and latency, while significantly degrade the learning accuracy of the model. This project revisits the privacy definitions, takes into account the properties of neural networks to propose or optimize privacy-preserving on-device deep learning mechanisms. The project approaches the problem from three perspectives: 1. In light of the subspace of neural network features, we characterize the relation between the model utility and its privacy, as well as seek the sweet spot in the trade-off between the two. 2. We introduce intermediate neural network features satisfying the property of k-anonymity, and modify operations or modules to accommodate such features to produce accurate results. 3. We propose to model the game between the data owner and the adversary who intends to acquire its data as multi-agent adversarial learning, and search for the optimal policy of the data owner to compose a local neural network with the consideration of application performance. The optimal policy yields intermediate features that are robust to adversarial attacks while producing accurate learning results when feeding to the cloud. We study privacy-preserving mechanisms from different aspects of neural networks, the results of which will serve as the basis for privacy-preserving on-device deep learning.

随着以大数据为驱动的深度学习算法的推广，移动设备成为各类数据的入口，云端则为深度学习提供算力与平台。而在移动端和云端的联合部署中，用户隐私的泄露已成为被广泛关注的问题。传统的隐私保护机制缺少对移动设备和神经网络本身特点的考虑，多以高能耗、高时延、低学习精度为代价。本项目拟在移动云计算框架内，以隐私定义为出发点，结合深度神经网络的特性，提出并优化隐私保护机制，侧重三方面：1.刻画网络特征对结果精度的差异化影响，分析隐私条件限制，求解数据隐私性与模型精度之间的最优平衡点；2.引入复数网络概念，设计网络算子模块，保证输出结果准确性与输入数据匿名性；3.建立网络特征对抗学习模型，在满足移动端运算性能的前提下优化网络结构选择策略，实现中层表达的隐私可保护、结果可计算。此三方面从神经网络不同层面系统性研究用户数据隐私保护机制，形成一套有机整体，将为移动深度学习系统的推广普及提供有益思考和探索。

随着以大数据为驱动的深度学习算法的推广，移动设备成为各类数据的入口，云端则为深度学习提供算力与平台。而在移动端和云端的联合部署中，用户隐私的泄露已成为被广泛关注的问题。传统的隐私保护机制缺少对移动设备和神经网络本身特点的考虑，多以高能耗、高时延、低学习精度为代价。本项目在移动云计算框架内，以隐私定义为出发点，结合深度神经网络的特性，提出并优化隐私保护机制，侧重三方面：1.从差分隐私定义出发，分析深度模型上的隐私条件限制，求解数据隐私性与模型精度之间的最优平衡点；2.引入复数网络概念，设计网络算子模块，保证输出结果准确性与输入数据匿名性；3.建立网络特征对抗学习模型，在满足移动端运算性能的前提下优化网络结构选择策略，实现中层表达的隐私可保护、结果可计算。此三方面从神经网络不同层面系统性研究用户数据隐私保护机制，形成一套有机整体，将为移动深度学习系统的推广普及提供有益思考和探索。