360度全景视频流式传输中的码率自适应机制研究

As a new type of multimedia content, 360-degree video, which provides a unique panorama viewing experience, is increasingly gaining users' attention. As a 360-degree video usually has much more pixels than a conventional video and a user can only have a limited viewport, streaming the whole frame, the current widely used approach, is wasteful in terms of bandwidth resource. Tiled-based streaming is thus proposed to selectively transmit a part of the frame. 360-degree video has also brought new challenges to the tiled-based streaming system. First, the user's quality-of-experience (QoE) is largely affected by his/her watching behavior during the playback as his/her Field-of-View (FoV) and Region-of-Interest (ROI) changes. Second, limited by their hardware capability, mobile terminals cannot well-support the processing of future multimedia content. To handle these challenges, in this project, we target to investigate the rate adaptation scheme in 360-degree video streaming. Based on the analysis and the prediction of the user behaviors, we will propose the mathematical model for user QoE. With the consideration of multiple factors, including user behavior and hardware capability, to optimize the user QoE, we will explore the design of rate adaptation algorithms through two approaches, Model Predictive Control and reinforcement learning. To sum up, the results of this project can provide better support to the multimedia services on mobile terminals, e.g., virtual reality, and has the great potential to be applicable to a wide range of future multimedia applications.

作为一种新兴的多媒体形式，360度视频提供了一种独特的全景观看体验，得到了越来越多的用户关注。由于360度全景视频相比于传统视频有更多的像素，而用户往往只有有限的视区，现行的传输完整帧的方式是比较浪费带宽资源的。所以图块化传输被提出，用来选择性地传输视频画面的一部分。对此，360度全景视频对其流式传输带来了新的挑战。首先，用户行为，反应在其视区和感兴趣区上，对用户体验质量有重要影响，其次，受限于硬件能力，移动终端已经呈现无法满足未来多媒体内容处理的趋势。在新的形势下，本项目拟研究360度全景视频流式传输中的码率自适应机制。从对用户行为分析和预测出发，对用户体验质量建立表达模型。以优化用户主观体验为目标，结合用户行为和硬件能力等因素，运用模型预测控制和强化学习的方法，探索码率自适应算法的设计。本项目的成功实施可以对虚拟现实等移动平台上的多媒体应用起到有力的支撑和推动作用，具有广阔的应用前景。

作为一种新兴的多媒体形式，360度视频提供了一种独特的全景观看体验，得到了越来越多的用户关注。由于360度全景视频相比于传统视频有更多的像素，而用户往往只有有限的视区，现行的传输完整帧的方式是比较浪费带宽资源的。所以图块化传输被提出，用来选择性地传输视频画面的一部分。对此，360度全景视频对其流式传输带来了新的挑战。首先，用户行为，反应在其视区和感兴趣区上，对用户体验质量有重要影响，其次，受限于硬件能力，移动终端已经呈现无法满足未来多媒体内容处理的趋势。在新的形势下，本项目拟研究360度全景视频流式传输中的码率自适应机制。从对用户行为分析和预测出发，对用户体验质量建立表达模型。以优化用户主观体验为目标，结合用户行为和硬件能力等因素，运用模型预测控制和强化学习等方法，探索码率自适应算法的设计。本项目的成功实施可以对虚拟现实等移动平台上的多媒体应用起到有力的支撑和推动作用，具有广阔的应用前景。