视频人体意外动作识别与定位方法研究

Video action analysis and understanding is a key research interest in computer vision. It has developed from early simple action recognition under restricted conditions to complex action recognition in real natural scenes. Recent researches stay at the perception of action appearance, which is difficult to understand the human psychological concept of action intention and cannot recognize and localize human unintentional actions. In order to solve the problem of model generalization caused by spurious correlation of human unintentional action in the real natural videos, this project proposes a method for recognizing and localizing unintentional action with counterfactual examples. It enables the model to learn the causal logic and achieve stable predictions by introducing causal inference theory to design a causal model, retrieving counterfactual examples to generate the external vision knowledge, constructing the contrastive learning framework between counterfactual and factual examples. To satisfy the generalization requirements caused by the uncertain labels of unintentional actions in the real natural videos, this project proposes a method for recognizing and localizing unintentional actions based on the probabilistic temporal-labels. It guides the model training and improves the generalization performance by introducing an attention mechanism to design the online aggregation learning framework, modeling uncertain labels to generate probabilistic temporal-labels, and learning reliable supervision information. The expected results of this project will promote the popularization and application of the technology of video action analysis and understanding in the fields of health monitoring, intelligent security, and autonomous driving.

视频动作分析与理解是计算机视觉领域的重点研究方向，从早期受限条件下的简单动作识别发展到真实自然场景下的复杂动作识别。目前的研究工作停留于动作表观的感知，很难理解“动作意图”这一人类心理学概念，无法实现人体意外动作的识别与定位。针对真实自然场景下人体意外动作的虚假关联引发的模型泛化问题，本项目提出基于反事实样本的人体意外动作识别与定位方法，通过引入因果推断理论以设计因果模型，检索反事实样本以生成外部视觉知识，建立事实与反事实对比学习框架，使得模型学得因果逻辑并实现稳定预测。针对人体意外动作的不确定标注引发的模型泛化需求，本项目提出基于概率时间标注的人体意外动作识别与定位方法，通过引入注意机制以设计在线聚合学习框架，建模不确定标注以生成概率时间标注，学习可靠的监督信号，指导模型训练并提升泛化性能。本项目的预期研究成果将推动视频动作分析与理解技术在健康监测、智能安防以及自动驾驶等领域的推广应用。

视频人体动作分析是计算机视觉领域的重点研究方向，具有广阔的应用前景，是视觉监控、数字媒体、人机交互等领域的基础性关键技术。目前，视频人体动作分析领域的模型范式在纯视觉层面的性能不断取得突破且日渐趋于饱和，其发展正面临来自理解人体动作认知建模的高层技术挑战。本项目立足于人体意外动作定位任务，从虚假关联和不确定性两方面挑战展开研究。针对计算机不具备理解人类动作意图的知识、误导模型学习虚假关联这一问题，提出基于反事实样本的意外动作定位方法，该方法引入因果推断理论设计因果模型、检索反事实样本生成外部视觉知识、建立事实与反事实对比学习框架，使得模型可学得因果逻辑、消除虚假关联、实现稳定预测。针对人体动作意向性标注存在不确定性、导致模型缺少可靠的监督信号这一问题，提出基于概率时间建模的意外动作定位方法，该方法在线学习人体意外动作监督信号、表征动作从有意转变为意外的渐进性和标注的不确定性，构建多尺度监督粒度、联合指导模型训练、解决标注不确定性引发的模型泛化能力差的问题。本项目通过解决视频人体动作在表征空间中的可解释性这一科学问题，从人体动作认知建模这一层面突破现有方法在纯视觉层面研究人体动作的局限，赋予模型在真实自然场景下理解动作意图、评价他人行为的能力，促进其在健康监测、自动驾驶、竞技体育等领域的推广应用。