基于眼动追踪的智能教学Agent情感交互与认知支持模型研究

The space and time gap between instructors and student would easily cause lack of emotional information in distance education, which might lead to students' helpless and lost mood during learning process. In order to deal with this problem, Pedagogical Agent arises at the historic moment, which present in the computer assisted learning process with fresh personification image that is beneficial to increase learners' interest and reduce cognitive loads. However, there are two kinds of defects in most of the exist pedagogical agents: one is they only have sigal-direction emotional transmission function and lack of interaction; the second fault is their cognitive function is weak and the effect on enhancing learning is not stable. Some learners even thank that the agent interfere with their normal browsing and reading. Based on this, this project attempts to use eye movement tracking technique to add "eyes" on pedagogical agents, in order to monitor distance learners' eye movement states and their sight path. The eye movement tracking data would interact with learner's face expression data which was achieved by camera, so as to guarantee the results of emotional identification are correct. The sight path tracking data would go with the learning contents and be coded to indentify learner's cognitive status. The emotion and cognition coupling model would be set up to deduce the optimal emotional feedback (Agent face expression, language, motion) and cognitive feedback (knowledge related hints, learning method recommendations). Agent prototype would be established and applied in a systematic educational experiment, in order to analyze the cognitive processing mechanism of distance learning with intelligent pedagogical agent.

师生和生生间的时空分离状态容易导致远程学习过程中情感信息的缺失，使学习者产生孤独无助和迷茫懒散的情绪。为了弥补这一缺陷，教学Agent应运而生，以鲜活的拟人形象出现在计算机辅助教学过程中，有利于增加趣味性和减轻认知负荷。然而现有的Agent普遍存在两方面问题：一是只具备单向的情感传输功能而缺乏情感交互性；二是认知推断功能薄弱，对学习效果的促进作用不稳定，一些学习者甚至认为Agent干扰了他们正常的浏览和阅读。基于此，本项目尝试利用眼动追踪技术为教学Agent添上"眼睛"，以实时监测学习者的眼动状态和视线路径，与摄像头同步监测到的表情状态相互验证和修正，编码识别出学习者的情感状态和认知状态，建立情感与认知耦合模型，推断并呈现Agent最优的情感反馈（表情、语言、动作）和认知反馈（知识点相关提示、学习方法建议）。建立Agent原型并开展教学实验分析其对远程学习者认知加工过程的作用机制。

情感与认知状态的准确识别是实现远程学习者与教学Agent有效互动的基础。学习者的眼动追踪数据是判断学习者实时状态的重要指标，也是学习者与Agent进行情感交互的重要依据。本项目构建了的基于智能Agent的远程学习者情感与认知识别模型。该模型主要在两个方面作了创新性的尝试：第一，将眼动追踪与表情识别技术相结合，采用眼动追踪与表情监控迭代识别的方法，有助于提高模式识别的精确度，为人工智能情感识别研究提供了新途径和新思路。第二，考虑到学习者的情感状态与认知状态是相互作用和相互影响的，采用将情感与认知识别过程相耦合的方法，优化Agent的反馈，使其在提高学习者的兴趣和愉悦感的同时，更有针对性地提供知识点提示和学习方法建议，促进学习者的认知发展。项目组共发表学术论文12篇，其中7篇被SSCI和SCI检索，4篇被CSSCI检索。所取得的主要成果有：（1）建立了基于智能Agent的远程学习者情感状态识别模型，（2）梳理了基于虚拟助理的远程学习支持服务及技术难点，（3）基于眼动追踪数据构建了预测在线学习者语文阅读能力的计算模型，（4）利用眼动追踪和左右阅读范式对学习者阅读水平进行测定，探讨学习者阅读能力识别的眼动指标标定问题；（5）在分析眼动指标的过程中，我们还发现阅读水平不同的男女学习者在眼动模式上存在显著差异；（6）完成了以问题为导向的智能导师系统的模型设计和效果验证；（7）设计了教学代理的情感与认知反馈方案；（8）将基于眼动识别的智能教学代理整合进智慧教育环境中。