适用于盲人儿童的基于听觉和触觉的多模态交互式实物编程环境研究及设计

While the trend of kids coding has boosted globally, the current highly visual kids-coding tools are not suitable for blind children, causing an unequal situation for them to learn coding and computational thinking. There currently lacks sufficient research on how to make coding education suitable for this group of children, and there is no specific learning tool for them all over the world. Targeting blind children from 7 to 10 years-old, this project aims to investigate the user model and the interaction model of blind children coding, and develop an auditory- and tactile-based multimodal tangible programming environment. There are five main parts in this project: Firstly, we will investigate the perceptual and the cognitive ability of blind kids through field study and participatory design workshop; we then conceptualize the user model and the interaction model of blind children learning coding; Based on the theoretical models and the design requirement of “low floor high ceiling and wide walls”, we will design and develop a set of auditory- and tactile-based tangible coding input and output modules, and customizable tangible coding sprites; and then develop a list of integrated coding environments by combining different input and output modules; Finally, we will conduct between-subject user studies, to compare the user experience and the learning outcome of different prototypes. In a macro scale, the proposed system will be also suitable for non-blind children, making it possible for blind and non-blind children to collaboratively learn coding together. With this project, we aim to provide solutions on the problem of blind children learning programming, and make children coding education more popularized.

儿童编程教育已经在国内外受到越来越多的关注。但是，现今流行的基于视觉的儿童编程环境对于盲童并不适用，使盲童得不到同等的学习条件和机会。本项目以7-10岁盲童作为目标用户，先针对粤港澳地区的盲童做试点，研究基于听觉与触觉的多模态交互式实物儿童编程环境。本项目将开展以下几方面的工作：实地调研盲童的感知和认知能力以及对计算机编程的直观认识；建立盲童编程的用户模型及交互模型；根据“低地板高天花板宽墙壁”的特点，研究开发基于听觉与触觉的程序输入输出模块以及可自定义的实物程序控件；整合开发多模态交互式实物编程环境系统；对不同的系统原型进行用户评估，对比用户体验和学习效果。广义上，本项目拟研发的基于听觉和触觉的实物编程环境也适用于非盲儿童，为盲人儿童和非盲儿童协同学习提供可能。本项目的研究工作将解决盲童编程环境设计和开发面临的关键问题，使儿童计算机编程教育得到更广泛的普及。

儿童编程教育已经在国内外受到越来越多的关注。但是，现今流行的基于视觉的儿童编程环境对于盲童并不适用，使盲童得不到同等的学习条件和机会。本项目以7-10岁盲童作为目标用户，研究基于听觉与触觉的多模态交互式实物儿童编程环境及工具。本项目开展了以下几方面的工作：通过实地考察建立盲童编程的用户模型及交互模型；根据“低地板高天花板宽墙壁”的特点，研究开发基于听觉与触觉的程序输入输出模块以及可自定义的实物程序控件及学习平台；对不同的系统原型进行用户评估，对比用户体验和学习效果;探索适用于盲人的多模态前沿技术。通过实地考察，我们发现盲童主要通过触摸和体感来学习不同的知识概念。因此，我们针对性地设计了基.于音乐的用于视障儿童学习计算机编程的实体编程模块CodeRhythm，用户实验体现了系统优良的用户体验。在前沿探索方面，我们实验了不同的触觉反馈技术，包括热感、震感、压感等等，并探索了它们在不同领域的针对盲人人机交互的应用。项目成果发表于人机交互相关领域的优秀会议及期刊，部分获得优秀论文奖。