注意对听觉引发的视觉增强效应的调节机制

Sound can enhance visual perception. Specifically, sound that temporally and semantically matches the visual target can improve visual detection (Noesselt et al., 2010; Thomas & Shiffrar, 2010), decrease interocular suppression (Alsius & Munhall, 2013; Tan & Yeh, 2015), and increase the dominance duration during binocular rivalry (Conrad, Bartels, Kleiner, & Noppeney, 2010; van Ee, van Boxtel, Parker, & Alais, 2009). But there were tremendous differences in how studies manipulated attention when such sound-driven visual facilitation was observed. In some cases, attending to sound was necessary for visual facilitation (van Ee et al., 2009), while in other cases, visual facilitation appeared even when sound was intentionally ignored or inhibited (Alsius & Munhall, 2013; Conrad et al., 2010). Generally, most studies just described the sound as irrelevant to the task or even never mentioned about it (Noesselt et al., 2010; Tan & Yeh, 2015; Thomas & Shiffrar, 2010). However, such instruction made it impossible to measure to what extent observers attended to the sound..Thus it remains unclear whether and how attention modulates the auditory-driven visual facilitation. For example, whether such facilitation depends on attending to the sound, whether its dependence on attending to the sound changes from situation to situation, and whether there are some other factors that also play a role in how attention modulates the auditory-driven visual facilitation. The present program is designed to address these issues with systematic manipulation of attention. .Elucidating the relationship between attention and auditory-driven visual facilitation can explain the inconsistencies among previous studies, deepen our understanding on the processes of audiovisual integration, and provide some tips on how to manipulate attention to guarantee the visual facilitation driven by sound in various situations.

听觉可以对视觉加工起到增强作用，具体表现为与视觉刺激在时间上同步或意义上一致的听觉刺激能够有助于视觉检测、帮助视觉刺激更早突破抑制进入意识或者在意识中停留更长的时间。但是对比这些研究，可以发现它们在报告听觉对视觉的增强效应时，对注意的操纵有很大的差异。既有研究表明注意听觉刺激是增强效应发生的必要条件，也有研究认为不需要注意甚至抑制听觉刺激就足以引发增强效应。另外的研究则没有量化被试对听觉刺激的注意程度。.因此，本项目旨在系统地揭示注意与听觉引发的视觉增强效应的关系：它是否是这种增强效应发生的必要条件，又是否会随着任务情境的变化而变得不再必要，是否存在其它因素影响注意对听觉引发的视觉增强的调节机制。.项目的研究成果既可以解释现有研究的差异，也有助于深入理解视听整合的加工过程，还可以为以后的研究提供设计与指导语的参考。

多感觉整合可以增强单通道信息的加工。与视觉刺激在时间上同步或意义上一致的听觉刺激能够有助于视觉检测 (Noesselt et al., 2010; Thomas & Shiffrar, 2010)、帮助视觉刺激更早突破抑制进入意识 (Alsius & Munhall, 2013; Tan & Yeh, 2015) 或者在意识中停留更长的时间 (Conrad, Bartels, Kleiner, & Noppeney, 2010; van Ee, van Boxtel, Parker, & Alais, 2009)。本项目关注两个问题。第一，视听整合是否能够增强对于连续视觉刺激的选择性注意，这种增强是否自动捕获的、其发生的必要条件和神经基础。第二，视听整合是否能够调节瞳孔的光反射（强光导致瞳孔收缩的反射），这种调节是否受到注意影响。研究一要求被试完成一个视觉检测任务，同时利用脑电技术记录被试完成任务时的神经活动。研究发现，同步的听觉节律能够增强对目标视觉节律的选择性注意，提高了行为检测的正确率和视觉神经振荡的振幅；并且，在比较一致的听觉节律与没有听觉输入时，枕顶视觉电极与前额听觉电极间的功能连接越强，行为辨别的正确率就越高。结果表明，视听整合对视觉选择性注意的增强可能通过视听皮层的相位同步化来实现。研究二利用周期性明暗变化的视觉刺激来诱发瞳孔光反射，并操纵听觉刺激与视觉刺激的时间同步性。结果发现，与视觉刺激同步呈现的听觉刺激可以抑制瞳孔光反射，并且这种抑制只发生在中央视野且不受注意负荷的调制。这提示我们视听整合对瞳孔光反射的调节，可能缘于视听整合对瞳孔光反射环路的直接抑制。