Effects of Within-Modal Congruency, Cross-Modal Congruency, and Temporal Asynchrony on the Perception of Perceived Audio-Visual Distance

When making decisions as to whether or not to bind auditory and visual information, temporal, spatial and congruency factors all contribute to the acceptance or rejection of multi-modal unity. While many of these factors have been studied in isolation, it is important to examine how they interact in a dynamic setting, in addition to evaluating ideas about the intrinsic relation between audition and the processing of time, and vision and the processing of space. Four experiments are presented, placing auditory and visual stimuli in a competitive binding scenario, to compare the effects of temporal and spatial factors both within and between modalities. Results support the dominance of auditory factors in temporal decision-making, and visual factors in spatial decision-making, with additional evidence for the presence of visual looming. With respect to audio-visual binding, the findings indicate precedence for temporal factors, with reliance on congruency factors only when the stimulus pairings are temporally ambiguous.