VISUAL INPUT AIDS SELECTIVE SPEECH TRACKING AT A “COCKTAIL PARTY”

Our ability to attend to a particular conversation amidst competing input streams (e.g. other speakers), epitomized by the “cocktail party” problem, is remarkable. One neuronal mechanism proposed for achieving this feat is synchronization of low-frequency neural activity in auditory cortex (1-7Hz) to the temporal envelope of the attended stream (‘selective speech tracking'). Topdown attention may also aid this tracking by predicting upcoming temporal events in the stream. How these two mechanisms are integrated and consequently, how attended speech is selectively tracked, remains unresolved. We investigated whether congruent visual input of the attended speaker contributes to selective speech tracking in auditory cortex. Since lip and head movements during speech are highly correlated with the temporal envelope of speech, but precede them by ~150 ms, visual input provides excellent predictive cues for the timing of upcoming acoustics. We presented natural speech to participants while recording neuromagnetic activity. Speech segments were presented alone (single speaker) or concurrently with another speech segment (cocktail party). In addition, audio was either presented alone (A) or accompanied by a video of the speaker (AV). In the Single Speaker condition we found selective low-frequency speech tracking in both AV and A conditions. However, in the Cocktail Party condition selective speech tracking was only found when visual input was provided, but not when listening to a ‘cocktail party of voices'. These results suggest that under perceptually challenging situations such as the ‘cocktail party', visual input plays a key role in focusing attention and directly influences activity in auditory cortex