Audio-visual speech perception is special.

Jyrki Tuomainen; Tobias S Andersen; Kaisa Tiippana; Mikko Sams

doi:10.1016/j.cognition.2004.10.004

Home/Archive/PubMed

PubMed · 15833302

Audio-visual speech perception is special.

Jyrki Tuomainen·Tobias S Andersen·Kaisa Tiippana·Mikko Sams

View original source ↗Export metadata ↓

Abstract

In face-to-face conversation speech is perceived by ear and eye. We studied the prerequisites of audio-visual speech perception by using perceptually ambiguous sine wave replicas of natural speech as auditory stimuli. When the subjects were not aware that the auditory stimuli were speech, they showed only negligible integration of auditory and visual stimuli. When the same subjects learned to perceive the same auditory stimuli as speech, they integrated the auditory and visual stimuli in a similar manner as natural speech. These results demonstrate the existence of a multisensory speech-specific mode of perception.

Explore related subjects

Association Learning Attention Discrimination Learning Humans Lipreading Phonetics Sound Spectrography Speech Acoustics Speech Perception Journal Article Research Support, Non-U.S. Gov't

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Jyrki Tuomainen, Tobias S Andersen, Kaisa Tiippana, Mikko Sams. 2004-12-30. Audio-visual speech perception is special.. https://doi.org/10.1016/j.cognition.2004.10.004

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2007

Associative learning and the control of human dietary behavior.

Most of our food likes and disliked are learned. Relevant forms of associative learning have been identified in animals. However, observations of the same associative processes are relatively scarce in humans. The first section of this paper outlines reasons why this might be the case. Emphasis is placed on recent research exploring individual differences and the importance or otherwise of hunger and contingency awareness. The second section briefly considers the effect of learning on meal size, and the author revisits the question of how learned associations might come to influence energy intake in humans.

Association Learning↗

PubMed2007

Network model of decreased context utilization in autism spectrum disorder.

Individuals with autism spectrum disorders (ASD) demonstrate impaired utilization of context, which allows for superior performance on the "false memory" task. We report the application of a simplified parallel distributed processing model of context utilization to the false memory task. For individuals without ASD, experiments support a model wherein presentation of one word, e.g., ''apple,'' strongly activates the neighboring nodes of closely related words such as ''fruit,'' ''tree,'' whereas in ASD these neighboring nodes are relatively less activated. We demonstrate this model to be consistent with the superior performance on recognition testing on the false memory test, but not on free recall. This may have an anatomic basis in diminished hippocampal neuronal arborization and the abnormal minicolumnar pathology in ASD.

Association Learning↗

PubMed2007

Rapid learning of syllable classes from a perceptually continuous speech stream.

To learn a language, speakers must learn its words and rules from fluent speech; in particular, they must learn dependencies among linguistic classes. We show that when familiarized with a short artificial, subliminally bracketed stream, participants can learn relations about the structure of its words, which specify the classes of syllables occurring in first and last word positions. By studying the effect of familiarization length, we compared the general predictions of associative theories of learning and those of models postulating separate mechanisms for quickly extracting the word structure and for tracking the syllable distribution in the stream. As predicted by the dual-mechanism model, the preference for structurally correct items was negatively correlated with the familiarization length. This result is difficult to explain by purely associative schemes; an extensive set of neural network simulations confirmed this difficulty. Still, we show that powerful statistical computations operating on the stream are available to our participants, as they are sensitive to co-occurrence statistics among non-adjacent syllables. We suggest that different learning mechanisms analyze speech on-line: A rapid mechanism extracting structural information about the stream, and a slower mechanism detecting statistical regularities among the items occurring in it.

Association Learning↗