[Intermodal timing cues for audio-visual speech recognition].

Masahiro Hashimoto; Masaharu Kumashiro

doi:10.7888/juoeh.26.215

Home/Archive/PubMed

PubMed · 15244074

[Intermodal timing cues for audio-visual speech recognition].

Masahiro Hashimoto·Masaharu Kumashiro

View original source ↗Export metadata ↓

Abstract

The purpose of this study was to investigate the limitations of lip-reading advantages for Japanese young adults by desynchronizing visual and auditory information in speech. In the experiment, audio-visual speech stimuli were presented under the six test conditions: audio-alone, and audio-visually with either 0, 60, 120, 240 or 480 ms of audio delay. The stimuli were the video recordings of a face of a female Japanese speaking long and short Japanese sentences. The intelligibility of the audio-visual stimuli was measured as a function of audio delays in sixteen untrained young subjects. Speech intelligibility under the audio-delay condition of less than 120 ms was significantly better than that under the audio-alone condition. On the other hand, the delay of 120 ms corresponded to the mean mora duration measured for the audio stimuli. The results implied that audio delays of up to 120 ms would not disrupt lip-reading advantage, because visual and auditory information in speech seemed to be integrated on a syllabic time scale. Potential applications of this research include noisy workplace in which a worker must extract relevant speech from all the other competing noises.

Explore related subjects

Acoustic Stimulation Adult Auditory Perception Cues Humans Lipreading Male Noise, Occupational Photic Stimulation Speech Perception Time Visual Perception English Abstract Journal Article

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Masahiro Hashimoto, Masaharu Kumashiro. 2004-06-01. [Intermodal timing cues for audio-visual speech recognition].. https://doi.org/10.7888/juoeh.26.215

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2008

Sensory filtering phenomenology in PTSD.

Disrupted sensory filtering, or problems with suppressing irrelevant environmental sensory stimuli, has been reported in individuals with posttraumatic stress disorder (PTSD). However, the relationship of sensory filtering deficits to specific PTSD symptoms versus an association with general trauma exposure is unclear. These relationships were examined by administering self-report measures of trauma exposure, PTSD, and sensory gating phenomenology to undergraduate participants with PTSD (n=32), with trauma history but without PTSD (n=144), and with minimal trauma history (n=153). Subjects with PTSD reported greater filtering disruption than individuals in the trauma only and low trauma groups, who did not differ. Individuals endorsing reexperiencing and numbing symptoms, and females endorsing hypervigilance, reported disrupted sensory filtering phenomenology. These results suggest that impaired filtering differentiates between individuals with PTSD symptoms and asymptomatic individuals exposed to multiple traumas and low-trauma controls.

Acoustic Stimulation↗

PubMed2007

Hypoglycemia reduces the blood-oxygenation level dependent signal in primary auditory and visual cortex: a functional magnetic resonance imaging study.

Studies of the effects of hypoglycemia on the brain using neurocognitive testing have suggested that mainly complex functions subserved by secondary and tertiary cortex are affected by mild to moderate hypoglycemia and that intensively treated patients with Type I diabetes mellitus (T1DM) may have altered sensitivity to the central nervous system effects of hypoglycemia. Functional magnetic resonance imaging provides a sensitive, regionally-specific probe of possible neurophysiologic changes related to hypoglycemia in the brain. Eleven intensively-treated T1DM patients and 11 matched non-diabetic controls took part in a 2-day protocol in which functional magnetic resonance imaging (MRI) was used to measure changes in the patterns of brain activation produced by simple auditory and visual stimuli in different conditions. On one day, participants were euglycemic the entire time. On the other day, an initial 50-min euglycemic period was followed by a 50-min hypoglycemic period. Results indicated that hypoglycemia reduced the amplitude of the blood-oxygenation level dependent response in primary auditory and visual cortex to simple auditory and visual stimuli. The latency and duration of the transient hemodynamic response function were not affected. Responses to hypoglycemia were similar in diabetic and non-diabetic participants. These results suggest that mild to moderate hypoglycemia may alter the balance of blood flow and oxygen extraction when glucose levels are lowered. Intensively-treated T1DM, with its attendant frequent hypoglycemic episodes, did not seem to alter hypoglycemic responses in primary visual and auditory cortex.

Acoustic Stimulation↗

PubMed2007

Song discrimination learning in zebra finches induces highly divergent responses to novel songs.

Perceptual biases can shape the evolution of signal form. Understanding the origin and direction of such biases is therefore crucial for understanding signal evolution. Many animals learn about species-specific signals. Discrimination learning using simple stimuli varying in one dimension (e.g. amplitude, wavelength) can result in perceptual biases with preferences for specific novel stimuli, depending on the stimulus dimensions. We examine how this translates to discrimination learning involving complex communication signals; birdsongs. Zebra finches (Taeniopygia guttata) were trained to discriminate between two artificial songs, using a Go/No-Go procedure. The training songs in experiment 1 differed in the number of repeats of a particular element. The songs in experiment 2 differed in the position of an odd element in a series of repeated elements. We examined generalization patterns by presenting novel songs with more or fewer repeated elements (experiment 1), or with the odd element earlier or later in the repeated element sequence (experiment 2). Control birds were trained with only one song. The generalization curves obtained from (i) control birds, (ii) experimental birds in experiment 1, and (iii) experimental birds in experiment 2 showed large and systematic differences from each other. Birds in experiment 1, but not 2, responded more strongly to specific novel songs than to training songs, showing 'peak shift'. The outcome indicates that learning about communication signals may give rise to perceptual biases that may drive signal evolution.

Acoustic Stimulation↗