Analysis of speechreading cues using an interleaved technique.

Abstract

A method of locating visible differences between "homophenous" phonemes has been developed. Alternating frames from two video-recorded VCV utterances that differ only in the consonant are interleaved. The output contains odd frames of one utterance and even frames of the other. Presentation of these frames at 30 frames/sec results in a video recording that contains areas of jitter in those regions of the face that are not in identical positions at the same time during the production of the two utterances. The areas of jitter indicate the points of interest for detailed measurement. For homophenous VCV pairs involving /p/, /b/, and /m/, interleaving resulted in strong jitter in the lower lip and cheeks. There was no jitter when two utterances of the same phoneme were interleaved. A concurrent perceptual study showed that good speechreaders were able to discriminate between members of a homophenous pair in a forced-choice experiment.