P Duchnowski | PubMed Health

PubMed2000

Development of speechreading supplements based on automatic speech recognition.

In manual-cued speech (MCS) a speaker produces hand gestures to resolve ambiguities among speech elements that are often confused by speechreaders. The shape of the hand distinguishes among consonants; the position of the hand relative to the face distinguishes among vowels. Experienced receivers of MCS achieve nearly perfect reception of everyday connected speech. MCS has been taught to very young deaf children and greatly facilitates language learning, communication, and general education. This manuscript describes a system that can produce a form of cued speech automatically in real time and reports on its evaluation by trained receivers of MCS. Cues are derived by a hidden markov models (HMM)-based speaker-dependent phonetic speech recognizer that uses context-dependent phone models and are presented visually by superimposing animated handshapes on the face of the talker. The benefit provided by these cues strongly depends on articulation of hand movements and on precise synchronization of the actions of the hands and the face. Using the system reported here, experienced cue receivers can recognize roughly two-thirds of the keywords in cued low-context sentences correctly, compared to roughly one-third by speechreading alone (SA). The practical significance of these improvements is to support fairly normal rates of reception of conversational speech, a task that is often difficult via SA.

Adult↗

PubMed1995

Villchur revisited: another look at automatic gain control simulation of recruiting hearing loss.

An algorithm to simulate the effects of sensorineural hearing impairment on speech reception was investigated. Like that described by Villchur [J. Acoust. Soc. Am. 62, 665-674 (1977)], this simulation employs automatic gain control in independent frequency bands to reproduce the elevated audibility thresholds and loudness recruitment that are characteristic of this type of loss. In the present implementation, band gains are controlled in an effort to simulate loudness recruitment directly, using recruitment functions that depend only on the magnitude of hearing loss in the band. In a preliminary evaluation, two normal-hearing subjects listened to the simulation matched to hearing losses studied previously [Zurek and Delhorne, J. Acoust. Soc. Am. 82, 1548-1559 (1987)] with noise-masking simulations. This evaluation indicated that the present automatic gain control simulation yielded scores roughly similar to those of both the hearing-impaired listeners and the masked-normal listeners. In the more-detailed evaluation, the performance of three listeners with severe sensorineural hearing loss on several speech intelligibility tests was compared to that of normal-hearing subjects listening to the output of the simulation. These tests included consonant-vowel syllable identification and sentence keyword identification for several combinations of speech-to-noise ratio, frequency-gain characteristic, and overall level. Generally, the simulation algorithm reproduced speech intelligibility well, though there was a clear trend for the simulation to result in better intelligibility than observed for impaired listeners when high-frequency emphasis placed more of the speech spectrum above threshold at higher frequencies. Also, the hearing-impaired listener with the greatest loss showed the largest discrepancies with the simulation. Overall, however, the simulation provides a very good approximation to speech reception by hearing-impaired listeners. The results of this study, together with previous studies of noise-making simulation, suggest that threshold elevation and recruitment, which are necessary features of a simulation of cochlear hearing loss, can also be largely sufficient for simulating the speech-reception performance of listeners with moderate to severe hearing impairments.

Adult↗