Automatic recognition of second language speech-in-noise

Seung Eun Kim; Bronya R. Chernyak; Olga Seleznova; Joseph Keshet; Matthew Goldrick; Ann R. Bradlow

doi:10.1121/10.0024877

Automatic recognition of second language speech-in-noise

Seung Eun Kim, Bronya R. Chernyak, Olga Seleznova, Joseph Keshet, Matthew Goldrick, Ann R. Bradlow

Research output: Contribution to journal › Article › peer-review

9 Scopus citations

Abstract

Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.

Original language	English
Article number	025204
Journal	JASA Express Letters
Volume	4
Issue number	2
DOIs	https://doi.org/10.1121/10.0024877
State	Published - 1 Feb 2024
Externally published	Yes

Bibliographical note

Publisher Copyright:
© 2024 Author(s).

Funding

This work was supported by NSF DRL Grant No. 2219843 and BSF Grant No. 2022618. Thanks to Chun Chan for assistance with human data collection.

Funders	Funder number
NSF DRL	2219843
Bloom's Syndrome Foundation	2022618

Access to Document

10.1121/10.0024877

Cite this

@article{954bd1b2328d4ddd86413102d15c89b6,

title = "Automatic recognition of second language speech-in-noise",

abstract = "Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.",

author = "Kim, {Seung Eun} and Chernyak, {Bronya R.} and Olga Seleznova and Joseph Keshet and Matthew Goldrick and Bradlow, {Ann R.}",

note = "Publisher Copyright: {\textcopyright} 2024 Author(s).",

year = "2024",

month = feb,

day = "1",

doi = "10.1121/10.0024877",

language = "אנגלית",

volume = "4",

journal = "JASA Express Letters",

issn = "2691-1191",

publisher = "American Institute of Physics",

number = "2",

}

TY - JOUR

T1 - Automatic recognition of second language speech-in-noise

AU - Kim, Seung Eun

AU - Chernyak, Bronya R.

AU - Seleznova, Olga

AU - Keshet, Joseph

AU - Goldrick, Matthew

AU - Bradlow, Ann R.

PY - 2024/2/1

Y1 - 2024/2/1

N2 - Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.

AB - Measuring how well human listeners recognize speech under varying environmental conditions (speech intelligibility) is a challenge for theoretical, technological, and clinical approaches to speech communication. The current gold standard—human transcription—is time- and resource-intensive. Recent advances in automatic speech recognition (ASR) systems raise the possibility of automating intelligibility measurement. This study tested 4 state-of-the-art ASR systems with second language speech-in-noise and found that one, whisper, performed at or above human listener accuracy. However, the content of whisper's responses diverged substantially from human responses, especially at lower signal-to-noise ratios, suggesting both opportunities and limitations for ASR-based speech intelligibility modeling.

UR - http://www.scopus.com/inward/record.url?scp=85187299817&partnerID=8YFLogxK

U2 - 10.1121/10.0024877

DO - 10.1121/10.0024877

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

C2 - 38350077

AN - SCOPUS:85187299817

SN - 2691-1191

VL - 4

JO - JASA Express Letters

JF - JASA Express Letters

IS - 2

M1 - 025204

ER -

Automatic recognition of second language speech-in-noise

Abstract

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this