STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION

Dalia Sherman; Gershon Hazan; Sharon Gannot

doi:10.23919/EUSIPCO58844.2023.10289866

STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION

Dalia Sherman, Gershon Hazan, Sharon Gannot

Bar-Ilan University - The Alexander Kofkin Faculty of Engineering

Bar-Ilan University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

We present a study of a neural network-based method for speech emotion recognition that uses audio-only features. In the studied scheme, the acoustic features are extracted from the audio utterances and fed to a neural network that consists of convolutional neural networks (CNN) layers, bidirectional long short-term memory (BLSTM) combined with an attention mechanism layer, and a fully-connected layer. To illustrate and analyze the classification capabilities of the network, we used the t-distributed stochastic neighbor embedding (t-SNE) method. We evaluate our model using Ryerson audio-visual dataset of emotional speech and song (RAVDESS) and interactive emotional dyadic motion capture (IEMOCAP) datasets achieving weighted accuracy (WA) of 80% and 66%, respectively.

Original language	English
Title of host publication	31st European Signal Processing Conference, EUSIPCO 2023 - Proceedings
Publisher	European Signal Processing Conference, EUSIPCO
Pages	416-420
Number of pages	5
ISBN (Electronic)	9789464593600
DOIs	https://doi.org/10.23919/EUSIPCO58844.2023.10289866
State	Published - 2023
Event	31st European Signal Processing Conference, EUSIPCO 2023 - Helsinki, Finland Duration: 4 Sep 2023 → 8 Sep 2023

Publication series

Name	European Signal Processing Conference
ISSN (Print)	2219-5491

Conference

Conference	31st European Signal Processing Conference, EUSIPCO 2023
Country/Territory	Finland
City	Helsinki
Period	4/09/23 → 8/09/23

Bibliographical note

Publisher Copyright:
© 2023 European Signal Processing Conference, EUSIPCO. All rights reserved.

Funding

This project has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 871245. This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 871245.

Funders	Funder number
Horizon 2020 Framework Programme
Horizon 2020	871245

Keywords

Attention Mechanism
Deep Neural Network
Speech Emotion Recognition

Access to Document

10.23919/EUSIPCO58844.2023.10289866

Cite this

@inproceedings{d7c38fa56fa64e6698e8561a512ebd6a,

title = "STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION",

abstract = "We present a study of a neural network-based method for speech emotion recognition that uses audio-only features. In the studied scheme, the acoustic features are extracted from the audio utterances and fed to a neural network that consists of convolutional neural networks (CNN) layers, bidirectional long short-term memory (BLSTM) combined with an attention mechanism layer, and a fully-connected layer. To illustrate and analyze the classification capabilities of the network, we used the t-distributed stochastic neighbor embedding (t-SNE) method. We evaluate our model using Ryerson audio-visual dataset of emotional speech and song (RAVDESS) and interactive emotional dyadic motion capture (IEMOCAP) datasets achieving weighted accuracy (WA) of 80% and 66%, respectively.",

keywords = "Attention Mechanism, Deep Neural Network, Speech Emotion Recognition",

author = "Dalia Sherman and Gershon Hazan and Sharon Gannot",

note = "Publisher Copyright: {\textcopyright} 2023 European Signal Processing Conference, EUSIPCO. All rights reserved.; 31st European Signal Processing Conference, EUSIPCO 2023 ; Conference date: 04-09-2023 Through 08-09-2023",

year = "2023",

doi = "10.23919/EUSIPCO58844.2023.10289866",

language = "אנגלית",

series = "European Signal Processing Conference",

publisher = "European Signal Processing Conference, EUSIPCO",

pages = "416--420",

booktitle = "31st European Signal Processing Conference, EUSIPCO 2023 - Proceedings",

}

Sherman, D, Hazan, G & Gannot, S 2023, STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION. in 31st European Signal Processing Conference, EUSIPCO 2023 - Proceedings. European Signal Processing Conference, European Signal Processing Conference, EUSIPCO, pp. 416-420, 31st European Signal Processing Conference, EUSIPCO 2023, Helsinki, Finland, 4/09/23. https://doi.org/10.23919/EUSIPCO58844.2023.10289866

STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION. / Sherman, Dalia; Hazan, Gershon; Gannot, Sharon.
31st European Signal Processing Conference, EUSIPCO 2023 - Proceedings. European Signal Processing Conference, EUSIPCO, 2023. p. 416-420 (European Signal Processing Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION

AU - Sherman, Dalia

AU - Hazan, Gershon

AU - Gannot, Sharon

PY - 2023

Y1 - 2023

N2 - We present a study of a neural network-based method for speech emotion recognition that uses audio-only features. In the studied scheme, the acoustic features are extracted from the audio utterances and fed to a neural network that consists of convolutional neural networks (CNN) layers, bidirectional long short-term memory (BLSTM) combined with an attention mechanism layer, and a fully-connected layer. To illustrate and analyze the classification capabilities of the network, we used the t-distributed stochastic neighbor embedding (t-SNE) method. We evaluate our model using Ryerson audio-visual dataset of emotional speech and song (RAVDESS) and interactive emotional dyadic motion capture (IEMOCAP) datasets achieving weighted accuracy (WA) of 80% and 66%, respectively.

AB - We present a study of a neural network-based method for speech emotion recognition that uses audio-only features. In the studied scheme, the acoustic features are extracted from the audio utterances and fed to a neural network that consists of convolutional neural networks (CNN) layers, bidirectional long short-term memory (BLSTM) combined with an attention mechanism layer, and a fully-connected layer. To illustrate and analyze the classification capabilities of the network, we used the t-distributed stochastic neighbor embedding (t-SNE) method. We evaluate our model using Ryerson audio-visual dataset of emotional speech and song (RAVDESS) and interactive emotional dyadic motion capture (IEMOCAP) datasets achieving weighted accuracy (WA) of 80% and 66%, respectively.

KW - Attention Mechanism

KW - Deep Neural Network

KW - Speech Emotion Recognition

UR - http://www.scopus.com/inward/record.url?scp=85178367894&partnerID=8YFLogxK

U2 - 10.23919/EUSIPCO58844.2023.10289866

DO - 10.23919/EUSIPCO58844.2023.10289866

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85178367894

T3 - European Signal Processing Conference

SP - 416

EP - 420

BT - 31st European Signal Processing Conference, EUSIPCO 2023 - Proceedings

PB - European Signal Processing Conference, EUSIPCO

T2 - 31st European Signal Processing Conference, EUSIPCO 2023

Y2 - 4 September 2023 through 8 September 2023

ER -

STUDY OF SPEECH EMOTION RECOGNITION USING BLSTM WITH ATTENTION

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this