Exploiting the intermittency of speech for joint separation and diarization

Dionyssos Kounades-Bastian; Laurent Girin; Xavier Alameda-Pineda; Radu Horaud; Sharon Gannot

doi:10.1109/WASPAA.2017.8169991

Exploiting the intermittency of speech for joint separation and diarization

Dionyssos Kounades-Bastian, Laurent Girin, Xavier Alameda-Pineda, Radu Horaud, Sharon Gannot

Bar-Ilan University - The Alexander Kofkin Faculty of Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

Natural conversations are spontaneous exchanges involving two or more people speaking in an intermittent manner. Therefore one expects such conversation to have intervals where some of the speakers are silent. Yet, most (multichannel) audio source separation (MASS) methods consider the sound sources to be continuously emitting on the total duration of the processed mixture. In this paper we propose a probabilistic model for MASS where the sources may have pauses. The activity of the sources is modeled as a hidden state, the diarization state, enabling us to activate/de-Activate the sound sources at time frame resolution. We plug the diarization model within the spatial covariance matrix model proposed for MASS in [1], and obtain an improvement in performance over the state of the art when separating mixtures with intermittent speakers.

Original language	English
Title of host publication	2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	41-45
Number of pages	5
ISBN (Electronic)	9781538616321
DOIs	https://doi.org/10.1109/WASPAA.2017.8169991
State	Published - 7 Dec 2017
Event	2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017 - New Paltz, United States Duration: 15 Oct 2017 → 18 Oct 2017

Publication series

Name	IEEE Workshop on Applications of Signal Processing to Audio and Acoustics
Volume	2017-October

Conference

Conference	2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017
Country/Territory	United States
City	New Paltz
Period	15/10/17 → 18/10/17

Bibliographical note

Publisher Copyright:
© 2017 IEEE.

Keywords

Audio source separation
EM
spatial covariance matrix
speaker diarization

Access to Document

10.1109/WASPAA.2017.8169991

Cite this

Kounades-Bastian, D., Girin, L., Alameda-Pineda, X., Horaud, R., & Gannot, S. (2017). Exploiting the intermittency of speech for joint separation and diarization. In 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017 (pp. 41-45). (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics; Vol. 2017-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WASPAA.2017.8169991

Kounades-Bastian, Dionyssos ; Girin, Laurent ; Alameda-Pineda, Xavier et al. / Exploiting the intermittency of speech for joint separation and diarization. 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 41-45 (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics).

@inproceedings{5ed850ebd9af4c4292c02119c3e34ef4,

title = "Exploiting the intermittency of speech for joint separation and diarization",

abstract = "Natural conversations are spontaneous exchanges involving two or more people speaking in an intermittent manner. Therefore one expects such conversation to have intervals where some of the speakers are silent. Yet, most (multichannel) audio source separation (MASS) methods consider the sound sources to be continuously emitting on the total duration of the processed mixture. In this paper we propose a probabilistic model for MASS where the sources may have pauses. The activity of the sources is modeled as a hidden state, the diarization state, enabling us to activate/de-Activate the sound sources at time frame resolution. We plug the diarization model within the spatial covariance matrix model proposed for MASS in [1], and obtain an improvement in performance over the state of the art when separating mixtures with intermittent speakers.",

keywords = "Audio source separation, EM, spatial covariance matrix, speaker diarization",

author = "Dionyssos Kounades-Bastian and Laurent Girin and Xavier Alameda-Pineda and Radu Horaud and Sharon Gannot",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017 ; Conference date: 15-10-2017 Through 18-10-2017",

year = "2017",

month = dec,

day = "7",

doi = "10.1109/WASPAA.2017.8169991",

language = "אנגלית",

series = "IEEE Workshop on Applications of Signal Processing to Audio and Acoustics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "41--45",

booktitle = "2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017",

address = "ארצות הברית",

}

Kounades-Bastian, D, Girin, L, Alameda-Pineda, X, Horaud, R & Gannot, S 2017, Exploiting the intermittency of speech for joint separation and diarization. in 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, vol. 2017-October, Institute of Electrical and Electronics Engineers Inc., pp. 41-45, 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017, New Paltz, United States, 15/10/17. https://doi.org/10.1109/WASPAA.2017.8169991

Exploiting the intermittency of speech for joint separation and diarization. / Kounades-Bastian, Dionyssos; Girin, Laurent; Alameda-Pineda, Xavier et al.
2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 41-45 (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics; Vol. 2017-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Exploiting the intermittency of speech for joint separation and diarization

AU - Kounades-Bastian, Dionyssos

AU - Girin, Laurent

AU - Alameda-Pineda, Xavier

AU - Horaud, Radu

AU - Gannot, Sharon

PY - 2017/12/7

Y1 - 2017/12/7

N2 - Natural conversations are spontaneous exchanges involving two or more people speaking in an intermittent manner. Therefore one expects such conversation to have intervals where some of the speakers are silent. Yet, most (multichannel) audio source separation (MASS) methods consider the sound sources to be continuously emitting on the total duration of the processed mixture. In this paper we propose a probabilistic model for MASS where the sources may have pauses. The activity of the sources is modeled as a hidden state, the diarization state, enabling us to activate/de-Activate the sound sources at time frame resolution. We plug the diarization model within the spatial covariance matrix model proposed for MASS in [1], and obtain an improvement in performance over the state of the art when separating mixtures with intermittent speakers.

AB - Natural conversations are spontaneous exchanges involving two or more people speaking in an intermittent manner. Therefore one expects such conversation to have intervals where some of the speakers are silent. Yet, most (multichannel) audio source separation (MASS) methods consider the sound sources to be continuously emitting on the total duration of the processed mixture. In this paper we propose a probabilistic model for MASS where the sources may have pauses. The activity of the sources is modeled as a hidden state, the diarization state, enabling us to activate/de-Activate the sound sources at time frame resolution. We plug the diarization model within the spatial covariance matrix model proposed for MASS in [1], and obtain an improvement in performance over the state of the art when separating mixtures with intermittent speakers.

KW - Audio source separation

KW - EM

KW - spatial covariance matrix

KW - speaker diarization

UR - http://www.scopus.com/inward/record.url?scp=85042388426&partnerID=8YFLogxK

U2 - 10.1109/WASPAA.2017.8169991

DO - 10.1109/WASPAA.2017.8169991

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:85042388426

T3 - IEEE Workshop on Applications of Signal Processing to Audio and Acoustics

SP - 41

EP - 45

BT - 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017

Y2 - 15 October 2017 through 18 October 2017

ER -

Kounades-Bastian D, Girin L, Alameda-Pineda X, Horaud R, Gannot S. Exploiting the intermittency of speech for joint separation and diarization. In 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 41-45. (IEEE Workshop on Applications of Signal Processing to Audio and Acoustics). doi: 10.1109/WASPAA.2017.8169991

Exploiting the intermittency of speech for joint separation and diarization

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this