Simultaneous Tracking and Separation of Multiple Sources Using Factor Graph Model

Koby Weisberg; Bracha Laufer-Goldshtein; Sharon Gannot

doi:10.1109/taslp.2020.3028650

Simultaneous Tracking and Separation of Multiple Sources Using Factor Graph Model

Koby Weisberg, Bracha Laufer-Goldshtein, Sharon Gannot

Bar-Ilan University - The Alexander Kofkin Faculty of Engineering

Research output: Contribution to journal › Article › peer-review

7 Scopus citations

Abstract

In this article, we present an algorithm for direction of arrival (DOA) tracking and separation of multiple speakers with a microphone array using the factor graph statistical model. In our model, the speakers can be located in one of a predefined set of candidate DOAs, and each time-frequency (TF) bin can be associated with a single speaker. Accordingly, by attributing a statistical model to both the DOAs and the associations, as well as to the microphone array observations given these variables, we show that the conditional probability of these variables given the microphone array observations can be modeled as a factor graph. Using the loopy belief propagation (LBP) algorithm, we derive a novel inference scheme which simultaneously estimates both the DOAs and the associations. These estimates are used in turn for separating the sources, by directing a beamformer towards the estimated DOAs, and then applying a TF masking according to the estimated associations. A comprehensive experimental study demonstrates the benefits of the proposed algorithm in both simulated data and real-life measurements recorded in our laboratory.

Original language	English
Article number	9212589
Pages (from-to)	2848-2864
Number of pages	17
Journal	IEEE/ACM Transactions on Audio Speech and Language Processing
Volume	28
DOIs	https://doi.org/10.1109/taslp.2020.3028650
State	Published - 2020

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

Funding

Manuscript received January 11, 2020; revised July 24, 2020 and September 12, 2020; accepted September 14, 2020. Date of publication October 5, 2020; date of current version October 31, 2020. This work was supported by the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement 871245. The work of Bracha Laufer-Goldshtein was supported by the Adams Fellowship Program of the Israel Academy of Sciences and Humanities. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Stefan Bilbao. (Corresponding author: Sharon Gannot.) The authors are with the Faculty of Engineering, Bar-Ilan University, Ramat-Gan 5290002, Israel (e-mail: [email protected]; [email protected]; [email protected]). Digital Object Identifier 10.1109/TASLP.2020.3028650

Funders	Funder number
Horizon 2020 Framework Programme	871245
Israel Academy of Sciences and Humanities

Keywords

Speaker tracking
factor graphs
loopy belief propagation (LBP)
speaker separation

Access to Document

10.1109/taslp.2020.3028650

Cite this

@article{638d9ade45004eeaaca4493cbc4c3e6d,

title = "Simultaneous Tracking and Separation of Multiple Sources Using Factor Graph Model",

abstract = "In this article, we present an algorithm for direction of arrival (DOA) tracking and separation of multiple speakers with a microphone array using the factor graph statistical model. In our model, the speakers can be located in one of a predefined set of candidate DOAs, and each time-frequency (TF) bin can be associated with a single speaker. Accordingly, by attributing a statistical model to both the DOAs and the associations, as well as to the microphone array observations given these variables, we show that the conditional probability of these variables given the microphone array observations can be modeled as a factor graph. Using the loopy belief propagation (LBP) algorithm, we derive a novel inference scheme which simultaneously estimates both the DOAs and the associations. These estimates are used in turn for separating the sources, by directing a beamformer towards the estimated DOAs, and then applying a TF masking according to the estimated associations. A comprehensive experimental study demonstrates the benefits of the proposed algorithm in both simulated data and real-life measurements recorded in our laboratory.",

keywords = "Speaker tracking, factor graphs, loopy belief propagation (LBP), speaker separation",

author = "Koby Weisberg and Bracha Laufer-Goldshtein and Sharon Gannot",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.",

year = "2020",

doi = "10.1109/taslp.2020.3028650",

language = "אנגלית",

volume = "28",

pages = "2848--2864",

journal = "IEEE/ACM Transactions on Audio Speech and Language Processing",

issn = "2329-9290",

publisher = "IEEE Advancing Technology for Humanity",

}

TY - JOUR

T1 - Simultaneous Tracking and Separation of Multiple Sources Using Factor Graph Model

AU - Weisberg, Koby

AU - Laufer-Goldshtein, Bracha

AU - Gannot, Sharon

PY - 2020

Y1 - 2020

N2 - In this article, we present an algorithm for direction of arrival (DOA) tracking and separation of multiple speakers with a microphone array using the factor graph statistical model. In our model, the speakers can be located in one of a predefined set of candidate DOAs, and each time-frequency (TF) bin can be associated with a single speaker. Accordingly, by attributing a statistical model to both the DOAs and the associations, as well as to the microphone array observations given these variables, we show that the conditional probability of these variables given the microphone array observations can be modeled as a factor graph. Using the loopy belief propagation (LBP) algorithm, we derive a novel inference scheme which simultaneously estimates both the DOAs and the associations. These estimates are used in turn for separating the sources, by directing a beamformer towards the estimated DOAs, and then applying a TF masking according to the estimated associations. A comprehensive experimental study demonstrates the benefits of the proposed algorithm in both simulated data and real-life measurements recorded in our laboratory.

AB - In this article, we present an algorithm for direction of arrival (DOA) tracking and separation of multiple speakers with a microphone array using the factor graph statistical model. In our model, the speakers can be located in one of a predefined set of candidate DOAs, and each time-frequency (TF) bin can be associated with a single speaker. Accordingly, by attributing a statistical model to both the DOAs and the associations, as well as to the microphone array observations given these variables, we show that the conditional probability of these variables given the microphone array observations can be modeled as a factor graph. Using the loopy belief propagation (LBP) algorithm, we derive a novel inference scheme which simultaneously estimates both the DOAs and the associations. These estimates are used in turn for separating the sources, by directing a beamformer towards the estimated DOAs, and then applying a TF masking according to the estimated associations. A comprehensive experimental study demonstrates the benefits of the proposed algorithm in both simulated data and real-life measurements recorded in our laboratory.

KW - Speaker tracking

KW - factor graphs

KW - loopy belief propagation (LBP)

KW - speaker separation

UR - http://www.scopus.com/inward/record.url?scp=85095969768&partnerID=8YFLogxK

U2 - 10.1109/taslp.2020.3028650

DO - 10.1109/taslp.2020.3028650

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:85095969768

SN - 2329-9290

VL - 28

SP - 2848

EP - 2864

JO - IEEE/ACM Transactions on Audio Speech and Language Processing

JF - IEEE/ACM Transactions on Audio Speech and Language Processing

M1 - 9212589

ER -

Simultaneous Tracking and Separation of Multiple Sources Using Factor Graph Model

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this