A deep learning approach to unsupervised ensemble learning

Uri Shaham; Xiuyuan Cheng; Omer Dror; Ariel Jaffe; Boaz Nadler; Joseph Chang; Yuval Kluger

A deep learning approach to unsupervised ensemble learning

Uri Shaham, Xiuyuan Cheng, Omer Dror, Ariel Jaffe, Boaz Nadler, Joseph Chang, Yuval Kluger

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Scopus citations

Abstract

We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.

Original language	English
Title of host publication	33rd International Conference on Machine Learning, ICML 2016
Editors	Maria Florina Balcan, Kilian Q. Weinberger
Publisher	International Machine Learning Society (IMLS)
Pages	72-85
Number of pages	14
ISBN (Electronic)	9781510829008
State	Published - 2016
Event	33rd International Conference on Machine Learning, ICML 2016 - New York City, United States Duration: 19 Jun 2016 → 24 Jun 2016

Publication series

Name	33rd International Conference on Machine Learning, ICML 2016
Volume	1

Conference

Conference	33rd International Conference on Machine Learning, ICML 2016
Country/Territory	United States
City	New York City
Period	19/06/16 → 24/06/16

Bibliographical note

Funding Information:
The authors thank George Linderman, Alex Cloninger, Tingting Jiang, Raphy Coifman, Sahand Negahban, Andrew Barron, Alex Kovner, Shahar Kovalsky, Maria Angelica Cueto, Jason Morton, and Bernd Sturmfels for their help. This research was funded by the Intel Collaborative Research Institute for Computational Intelligence (B.N.) and by NIH grant 1R01HG008383-01 Al (Y.K. and B.N.).

Cite this

Shaham, U., Cheng, X., Dror, O., Jaffe, A., Nadler, B., Chang, J., & Kluger, Y. (2016). A deep learning approach to unsupervised ensemble learning. In M. F. Balcan, & K. Q. Weinberger (Eds.), 33rd International Conference on Machine Learning, ICML 2016 (pp. 72-85). (33rd International Conference on Machine Learning, ICML 2016; Vol. 1). International Machine Learning Society (IMLS).

@inproceedings{36febacd2d534417b893374f1ca6d085,

title = "A deep learning approach to unsupervised ensemble learning",

abstract = "We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.",

author = "Uri Shaham and Xiuyuan Cheng and Omer Dror and Ariel Jaffe and Boaz Nadler and Joseph Chang and Yuval Kluger",

note = "Funding Information: The authors thank George Linderman, Alex Cloninger, Tingting Jiang, Raphy Coifman, Sahand Negahban, Andrew Barron, Alex Kovner, Shahar Kovalsky, Maria Angelica Cueto, Jason Morton, and Bernd Sturmfels for their help. This research was funded by the Intel Collaborative Research Institute for Computational Intelligence (B.N.) and by NIH grant 1R01HG008383-01 Al (Y.K. and B.N.).; 33rd International Conference on Machine Learning, ICML 2016 ; Conference date: 19-06-2016 Through 24-06-2016",

year = "2016",

language = "אנגלית",

series = "33rd International Conference on Machine Learning, ICML 2016",

publisher = "International Machine Learning Society (IMLS)",

pages = "72--85",

editor = "Balcan, {Maria Florina} and Weinberger, {Kilian Q.}",

booktitle = "33rd International Conference on Machine Learning, ICML 2016",

}

Shaham, U, Cheng, X, Dror, O, Jaffe, A, Nadler, B, Chang, J & Kluger, Y 2016, A deep learning approach to unsupervised ensemble learning. in MF Balcan & KQ Weinberger (eds), 33rd International Conference on Machine Learning, ICML 2016. 33rd International Conference on Machine Learning, ICML 2016, vol. 1, International Machine Learning Society (IMLS), pp. 72-85, 33rd International Conference on Machine Learning, ICML 2016, New York City, United States, 19/06/16.

A deep learning approach to unsupervised ensemble learning. / Shaham, Uri; Cheng, Xiuyuan; Dror, Omer et al.
33rd International Conference on Machine Learning, ICML 2016. ed. / Maria Florina Balcan; Kilian Q. Weinberger. International Machine Learning Society (IMLS), 2016. p. 72-85 (33rd International Conference on Machine Learning, ICML 2016; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A deep learning approach to unsupervised ensemble learning

AU - Shaham, Uri

AU - Cheng, Xiuyuan

AU - Dror, Omer

AU - Jaffe, Ariel

AU - Nadler, Boaz

AU - Chang, Joseph

AU - Kluger, Yuval

N1 - Funding Information: The authors thank George Linderman, Alex Cloninger, Tingting Jiang, Raphy Coifman, Sahand Negahban, Andrew Barron, Alex Kovner, Shahar Kovalsky, Maria Angelica Cueto, Jason Morton, and Bernd Sturmfels for their help. This research was funded by the Intel Collaborative Research Institute for Computational Intelligence (B.N.) and by NIH grant 1R01HG008383-01 Al (Y.K. and B.N.).

PY - 2016

Y1 - 2016

N2 - We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.

AB - We show how deep learning methods can be applied in the context of crowdsourcing and unsupervised ensemble learning. First, we prove that the popular model of Dawid and Skene, which assumes that all classifiers are conditionally independent, is equivalent to a Restricted Boltzmann Machine (RBM) with a single hidden node. Hence, under this model, the posterior probabilities of the true labels can be instead estimated via a trained RBM. Next, to address the more general case, where classifiers may strongly violate the conditional independence assumption, we propose to apply RBM-based Deep Neural Net (DNN). Experimental results on various simulated and real-world datasets demonstrate that our proposed DNN approach outperforms other state-of-the-art methods, in particular when the data violates the conditional independence assumption.

UR - http://www.scopus.com/inward/record.url?scp=84997701143&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???

AN - SCOPUS:84997701143

T3 - 33rd International Conference on Machine Learning, ICML 2016

SP - 72

EP - 85

BT - 33rd International Conference on Machine Learning, ICML 2016

A2 - Balcan, Maria Florina

A2 - Weinberger, Kilian Q.

PB - International Machine Learning Society (IMLS)

T2 - 33rd International Conference on Machine Learning, ICML 2016

Y2 - 19 June 2016 through 24 June 2016

ER -

A deep learning approach to unsupervised ensemble learning

Abstract

Publication series

Conference

Bibliographical note

Other files and links

Fingerprint

Cite this