Training deep neural-networks using a noise adaptation layer

Jacob Goldberger, Ehud Ben-Reuven

Research output: Contribution to conferencePaperpeer-review

249 Scopus citations


The availability of large datsets has enabled neural networks to achieve impressive recognition results. However, the presence of inaccurate class labels is known to deteriorate the performance of even the best classifiers in a broad range of classification problems. Noisy labels also tend to be more harmful than noisy attributes. When the observed label is noisy, we can view the correct label as a latent random variable and model the noise processes by a communication channel with unknown parameters. Thus we can apply the EM algorithm to find the parameters of both the network and the noise and estimate the correct label. In this study we present a neural-network approach that optimizes the same likelihood function as optimized by the EM algorithm. The noise is explicitly modeled by an additional softmax layer that connects the correct labels to the noisy ones. This scheme is then extended to the case where the noisy labels are dependent on the features in addition to the correct labels. Experimental results demonstrate that this approach outperforms previous methods.

Original languageEnglish
StatePublished - 2017
Event5th International Conference on Learning Representations, ICLR 2017 - Toulon, France
Duration: 24 Apr 201726 Apr 2017


Conference5th International Conference on Learning Representations, ICLR 2017

Bibliographical note

Funding Information:
This work is supported by the Intel Collaborative Research Institute for Computational Intelligence (ICRI-CI).

Publisher Copyright:
© ICLR 2019 - Conference Track Proceedings. All rights reserved.


Dive into the research topics of 'Training deep neural-networks using a noise adaptation layer'. Together they form a unique fingerprint.

Cite this