Training deep neural-networks using a noise adaptation layer

Jacob Goldberger, Ehud Ben-Reuven

Research output: Contribution to conferencePaperpeer-review

362 Scopus citations

Abstract

The availability of large datsets has enabled neural networks to achieve impressive recognition results. However, the presence of inaccurate class labels is known to deteriorate the performance of even the best classifiers in a broad range of classification problems. Noisy labels also tend to be more harmful than noisy attributes. When the observed label is noisy, we can view the correct label as a latent random variable and model the noise processes by a communication channel with unknown parameters. Thus we can apply the EM algorithm to find the parameters of both the network and the noise and estimate the correct label. In this study we present a neural-network approach that optimizes the same likelihood function as optimized by the EM algorithm. The noise is explicitly modeled by an additional softmax layer that connects the correct labels to the noisy ones. This scheme is then extended to the case where the noisy labels are dependent on the features in addition to the correct labels. Experimental results demonstrate that this approach outperforms previous methods.

Original languageEnglish
StatePublished - 2017
Event5th International Conference on Learning Representations, ICLR 2017 - Toulon, France
Duration: 24 Apr 201726 Apr 2017

Conference

Conference5th International Conference on Learning Representations, ICLR 2017
Country/TerritoryFrance
CityToulon
Period24/04/1726/04/17

Bibliographical note

Publisher Copyright:
© ICLR 2019 - Conference Track Proceedings. All rights reserved.

Funding

This work is supported by the Intel Collaborative Research Institute for Computational Intelligence (ICRI-CI).

FundersFunder number
Intel Collaboration Research Institute for Computational Intelligence

    Fingerprint

    Dive into the research topics of 'Training deep neural-networks using a noise adaptation layer'. Together they form a unique fingerprint.

    Cite this