Abstract
How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions.
Original language | English |
---|---|
Title of host publication | 2020 IEEE Information Theory Workshop, ITW 2020 |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
ISBN (Electronic) | 9781728159621 |
DOIs | |
State | Published - 11 Apr 2021 |
Event | 2020 IEEE Information Theory Workshop, ITW 2020 - Virtual, Riva del Garda, Italy Duration: 11 Apr 2021 → 15 Apr 2021 |
Publication series
Name | 2020 IEEE Information Theory Workshop, ITW 2020 |
---|
Conference
Conference | 2020 IEEE Information Theory Workshop, ITW 2020 |
---|---|
Country/Territory | Italy |
City | Virtual, Riva del Garda |
Period | 11/04/21 → 15/04/21 |
Bibliographical note
Publisher Copyright:© 2021 IEEE.