Numerical study of back-propagation learning algorithms for multilayer networks

A back-propagation learning algorithm is examined numerically for feedforward multilayer networks with one-hidden-layer functions as a parity machine or as a committeemachine of the internal representation of the hidden units. It is found that the maximal knowntheoretical capacity is saturated and that the convergent time is not exponential with the size ofthe system. The results also indicate the possibility of a replica-symmetry-breaking phase withthe lack of local minima.