Analytical study of the interplay between architecture and predictability

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

We study model feed forward networks as time series predictors in the stationary limit. The focus is on complex, yet non-chaotic, behavior. The main question we address is whether the asymptotic behavior is governed by the architecture, regardless the details of the weights. We find hierarchies among classes of architectures with respect to the attractor dimension of the long term sequence they are capable of generating; larger number of hidden units can generate higher dimensional attractors. In the case of a perceptron, we develop the stationary solution for general weights, and show that the flow is typically one dimensional. The relaxation time from an arbitrary initial condition to the stationary solution is found to scale linearly with the size of the network. In multilayer networks, the number of hidden units gives bounds on the number and dimension of the possible attractors. We conclude that long term prediction (in the non-chaotic regime) with such models is governed by attractor dynamics related to the architecture.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems 10 - Proceedings of the 1997 Conference, NIPS 1997
PublisherNeural information processing systems foundation
Pages315-321
Number of pages7
ISBN (Print)0262100762, 9780262100762
StatePublished - 1998
Event11th Annual Conference on Neural Information Processing Systems, NIPS 1997 - Denver, CO, United States
Duration: 1 Dec 19976 Dec 1997

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Conference

Conference11th Annual Conference on Neural Information Processing Systems, NIPS 1997
Country/TerritoryUnited States
CityDenver, CO
Period1/12/976/12/97

Fingerprint

Dive into the research topics of 'Analytical study of the interplay between architecture and predictability'. Together they form a unique fingerprint.

Cite this