Analytical study of time series generation by feed-forward networks

Generation of time series is studied analytically for a generalization of the bit generator to continuous activation functions and multilayer architectures. The network exhibits at asymptotically large times the following characteristic features: (a) flows can be periodic or quasiperiodic depending on the phase of the weights, (b) the dimension of the attractor is a function of the gain of the activation function and the number of hidden units, and (c) a phase shift in the weights results in a frequency shift in the output, so that the system operates as a phase detector.