Adaptive nodes enrich nonlinear cooperative learning beyond traditional adaptation by links

Shira Sardi, Roni Vardi, Amir Goldental, Anton Sheinin, Herut Uzan, Ido Kanter

Research output: Contribution to journalArticlepeer-review

20 Scopus citations

Abstract

Physical models typically assume time-independent interactions, whereas neural networks and machine learning incorporate interactions that function as adjustable parameters. Here we demonstrate a new type of abundant cooperative nonlinear dynamics where learning is attributed solely to the nodes, instead of the network links which their number is significantly larger. The nodal, neuronal, fast adaptation follows its relative anisotropic (dendritic) input timings, as indicated experimentally, similarly to the slow learning mechanism currently attributed to the links, synapses. It represents a non-local learning rule, where effectively many incoming links to a node concurrently undergo the same adaptation. The network dynamics is now counterintuitively governed by the weak links, which previously were assumed to be insignificant. This cooperative nonlinear dynamic adaptation presents a self-controlled mechanism to prevent divergence or vanishing of the learning parameters, as opposed to learning by links, and also supports self-oscillations of the effective learning parameters. It hints on a hierarchical computational complexity of nodes, following their number of anisotropic inputs and opens new horizons for advanced deep learning algorithms and artificial intelligence based applications, as well as a new mechanism for enhanced and fast learning by neural networks.

Original languageEnglish
Article number5100
JournalScientific Reports
Volume8
Issue number1
DOIs
StatePublished - 23 Mar 2018

Bibliographical note

Publisher Copyright:
© 2018 The Author(s).

Funding

We thank Moshe Abeles for stimulating discussions. A technical assistance by Hana Arnon is acknowledged. This research was supported by the TELEM grant of the Council for Higher Education of Israel.

FundersFunder number
Council for Higher Education

    Fingerprint

    Dive into the research topics of 'Adaptive nodes enrich nonlinear cooperative learning beyond traditional adaptation by links'. Together they form a unique fingerprint.

    Cite this