期刊
FRONTIERS IN COMPUTATIONAL NEUROSCIENCE
卷 15, 期 -, 页码 -出版社
FRONTIERS MEDIA SA
DOI: 10.3389/fncom.2021.627357
关键词
equilibrium propagation; deep learning; small-world; layer-skipping connections; neuromorphic computing; biologically-motivated
The paper proposes a method to alleviate the vanishing gradient problem by replacing some connections in a layered network, inspired by small-world networks. This approach is convenient to implement in neuromorphic hardware and is biologically plausible.
Equilibrium propagation is a learning framework that marks a step forward in the search for a biologically-plausible implementation of deep learning, and could be implemented efficiently in neuromorphic hardware. Previous applications of this framework to layered networks encountered a vanishing gradient problem that has not yet been solved in a simple, biologically-plausible way. In this paper, we demonstrate that the vanishing gradient problem can be mitigated by replacing some of a layered network's connections with random layer-skipping connections in a manner inspired by small-world networks. This approach would be convenient to implement in neuromorphic hardware, and is biologically-plausible.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据