4.8 Article

Preventing Vanishing Gradient Problem of Hardware Neuromorphic System by Implementing Imidazole-Based Memristive ReLU Activation Neuron

期刊

ADVANCED MATERIALS
卷 35, 期 24, 页码 -

出版社

WILEY-V C H VERLAG GMBH
DOI: 10.1002/adma.202300023

关键词

deep neural network; initiated chemical vapor deposition; neuromorphic computing; ReLU activation neuron; vanishing gradient problem

向作者/读者索取更多资源

With advances in artificial intelligent services, brain-inspired neuromorphic systems with synaptic devices are recently attracting significant interest to solve the von Neumann bottleneck. However, deep neural networks suffer from huge power consumption and vanishing gradient problem. This research proposes a memristor-based compact and energy-efficient neuron device to implement the ReLU activation function, successfully resolving the vanishing gradient problem and achieving high-density and energy-efficient hardware neuromorphic systems.
With advances in artificial intelligent services, brain-inspired neuromorphic systems with synaptic devices are recently attracting significant interest to circumvent the von Neumann bottleneck. However, the increasing trend of deep neural network parameters causes huge power consumption and large area overhead of a nonlinear neuron electronic circuit, and it incurs a vanishing gradient problem. Here, a memristor-based compact and energy-efficient neuron device is presented to implement a rectifying linear unit (ReLU) activation function. To emulate the volatile and gradual switching of the ReLU function, a copolymer memristor with a hybrid structure is proposed using a copolymer/inorganic bilayer. The functional copolymer film developed by introducing imidazole functional groups enables the formation of nanocluster-type pseudo-conductive filaments by boosting the nucleation of Cu nanoclusters, causing gradual switching. The ReLU neuron device is successfully demonstrated by integrating the memristor with amorphous InGaZnO thin-film transistors, and achieves 0.5 pJ of energy consumption based on sub-10 mu A operation current and high-speed switching of 650 ns. Furthermore, device-to-system-level simulation using neuron devices on the MNIST dataset demonstrates that the vanishing gradient problem is effectively resolved by five-layer deep neural networks. The proposed neuron device will enable the implementation of high-density and energy-efficient hardware neuromorphic systems.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.8
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据