A Reconfigurable Two-WSe2-Transistor Synaptic Cell for Reinforcement Learning

Reward-modulated spike-timing-dependent plasticity (R-STDP) is a brain-inspired reinforcement learning (RL) rule, exhibiting potential for decision-making tasks and artificial general intelligence. However, the hardware implementation of the reward-modulation process in R-STDP usually requires complicated Si complementary metal-oxide-semiconductor (CMOS) circuit design that causes high power consumption and large footprint. Here, a design with two synaptic transistors (2T) connected in a parallel structure is experimentally demonstrated. The 2T unit based on WSe2 ferroelectric transistors exhibits reconfigurable polarity behavior, where one channel can be tuned as n-type and the other as p-type due to nonvolatile ferroelectric polarization. In this way, opposite synaptic weight update behaviors with multilevel (>6 bit) conductance states, ultralow nonlinearity (0.56/-1.23), and large G(max)/G(min) ratio of 30 are realized. By applying positive/negative reward to (anti-)STDP component of 2T cell, R-STDP learning rules are realized for training the spiking neural network and demonstrated to solve the classical cart-pole problem, exhibiting a way for realizing low-power (32 pJ per forward process) and highly area-efficient (100 mu m(2)) hardware chip for reinforcement learning.

Automated summary

This study demonstrates a 2T synaptic transistor design based on WSe2 ferroelectric transistors for implementing R-STDP learning rules. By utilizing the characteristics of ferroelectric polarization, multilevel conductance states and ultra-low nonlinearity are achieved. By applying rewards to the 2T unit, a spiking neural network is successfully trained and the classical cart-pole problem is solved.