Adaptive Memory of a Neuromorphic Transistor with Multi-Sensory Signal Fusion

One of the ultimate goals of artificial intelligenceis to achievethe capability of memory evolution and adaptability to changing environments,which is termed adaptive memory. To realize adaptive memory in artificialneuromorphic devices, artificial synapses with multi-sensing capabilityare required to collect and analyze various sensory cues from theexternal changing environments. However, due to the lack of platformsfor mediating multiple sensory signals, most artificial synapses havebeen mainly limited to unimodal or bimodal sensory devices. Herein,we present a multi-modal artificial sensory synapse (MASS) based onan organic synapse to realize sensory fusion and adaptive memory.The MASS receives optical, electrical, and pressure information andin turn generates typical synaptic behaviors, mimicking the multi-sensoryneurons in the brain. Sophisticated synaptic behaviors, such as Pavloviandogs, writing/erasing, signal accumulation, and offset, were emulatedto demonstrate the joint efforts of bimodal sensory cues. Moreover,associative memory can be formed in the device and be subsequentlyreshaped by signals from a third perception, mimicking modificationof the memory and cognition when encountering a new environment. OurMASS provides a step toward next-generation artificial neural networkswith an adaptive memory capability.

Automated summary

The study introduces a multi-modal artificial sensory synapse (MASS) based on an organic synapse to achieve sensory fusion and adaptive memory. The MASS can receive optical, electrical, and pressure information and generate typical synaptic behaviors, resembling multi-sensory neurons in the brain. The research represents a significant step towards next-generation artificial neural networks with adaptive memory capabilities.