A 389 TOPS/W, Always ON Region Proposal Integrated Circuit Using In-Memory Computing in 65 nm CMOS

Neuromorphic vision sensors (NVSs) are key enablers of energy savings in Internet of Things (IoT)-based traffic monitoring and surveillance systems that exploit the temporal redundancy in video streams. However, for these scenarios, an object typically occupies a fraction of the full image frame leading to a significant spatial redundancy in the active image. Hence, there is a need for energy-efficient, dedicated hardware to detect the region of interests (RoI) to exploit spatial redundancy in the valid frames and reduce computations in the succeeding recognition modules. This article proposes a 9T-SRAM in-memory computing (IMC)-based region proposal (RP) network for event-based binary image (EBBI) frames from a NVS. The proposed 9T-SRAM cell enables a 1-D projection of objects on the horizontal and vertical axes of an image. An iterative and selective search (ISS) of the rising and falling edges of 1-D projection yields the coordinates of a bounding box encapsulating an object. To demonstrate the energy-saving and effectiveness of the algorithm, we fabricated the proposed architecture, RP integrated circuit (RPIC) in a 65 nm CMOS process. Tested with the video recordings from a Dynamic and Active-pixel Vision Sensor (DAVIS), the RPIC achieves a peak throughput of 1259 ft/s at 1 Meps event rate. Moreover, the proposed RP architecture achieves a high energy efficiency of 389 TOPS/W due to in-memory operation.

Automated summary

This article proposes a region proposal network based on neuromorphic vision sensors for energy savings in Internet of Things traffic monitoring systems. The algorithm utilizes one-dimensional projection of images for edge detection, leading to high energy efficiency and throughput.