AEGNN: Asynchronous Event-based Graph Neural Networks

The best performing learning algorithms devised for event cameras work by first converting events into dense representations that are then processed using standard CNNs. However, these steps discard both the sparsity and high temporal resolution of events, leading to high computational burden and latency. For this reason, recent works have adopted Graph Neural Networks (GNNs), which process events as static spatio-temporal graphs, which are inherently sparse. We take this trend one step further by introducing Asynchronous, Event-based Graph Neural Networks (AEGNNs), a novel event-processing paradigm that generalizes standard GNNs to process events as evolving spatio-temporal graphs. AEGNNs follow efficient update rules that restrict recomputation of network activations only to the nodes affected by each new event, thereby significantly reducing both computation and latency for eventby-event processing. AEGNNs are easily trained on synchronous inputs and can be converted to efficient, asynchronous networks at test time. We thoroughly validate our method on object classification and detection tasks, where we show an up to a 200-fold reduction in computational complexity (FLOPs), with similar or even better performance than state-of-the-art asynchronous methods. This reduction in computation directly translates to an 8fold reduction in computational latency when compared to standard GNNs, which opens the door to low-latency eventbased processing.

Automated summary

This research introduces a novel event-processing paradigm called Asynchronous, Event-based Graph Neural Networks (AEGNNs), which generalize standard GNNs to process events as evolving spatio-temporal graphs. AEGNNs significantly reduce computation and latency by only recomputing network activations for the nodes affected by each new event. The experimental results show a reduction in computational complexity by up to 200-fold and similar or even better performance compared to state-of-the-art asynchronous methods, making AEGNNs a promising approach for low-latency event-based processing.