Handling irregularly sampled signals with gated temporal convolutional networks

We investigate the sequential modeling problem and introduce a novel gating mechanism into the temporal convolutional network architectures. In particular, we introduce the gated temporal convolutional network architecture with elaborately tailored gating mechanisms. In our implementation, we alter the way in which the gradients flow and avoid the vanishing or exploding gradient and the dead ReLU problems. The proposed GTCN architecture is able to model the irregularly sampled sequences as well. In our experiments, we show that the basic GTCN architecture is superior to the generic TCN architectures in various benchmark tasks requiring the modeling of long-term dependencies and irregular sampling intervals. Moreover, we achieve the state-of-the-art results on the permuted sequential MNIST and the sequential CIFAR10 benchmarks with the basic structure.

Automated summary

This study investigates the problem of sequential modeling and introduces a novel gating mechanism into the architecture of temporal convolutional networks. The proposed gated temporal convolutional network is designed to address the issues of gradient flow, vanishing or exploding gradient, and dead ReLU. Furthermore, it is capable of modeling irregularly sampled sequences. Experimental results demonstrate that the basic gated temporal convolutional network outperforms generic architectures in tasks involving long-term dependencies and irregular sampling intervals. Additionally, state-of-the-art results are achieved on the permuted sequential MNIST and sequential CIFAR10 benchmarks using the basic structure.