A dimensionality reduction approach for convolutional neural networks

The focus of this work is on the application of classical Model Order Reduction techniques, such as Active Subspaces and Proper Orthogonal Decomposition, to Deep Neural Networks. We propose a generic methodology to reduce the number of layers in a pre-trained network by combining the aforementioned techniques for dimensionality reduction with input-output mappings, such as Polynomial Chaos Expansion and Feedforward Neural Networks. The motivation behind compressing the architecture of an existing Convolutional Neural Network arises from its usage in embedded systems with specific storage constraints. The conducted numerical tests demonstrate that the resulting reduced networks can achieve a level of accuracy comparable to the original Convolutional Neural Network being examined, while also saving memory allocation. Our primary emphasis lies in the field of image recognition, where we tested our methodology using VGG-16 and ResNet-110 architectures against three different datasets: CIFAR-10, CIFAR-100, and a custom dataset.

Automated summary

The aim of this research is to apply classical Model Order Reduction techniques to Deep Neural Networks, reducing the number of layers in a pre-trained network for embedded systems with storage constraints. By combining dimensionality reduction techniques with input-output mappings, the reduced networks achieve comparable accuracy to the original Convolutional Neural Network while saving memory allocation. The research focuses on image recognition, testing the methodology using VGG-16 and ResNet-110 architectures against CIFAR-10, CIFAR-100, and a custom dataset.