DeepStreamOS: Fast open-Set classification for convolutional neural networks

Convolutional Neural Networks (CNNs) can achieve state of the art results for visual recognition problems when the train and test data distributions are the same and when all classes in the test set are present in the training data. This is not representative of the real-world where the data evolves; existing classes change and new classes emerge. A traditional neural network only has the capacity to label instances with classes it has been trained on and cannot identify unknown classes. This can be of serious consequence in safety critical systems. The research field of Open-Set Classification provides potential solutions to overcome the identification of unknown classes in deep neural networks. In safety-critical systems, the speed with which unknown classes can be identified is also essential. Our system, termed DeepStreamOS, brings together the use of deep neural network activations with a stream-based outlier detection method for fast identification of instances that belong to unknown classes. DeepStreamOS uses all layers of a CNN to get a trajectory of the activations and applies a stream-based analysis method to determine if an instance belongs to an unknown class. We use CIFAR-10 and Fashion-MNIST datasets, withholding classes to apply as unknown data on VGG16 and MobileNet deep neural networks. Our system is compared with leading open-set classification methods: OpenMax and EVM. We show that DeepStreamOS outperforms OpenMax and EVM in most open-set classification scenarios and by a large margin on speed in all scenarios. (c) 2022 Elsevier B.V. All rights reserved.

Automated summary

Convolutional Neural Networks (CNNs) can achieve state of the art results for visual recognition problems when the data distributions are the same between train and test sets and all test set classes are present in the training data. However, in the real world, where data evolves and new classes emerge, traditional neural networks fail to identify unknown classes. Open-Set Classification research field provides potential solutions for this problem. In this study, a system called DeepStreamOS is proposed, which combines deep neural network activations with a stream-based outlier detection method to quickly identify instances belonging to unknown classes. Experimental results demonstrate that DeepStreamOS outperforms other open-set classification methods in most scenarios and significantly improves the speed of classification.