Journal
ELECTRONICS
Volume 10, Issue 16, Pages -Publisher
MDPI
DOI: 10.3390/electronics10161879
Keywords
incremental learning; catastrophic forgetting; CNN; progressive learning networks
Categories
Funding
- Institute for Information & communications Technology Promotion (IITP) - Korean government (MSIT) [2017-0-01772]
- Information & communications Technology Promotion (IITP) grant - Korean government (MSIT) [2020-0-00113]
Ask authors/readers for more resources
This paper introduces a novel incremental learning technique to address the catastrophic forgetting problem in CNN architectures, utilizing a progressive deep neural network to learn new classes incrementally while maintaining performance on old classes. Extensively evaluated on various datasets, experimental results demonstrate that the proposed network architecture not only alleviates catastrophic forgetting but also leverages prior knowledge through lateral connections to previously learned classes and their features.
In this paper, we present a novel incremental learning technique to solve the catastrophic forgetting problem observed in the CNN architectures. We used a progressive deep neural network to incrementally learn new classes while keeping the performance of the network unchanged on old classes. The incremental training requires us to train the network only for new classes and fine-tune the final fully connected layer, without needing to train the entire network again, which significantly reduces the training time. We evaluate the proposed architecture extensively on image classification task using Fashion MNIST, CIFAR-100 and ImageNet-1000 datasets. Experimental results show that the proposed network architecture not only alleviates catastrophic forgetting but can also leverages prior knowledge via lateral connections to previously learned classes and their features. In addition, the proposed scheme is easily scalable and does not require structural changes on the network trained on the old task, which are highly required properties in embedded systems.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available