Self-Supervised Curriculum-based Class Incremental Learning

Class-incremental learning, a sub-field of continual learning, faces catastrophic forgetting, where models forget previous tasks while learning new ones. Existing solutions fall into expansion-based, memory-based, and regularization-based approaches, with limited focus on the latter despite its deployability and efficiency. This paper introduces Self-Supervised Curriculum-based Class Incremental Learning (S ² C ² IL), a novel regularization-based algorithm that improves class-incremental performance without external memory or network expansion. S ² C ² IL leverages self-supervised learning to extract rich feature representations using a new pretext task based on stochastic label augmentation instead of image augmentation. To prevent pretext task-specific knowledge transfer, the final section of the pre-trained network is excluded in feature transfer. For downstream tasks, a curriculum strategy periodically adjusts the standard deviation of a filter fused with the network. Evaluated on split-CIFAR10, split-CIFAR100, split-SVHN, and split-TinyImageNet, S ² C ² IL achieves state-of-the-art results, outperforming existing regularization-based and memory-based class-incremental algorithms.