Continual Learning with Adaptive Memory Consolidation for Non-Stationary Data Streams

Catastrophic forgetting—the tendency of neural networks to abruptly lose previously acquired knowledge upon learning new tasks—remains a fundamental obstacle to continual learning in real-world deployments. We present AdaptiveCL, a continual learning framework using selective synaptic consolidation guided by task-relevance scores derived from gradient trajectory analysis. Unlike prior approaches that apply uniform regularization across all parameters, AdaptiveCL identifies task-critical synapses with high precision and applies targeted consolidation only where necessary. Our approach reduces forgetting by 67% on Split-CIFAR-100, 54% on Permuted MNIST, and 71% on a challenging sequential NLP benchmark, while maintaining 95% of original task performance and requiring minimal computational overhead relative to standard training.