Abstract:
State of the art Deep learning models, despite being at par to human level
in some of the challenging tasks, still suffer badly when they are put in the
condition where they have to learn with the passage of time. This open chal lenge problem of making deep learning model learn with the passage of time
is often called with synonymous names like Lifelong Learning, Incremental
Learning or Continual Learning etc. In each increment new classes / tasks are
introduced to the existing model and trained on them while maintaining the
accuracy on the previously learnt classes / tasks. But accuracy of the deep
learning model on the previously learnt classes / tasks decreases with each
increment. Main reason behind this accuracy drop is catastrophic forgetting,
an inherent flaw in the deep learning models, where weights learnt during the
past increments, get disturbed while learning the new classes / tasks from
new increment. Approaches have been proposed to mitigate or avoid this
catastrophic forgetting, such as use of knowledge distillation, rehearsal over
previous classes, or dedicated paths for different increments etc.
Here in my work I proposed a novel approach based on transfer learn ing methodology, which uses a combination of pre-trainied shared and fixed
network as backbone, along with a dedicated network extension in incremen tal setting for the learning of new tasks incrementally. Results have shown
that proposed architecture successfully bypasses the catastrophic forgetting
issue and completely eradicate the need of saved exemplars or retraining
phases which are required by the current state of the art model to maintain
performance, and still have performance comparable to the state of the art
incremental learning model.
