Loading…
Glossary term
When a model loses earlier skills while being trained on new tasks.
When a network is trained further on a new task, the weight changes can overwrite what it learned before, so it suddenly performs much worse on the old task.
Remedies include mixing old data into training, regularization, and adding small adapters instead of changing the base weights.
It is a risk whenever a model is fine-tuned and a reason continual learning is difficult.