Catastrophic Forgetting

Catastrophic forgetting, also known as catastrophic interference, is a common problem in many artificial neural network models. It refers to the catastrophic loss of previously learned responses when a network is trained with a single new response. This is a primary reason why artificial neural networks are not able to continuously learn from their surroundings and must be fully trained on an expected set of responses before being put into service.

Although catastrophic forgetting is a specific type of interference, interference is a more general phenomenon observed in natural learning systems such as humans, which tends to cause gradual losses. In contrast, losses caused by interference in artificial systems are catastrophic.

The stability-plasticity problem is sometimes promoted as a dilemma to explain away the problem of catastrophic forgetting in traditional artificial neural networks, even though natural systems have overcome it. More recently, the problem has been fully overcome in artificial neural networks through multi-temporal synapses.