Skip to content

Overcoming catastrophic forgetting in neural networks

James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska‐Barwińska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, Raia T. Hadsell

Proceedings of the National Academy of Sciences · 2017 · 8,816 citationsOpen access

No retraction on record in OpenAlex.Check Crossref now

Cite this paper

Kirkpatrick, J., Pascanu, R., Rabinowitz, N. C., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska‐Barwińska, A., Hassabis, D., Clopath, C., Kumaran, D., & Hadsell, R. T. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13), 3521–3526. https://doi.org/10.1073/pnas.1611835114

In text: (Kirkpatrick et al., 2017)

Ask a question about this research

One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.

Abstract

The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.

  1. Long Short-Term Memory1997
  2. Gradient-based learning applied to document recognition1998
  3. Particle swarm optimization2002
  4. Greedy function approximation: A gradient boosting machine.2001
  5. Statistical Learning Theory1999

Metadata from OpenAlex (CC0). Citations are generated from the published record.