Skip to content

Deep Reinforcement Learning with Double Q-Learning

Hado P. van Hasselt, Arthur Guez, David Silver

AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · 2016 · 6,775 citationsOpen access

No retraction on record in OpenAlex.Check Crossref now

Cite this paper

Hasselt, H. P. van, Guez, A., & Silver, D. (2016). Deep reinforcement learning with double Q-Learning. AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)). https://doi.org/10.1609/aaai.v30i1.10295

In text: (Hasselt et al., 2016)

Ask a question about this research

One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.

Abstract

The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.

  1. A Survey on Transfer Learning2009
  2. Reinforcement Learning: A Survey1996
  3. Learning long-term dependencies with gradient descent is difficult1994
  4. Overcoming catastrophic forgetting in neural networks2017
  5. A Comprehensive Survey on Transfer Learning2020

Metadata from OpenAlex (CC0). Citations are generated from the published record.