Deep Reinforcement Learning with Double Q-Learning
Hado P. van Hasselt, Arthur Guez, David Silver
AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · 2016 · 6,775 citationsOpen access
No retraction on record in OpenAlex.Check Crossref now
Cite this paper
Hasselt, H. P. van, Guez, A., & Silver, D. (2016). Deep reinforcement learning with double Q-Learning. AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)). https://doi.org/10.1609/aaai.v30i1.10295
In text: (Hasselt et al., 2016)
Ask a question about this research
One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.
Abstract
The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.
Related papers
Metadata from OpenAlex (CC0). Citations are generated from the published record.