Abstract

Deep reinforcement learning has been extensively studied, resulting in several extensions to DQN that improve its perfor- mance, such as replay buffer sampling strategies, distributional value representations, and double/dueling networks. Previ- ous works have examined these extensions in the context of either discrete action spaces or in conjunction with actor-critic learning algorithms, but there has been no investigation of combining them for deep value-based continuous control. We adapted the methods discussed in Rainbow DQN to RBF-DQN, a deep valued-based method for continuous control, showing improvements in baseline performance and sample efficiency. Rainbow RBF-DQN is able to outperform vanilla RBF-DQN on the most challenging tasks even outperforming state of the art policy gradient methods like SAC.