Deep Radial-Basis Value Functions for Continuous Control

Authors

  • Kavosh Asadi Amazon Web Services, Brown University
  • Neev Parikh Brown University
  • Ronald E. Parr Duke University
  • George D. Konidaris Brown University
  • Michael L. Littman Brown University

Keywords:

Reinforcement Learning

Abstract

A core operation in reinforcement learning (RL) is finding an action that is optimal with respect to a learned value function. This operation is often challenging when the learned value function takes continuous actions as input. We introduce deep radial-basis value functions (RBVFs): value functions learned using a deep network with a radial-basis function (RBF) output layer. We show that the maximum action-value with respect to a deep RBVF can be approximated easily and accurately. Moreover, deep RBVFs can represent any true value function owing to their support for universal function approximation. We extend the standard DQN algorithm to continuous control by endowing the agent with a deep RBVF. We show that the resultant agent, called RBF-DQN, significantly outperforms value-function-only baselines, and is competitive with state-of-the-art actor-critic algorithms.

Downloads

Published

2021-05-18

How to Cite

Asadi, K., Parikh, N., Parr, R. E., Konidaris, G. D., & Littman, M. L. (2021). Deep Radial-Basis Value Functions for Continuous Control. Proceedings of the AAAI Conference on Artificial Intelligence, 35(8), 6696-6704. Retrieved from https://ojs.aaai.org/index.php/AAAI/article/view/16828

Issue

Section

AAAI Technical Track on Machine Learning I