Button to scroll to the top of the page.

Events Calendar

Yearly View
View as List
Monthly View
By Month
Weekly View
By Week
Daily View
Download as iCal file
SDS Seminar Series: Yuguang Yue (Friday 4/23/21, 2pm)
Friday, April 23, 2021, 02:00pm - 03:00pm

The Spring 2021 SDS Seminar Series continues on Friday, April 23, 2021 from 2:00 p.m. to 3:00 p.m. via Zoom with Yuguang Yue (PhD student at the Department of Statistics and Data Sciences in the College of Natural Science at The University of Texas at Austin).

Please contact stat.admin@austin.utexas.edu for the Zoom link.

SDS Logo

Title: "Implicit Distributional Reinforcement Learning"

Abstract: To improve the sample efficiency of policy-gradient based reinforcement learning algorithms, we propose implicit distributional actor-critic (IDAC) that consists of a distributional critic, built on two deep generator networks (DGNs), and a semi-implicit actor (SIA), powered by a flexible policy distribution. We adopt a distributional perspective on the discounted cumulative return and model it with a state-action-dependent implicit distribution, which is approximated by the DGNs that take state-action pairs and random noises as their input. Moreover, we use the SIA to provide a semi-implicit policy distribution, which mixes the policy parameters with a reparameterizable distribution that is not constrained by an analytic density function. In this way, the policy's marginal distribution is implicit, providing the potential to model complex properties such as covariance structure and skewness, but its parameter and entropy can still be estimated. We incorporate these features with an off-policy algorithm framework to solve problems with continuous action space and compare IDAC with state-of-the-art algorithms on representative OpenAI Gym environments. We observe that IDAC outperforms these baselines in most tasks.