SDS Seminar Series – Dr. Amy Zhang

The Fall 2023 SDS Seminar Series continues on October 20th from 2:00 p.m. to 3:00 p.m. with Dr. Amy Zhang (Texas Instruments/Kilby Fellow in the Department of Electrical and Computer Engineering, University of Texas). This event is virtual.

Title: Bisimulation and Reinforcement Learning

Abstract: Most real world environments contain structure that would aid sample efficiency and generalization in control tasks if it were leveraged by algorithms.  In this talk, we propose to incorporate different assumptions that better reflect the real world and allow the design of novel algorithms with theoretical guarantees to address this fundamental problem. We first present how state abstractions can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction. Our goal is to learn state abstractions that both provide for effective downstream control and invariance to task-irrelevant details. We use bisimulation metrics to quantify behavioral similarity between states, and learn robust latent representations which encode only the task-relevant information from observations. We provide theoretical guarantees for the learned approximate abstraction and extend this notion to multi-task settings.