Events Calendar

Yearly View
View as List
Monthly View
By Month
Weekly View
By Week
Daily View
Download as iCal file
Seminar Series - Xi Chen
Friday, March 31, 2017, 02:00pm - 03:00pm
Contact Rachel Poole

Title: Statistical Inference for Model Parameters with Stochastic Gradient Descent

Abstract: In this talk, we investigate the problem of statistical inference of the true model parameters based on stochastic gradient descent (SGD). To this end, we propose two consistent estimators of the asymptotic covariance of the average iterate from SGD: (1) an intuitive plug-in estimator and (2) a computationally more efficient batch-means estimator, which only uses the iterates from SGD. As the SGD process forms a time-inhomogeneous Markov chain, our batch-means estimator with carefully chosen increasing batch sizes generalizes the classical batch-means estimator designed for time-homogenous Markov chains. Both proposed estimators allow us to construct asymptotically exact confidence intervals and hypothesis tests. We further discuss an extension to conducting inference based on SGD for high-dimensional linear regression.

Location: CLA 1.104