The series is envisioned as a vital contribution to the intellectual, cultural, and scholarly environment at The University of Texas at Austin for students, faculty, and the wider community. Each talk is free of charge and open to the public. For more information, contact Rachel Poole at rachel.poole[@]austin[dot]utexas[dot]edu.
 

Spring SEMINAR SERIES

January 26, 2018 – Eric Bickel & Chris Hadlock
(Department of Operations Research, The University of Texas at Austin)
"The Generalized J-QPD System” 
CBA 4.332, 2:00 to 3:00 PM

February 2, 2018 – Dr. Eric Bickel
(Department of Operations Research, The University of Texas at Austin)
"Decision Making under Partial Information” 
CBA 4.332, 2:00 to 3:00 PM

February 9, 2018 – Maurice Diesendruck
(Department of Statistics and Data Sciences, The University of Texas at Austin) 
“Importance Sampling in Generative Networks" 
CBA 4.332, 2:00 to 3:00 PM

February 16, 2018 – Erin Hartman 
(Department of Statistics and Political Science, UCLA)
“From SATE to PATT: combining experimental with observational studies to estimate population treatment effects" 
CBA 4.332, 2:00 to 3:00 PM

February 23, 2018 – Wesley Tansey
(Department of Systems Biology, Columbia University)
“Predictive Modeling of Treatment Efficacy in Cancer Cell Lines" 
CBA 4.332, 2:00 to 3:00 PM

March 2, 2018 – Suchi Saria
(Department of Computer Science, Johns Hopkins Whiting School of Engineering)
“Machine Learning and Counterfactual Reasoning for Individualized Healthcare" 
CBA 4.332, 2:00 to 3:00 PM

March 8, 2018 – Finale Doshi-Velez
(Department of Computer Science, Harvard Paulson School of Engineering and Applied Sciences) 
“Towards Personalized Antidepressant Recommendations with Prediction-Constrained Topic Models" 
GEA 114, 2:00 to 3:00 PM

March 23, 2018 – Jennifer Hill
(Department of Applied Statistics and Data Sciences, NYU Steinhardt) 
“Automated Versus do-it-yourself Methods for Causal Inference: Lessons Learned from a Data Analysis Competition" 
CBA 4.332, 2:00 to 3:00 PM

March 27, 2018 – Rachel Wang
(Department of Statistics, University of California, Berkeley) 
“Network Modeling of Topological Domains Using Hi-C Data" 
GEA 114, 2:00 to 3:00 PM

 March 30, 2018 – Robert Lunde
(Department of Statistics and Data Sciences, Carnegie Mellon University)
“Bootstrapping Generalization Error Bounds and Sample Splitting For Time Series" 
CBA 4.332, 2:00 to 3:00 PM

Eric Bickel & Chris Hadlock (Department of Operations Research, The University of Texas at Austin)

Title: The Generalized J-QPD Sytem

Abstract: It is common practice in decision analysis to elicit quantiles or cumulative probabilities of continuous uncertainties from a subject-matter expert, and then fit (e.g., by least-squares) a continuous probability distribution to the corresponding probability-quantile pairs; e.g., for subsequent Monte Carlo (MC) sampling, or discretization. This process often requires nonlinear, nonconvex optimization to generate the best fit, and the best fit often does not honor the assessed points. Johnson Quantile-Parameterized Distributions (J-QPDs) are parameterized by any symmetric percentile triplet (SPT) (e.g. the 10th -50th -90th) and support bounds. J-QPDs are smooth, highly flexible, and amenable to Monte Carlo simulation via inverse transform sampling. However, semi-bounded J-QPDs are limited to lognormal tails. We generalize the kernel distribution of J-QPD beyond the standard normal, generating new fat-tailed distribution systems that are more flexible than J-QPD. We also show how to augment the SPT/bound parameters with a tail parameter, lending separate control over the distribution body and tail. We then present advantages of our new generalized system over existing systems in the contexts of both expert elicitation and fitting to empirical data.

Dr. Eric Bickel (Department of Operations Research, The University of Texas at Austin)

Title: Decision Making under Partial Information

Abstract: The construction of a probabilistic model is a key step in most decision and risk analyses. Typically this is done by defining a single joint distribution in terms of marginal and conditional distributions. The difficulty of this approach is that often the joint distribution is underspecified. For example, we may lack knowledge of some marginal distributions or the underlying dependence structure.

In this talk, we present an approach for analyzing decisions in such cases. Specifically, we propose a simulation procedure to create a collection of joint distributions that match the known information. We demonstrate our procedure using an actual oil & gas exploration decision and compare our method to the use of copulas and maximum entropy.

Maurice Diesendruck (Department of Statistics and Data Sciences, The University of Texas at Austin)

Title: Importance Sampling in Generative Networks

Abstract: Generative adversarial networks (GANs) can be trained to model complex image distributions, where it is inconvenient or intractable to assign a probabilistic structure to the likelihood function. Earlier models achieved realistic samples, but suffered from mode collapse — when samples do not cover the full domain of the distribution. Even with proper coverage, however, another challenge emerges: At times, we wish to adjust a distribution for specific applications, or to correct for bias in a data set. We demonstrate how importance weights can be used in generative models, for a general class of distance measures, which includes the classical GAN loss, squared mean distance, and the maximum mean discrepancy (MMD). We provide a flexible way of re-balancing generated data and directing the generative function toward a target group, by interpreting a bias or modulation as a thinning function on the empirical data measure. This yields a manipulable discrepancy measure that can favor generative samples of a chosen class of labeled points.

Erin Hartman (Department of Statistics and Political Science, UCLA)

Title: From SATE to PATT: combining experimental with observational studies to estimate population treatment effects

Abstract: Randomized controlled trials (RCTs) can provide unbiased estimates of sample average treatment effects. However, a common concern is that RCTs may fail to provide unbiased estimates of population average treatment effects. We derive the assumptions that are required to identify population average treatment effects from RCTs. We provide placebo tests, which formally follow from the identifying assumptions and can assess whether they hold. We offer new research designs for estimating population effects that use non-randomized studies to ad- just the RCT data. This approach is considered in a cost-effectiveness analysis of a clinical intervention: pulmonary artery catheterization.

Title: Predictive Modeling of Treatment Efficacy in Cancer Cell Lines

Abstract: We present a case study of working directly with biologists to build a flexible-but-interpretable model of drug efficacy in cancer. Given a large dataset of high-throughput screening experiments, we have three analysis priorities: 1) determine which drugs worked, on which cell lines, and at which dosages; 2) build a powerful predictive model that can be used for new tumor samples; and 3) discover generalizable insights into the genomic drivers of treatment sensitivity and resistance in cancer. We discuss our approach in detail: handling contaminated data, combining deep learning with Bayesian hierarchical modeling, and uncovering significant gene-drug interactions. By keeping the biologist in the loop and building non-linear models that reason about uncertainty, we discover scientifically-meaningful interactions with false discovery rate control and produce a model that is currently being used to predict in vitro treatment efficacy at Columbia University Medical Center.

Suchi Saria (Department of Computer Science, Johns Hopkins Whiting School of Engineering)

Title: Machine Learning and Counterfactual Reasoning for Individualized Healthcare

Abstract: Decision-makers are faced with the challenge of estimating what is likely to happen when they take an action. For instance, would drug A or drug B lead to a better outcome for this patient? In an ideal (unrealistic) world, we would try out each action before choosing. Although this is impossible, we can sometimes learn models that predict the outcomes we would have observed after each action, which are called counterfactuals. We can help decision-makers choose better actions by learning to accurately predict counterfactuals, and our work advocates for algorithms that do this in lieu of classical supervised learning algorithms. In this talk, I will introduce Counterfactual Gaussian Process (CGP) as an application of this idea to a commonly encountered and challenging class of problems where outcomes are measured and actions are taken at discrete points in continuous-time. We design an adjusted maximum likelihood algorithm for learning the CGP from easily collected observational traces. We demonstrate the CGP on two decision-support tasks. First, we show that it can be used to produce more reliable risk predictions that are insensitive to action policies in the training data. Models trained using classical supervised objectives, however, are not stable to action policies and this can lead to harmful downstream decisions. Second, we show how the CGP can be used to compare counterfactuals and perform “what if?” reasoning by learning treatment effects using data from a real intensive care unit (ICU). Time permitting, I will show extensions of this framework to other disease areas and to the task of estimating individualized treatment responses. 

 Finale Doshi-Velez (Department of Computer Science, Harvard Paulson School of Engineering and Applied Sciences)

Bio: Finale Doshi-Velez is an assistant professor in computer science at the Harvard Paulson School of Engineering and Applied Sciences.  Prior to that, she completed her PhD at MIT and her masters at the University of Cambridge as a Marshall Scholar.  Her research focuses on the intersection of machine learning and healthcare.

Title: Towards Personalized Antidepressant Recommendations with Prediction-Constrained Topic Models 

Abstract: Major depressive disorder is one of the leading causes of morbidity worldwide.  Treatment for depression often requires medication, but choosing the best antidepressant for an individual is challenging.  About 50% of patients require trying more than one treatment before they find one that works, and 30% require more than two.  These numbers are daunting when each treatment may take several weeks to test. 
In this work, we begin by applying supervised topic models to predict individual anti-depressant response from the patient's electronic medical record, in hopes of creating a classifier that is both accurate and also interpretable/provides scientific insights about treatment response.  Along the way, we discover a fundamental statistical shortcoming of current the supervised topic modeling approaches and propose an alternative, prediction-constrained topic modeling, which addresses this shortcoming - and provides interpretable predictions.

Joint work with: Michael C Hughes, Tom McCoy, Roy Perlis, Leah Weiner, Gabe Hope, Erik Sudderth 

 Jennifer Hill (Department of Applied Statistics and Data Sciences, Columbia University)

Title: Automated Versus do-it-yourself Methods for Causal Inference: Lessons Learned from a Data Analysis Competition

Abstract: Statisticians have made great strides towards assumption-free estimation of causal estimands in the past few decades. However this explosion in research has resulted in a breadth of inferential strategies that both create opportunities for more reliable inference as well as complicate the choices that an applied researcher has to make and defend. Relatedly, researchers advocating for new methods typically compare their method to (at best) 2 or 3 other causal inference strategies and test using simulations that may or may not be designed to equally tease out flaws in all the competing methods. The causal inference data analysis challenge, "Is Your SATT Where It's At?", launched as part of the 2016 Atlantic Causal Inference Conference, sought to make progress with respect to both of these issues. The researchers creating the data testing grounds were distinct from the researchers submitting methods whose efficacy would be evaluated. Results from 30 competitors across the two versions of the competition (black box algorithms and do-it-yourself analyses) are presented along with post-hoc analyses that reveal information about the characteristics of causal inference strategies and settings that affect performance. The most consistent conclusion was that methods that flexibly model the response surface perform better overall than methods that fail to do so. R tools available to help researchers take advantage of the simulations and resulting datasets will be described.

 Rachel Wang (Department of Statistics, University of California, Berkeley)

Title: Network Modeling of Topological Domains Using Hi-C Data

Abstract: Chromosome conformation capture experiments such as Hi-C are used to map the three-dimensional spatial organization of genomes. One specific feature of the 3D organization is known as topologically associating domains (TADs), which are densely interacting, contiguous chromatin regions playing important roles in regulating gene expression. The structure of Hi-C data naturally inspires application of community detection methods. However, one of the drawbacks of community detection is that most methods take exchangeability of the nodes in the network for granted; whereas the nodes in this case, i.e. the positions on the chromosomes, are not exchangeable. We propose a network model for detecting TADs using Hi-C data that takes into account this non-exchangeability. In addition, our model explicitly makes use of cell-type specific CTCF binding sites as biological covariates and can be used to identify conserved TADs across multiple cell types. The model leads to a likelihood objective that can be efficiently optimized via relaxation. We also prove that when suitably initialized, this model finds the underlying TAD structure with high probability. Using simulated data, we show the advantages of our method and the caveats of popular community detection methods, such as spectral clustering, in this application. Applying our method to real Hi-C data, we demonstrate the domains identified have desirable epigenetic features and compare them across different cell types.

Robert Lunde (Department of Statistics and Data Sciences, Carnegie Mellon University) 

Title: Bootstrapping Generalization Error Bounds and Sample Splitting For Time Series