SDS Seminar Series – Matthew Vanaman, University of Texas at Austin

The Fall 2024 SDS Seminar Series will continue on September 13th from 2:00 p.m. to 3:00 p.m. with Dr. Matthew Vanaman (Postdoctoral Fellow, Department of Statistics and Data Sciences, University of Texas at Austin). This event is in-person in CBA 4.348.     

Title: Data Analysis from the Zoo to the Wild and Back

Abstract: Across the sciences, analysts are taught to analyze data in zoo-like classroom settings. In these settings, it is easy to distinguish the “well-trained” analysts from the feral ones: obey the zookeeper by identifying the appropriate model for your unrealistically clean data, properly understand how this model works under optimal conditions, interpret its output correctly and objectively, and report the results as a series of linear decisions with no apparent deviation from what was planned all along. Release this impressively domesticated analyst into the wild, and what could go wrong? They quickly find out. Confronted with entirely new challenges that the classroom can only imitate in their most idealized form, it becomes clear that real-world analysis requires an additional kind of expertise. I call this expertise analytic fluency. In asking one’s fellow analyst what these skills are, the answers often resemble something instinctual. Unfortunately, instincts cannot be taught, which is a problem given the litany of emerging data challenges facing new analysts, such as our massively increasing quantity of data, new kinds of measurements, replication crises, concerns about widespread use of questionable research practices, and a host of novel ethical challenges. It is therefore imperative that we elevate analytic fluency from the level of instincts to something explicit and formalized. In so doing, we prepare our next generation of analysts so that they do not have to rely on the slow-going instruction of experience. I take an initial step toward that end by reporting results from a qualitative pilot study probing what data analysts have learned from their experience. I consider what their testimonies imply about how we should teach data analysis, practice it, evaluate its success, and how our understanding of “good data analysis” might be revised to better reflect the conditions of the wild.