Fall Graduate Portfolio Colloquium - Shreya Shukla

The Fall 2023 Graduate Portfolio Colloquium will be held on December 5th from 10:30 a.m. to 1:30 p.m. Shreya Shukla's talk will be from 11:30 a.m. - 12:00 p.m. This event is virtual via Zoom.    

Title: System Evaluation by Betting Mechanism Portfolio in Scientific Computation

Abstract: In the realm of sequential testing for systems evaluation, the continuous collection of data allows for flexible decision-making, with the ability to halt the test at any juncture. An innovative approach involves incorporating a betting mechanism, where each output of the system is considered a wager, enabling a dynamic evaluation process. This concept aligns with the notion of "anytime valid inference," signifying that conclusions drawn at any interim point retain statistical validity.

In this methodology, a betting strategy is implemented, akin to an investment with some initial wealth, with the goal of sustaining wealth over successive bets until a predefined threshold is met and we lose all money. This adaptable strategy allows for a nuanced evaluation of the system's performance over time. This approach provides a continuous assessment that doesn't necessitate a comprehensive understanding of the entire distribution of the system's output. Instead, it relies on the knowledge of the declared expected value for the system and provides valuable insights even if the evaluation is halted prematurely.

This framework finds its application in traditional hypothesis testing of systems to evaluate if the systems deviate from their mean performance and therefore offers a dynamic and adaptable tool for evaluating system performance. Additionally, we investigate its application in monitoring time series prediction models, where the emphasis lies in tracking deviations from the expected value. It offers insights into identifying opportune moments for fine-tuning the model with new data from previous months. This adaptability ensures that adjustments can be made proactively, enhancing the model's ongoing relevance and effectiveness in capturing evolving patterns within the time series data.