Black-box ML models: Fit estimation and model selection

Abstract: As Machine Learning and Predictive Models become increasingly prevalent, so too does the risk of high-stakes errors. Particularly concerning are errors where the model is highly confident in an incorrect prediction - these are commonly referred to as Unknown Unknowns. More broadly, any instance where a model misjudges its own confidence can be viewed as a calibration error, indicating that the model is uncalibrated. While the detection of Unknown Unknowns is a well-studied topic, the practical value of each discovery is often unclear without additional context. We argue that the underlying goal of identifying Unknown Unknowns is to gain a deeper understanding of a model’s behavior - that is, to better comprehend the ”world” as represented by the model. More generally, this understanding can be achieved by improving the model’s calibration. This research aims to bridge these two areas by proposing an active learning framework that dynamically re-calibrates a predictive model, enhancing both its reliability and interpretability.