Whenever we asked you what’s the most important thing you’d like to see from nannyML research, you told us, “better performance estimation”. For the past year we’ve been working to deliver on that promise. We call it multi-calibrated confidence-based performance estimation (M-CBPE). It is model and data-type agnostic and works for any performance metric. M-CBPE does not need user input on the nature of the covariate shift as it fully learns from the data. We evaluated it on over 600 dataset-model pairs from US census data and compare it with multiple benchmarks using several evaluation metrics. Results show that M-CBPE is the best method to estimate the performance of classification models in any evaluation context. Join us on Thursday, February 1st at 5 PM CET (8 AM PST) as we welcome Wojtek Kuberski, co-founder of NannyML to present M-CBPE and how it compares to the SOTA.