You define a business case. You gather data. You spend hours doing EDA and uncovering all the interesting insights in your data. You build a model in a notebook and evaluate it on your test data, 0.8 ROC AUC, good enough to go into production. Your model gets there and after a few months you realise the real ROC AUC is closer to 0.6 and the business impact is not what you expected.

Given the cost of deployment and all the change management needed to go into production, your model might be even losing money. Proof of concepts are not enough.

Join Wojtek on the 27th June at 1.30 PM CET to learn more about running real world pilots of your machine learning systems, how to set up a pilot, and how, given relatively small data, you can use advanced probabilistic techniques to evaluate the success of your pilot.