Offline only pattern

Case

Situation

A business value of a machine learning model can be proven when the model is implemented to your production service, and improves business KPI or contribute on efficiency. The model’s utility can only be actually evaluated based on business, and it is not test data. The test dataset can be one of the criteria to decide releasing the model or not into production, but the test dataset is not business KPI. Even if your model scores 99.99% accuracy in a test dataset, you always have to reevaluate its actual impact in the business production environment to see if it makes improvement. If the model performs poor in the production, or make something worse, then it is recommended to decide shut down request to the model (with Parameter-based serving pattern) to avoid anymore negative impact.

Diagram

diagram

Pros

Cons

Work around