Inquiry Response: Using PSI to Monitor Models in Production
We’re implementing a model accuracy monitoring process for our forecast models to find out when they’re no longer effective and why. Do you have any recommendations?
THE BIG IDEAS:
- Input factors
- Model comparison
The factors that go into models can affect model performance. A common cause for model degradation is that a signal gets turned off because someone changes a data format in the data warehouse or decides a variable is no longer necessary.