QuestionQ62
VerifiedMonitoring AI solutionsAn ML model was deployed to production one year ago. Each month, you collect all raw requests sent to the model prediction service during the prior month. You send a subset of those requests to a human labeling service to assess the model’s performance.
After one year, you observe that model performance sometimes degrades substantially after a month, whereas at other times it takes several months to detect a performance decrease. The labeling service is expensive, but large performance degradations must also be avoided. You need to decide how often to retrain the model in order to maintain high performance while minimizing cost. What should you do?
Community Discussion