QuestionQ73

VerifiedML Solution Monitoring, Maintenance, and Security

A company has deployed an XGBoost prediction model to production to predict whether a customer is likely to cancel a subscription. The company uses Amazon SageMaker Model Monitor to identify deviations in the F1 score.

During a model-quality baseline analysis, the company recorded an F1-score threshold. After several months without change, the model's F1 score decreases significantly.

What could cause the reduced F1 score?

Explanation

Concept drift occurs when the relationship between prediction inputs and the real cancellation outcome changes over time. The model’s patterns, learned from the baseline data, then become less predictive for current customers, reducing F1 score. SageMaker Model Monitor evaluates model-quality metrics against ground-truth labels and supports detection of model-quality drift.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!