SnowPro Advanced Data ScientistDemo

Question

A Data Scientist trained a supervised classification model on manually labeled training data. It performed well in production for several months, until model users reported degraded prediction accuracy.

The users demonstrated the degradation by collecting a set of recent predictions together with their actual labels.

What should the Data Scientist do?

A Train the model again with the initial training data but different hyperparameters.
B Explore different model classes to account for the change in distributions and train the model with the initial training data.
C Downsample the initial training data to account for the feature drift and train a new model with the collected subset of the initial training data.
D Use the data set collected by the users to build a new representative labeled training data set and train a new model with the newly collected training data.

Explanation

Production accuracy degradation with recent ground-truth labels indicates that the model should be retrained using labeled data representative of the current production population. Using the newly collected labeled data allows the model to learn patterns present in the changed distribution; changing hyperparameters, model class, or sampling only the original data does not incorporate those current patterns. AWS documentation describes training a new model with an expanded dataset when an underlying pattern absent from the previous training data has caused poor model performance.

Learn more

Use Incremental Training in Amazon SageMaker AI

SnowPro Advanced Data ScientistDemo

AstroTutor

Notes

Readiness

QuestionQ63

Community Discussion