QuestionQ14

VerifiedML Solution Monitoring, Maintenance, and Security

A healthcare company uses an Amazon SageMaker AI endpoint to host a model that predicts patient readmission risk for hospitals. The company wants to predict patient readmissions with high accuracy and is willing to accept false positives. The current model’s performance has declined over the past year.

The company trains and deploys a new model as a shadow variant to test it on live traffic from hospitals. The company monitors the new model’s performance for one month. During that testing month, the shadow variant has higher recall than the existing model but lower precision.

What should the company do next?

Explanation

Recall measures how many actual patient readmissions the model identifies. Since detecting readmissions is the priority and false positives are acceptable, the higher-recall shadow model better satisfies the operating requirement despite its lower precision. SageMaker AI shadow testing compares a shadow variant with production traffic without returning shadow responses to callers; after satisfactory evaluation, the shadow variant can be promoted to production.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!