QuestionQ34

VerifiedDeployment and Orchestration of ML Workflows

An airline company uses an ML model to adjust ticket prices according to demand. The model runs on Amazon SageMaker real-time endpoints. In prior deployments, the model did not scale quickly enough when website traffic increased, causing delays in ticket-price adjustments.

An ML engineer must configure auto scaling for the SageMaker endpoints so they react rapidly to traffic changes. The solution must use target tracking scaling policies.

Which configuration will be MOST responsive to sudden traffic changes?

Explanation

InvocationsPerInstance is an appropriate utilization metric for target tracking on SageMaker endpoint variants. SageMaker invocation metrics can be published at 10-second intervals when enhanced metrics are enabled, allowing traffic changes to be detected much sooner than the default one-minute frequency. InvocationsPerInstance remains a standard metric; AWS documents high-resolution predefined scaling metrics separately for concurrent requests. The default 300-second scale-in cooldown is also less restrictive than a 600-second scale-in cooldown.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!