QuestionQ20

VerifiedDeployment and Orchestration of ML Workflows

A company plans to use Amazon SageMaker to host an ML model that runs on CPU and provides real-time predictions. The model will receive intermittent traffic during business hours and no traffic during some periods after business hours. The company requires the most cost-effective way to serve inference requests.

Which hosting option meets these requirements?

Explanation

Amazon SageMaker Serverless Inference is suited to synchronous workloads with intermittent traffic: it automatically scales with demand and scales to zero when no requests arrive, avoiding charges for idle compute. Provisioned concurrency can be scheduled higher during predictable business-hour demand to keep the endpoint warm and reduce cold starts.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!