AWS Certified Generative AI Developer - Professional AIP-C01

By Amazon · Question Mode

QuestionQ46

Operational Efficiency and Optimization for GenAI Applications

A company is building a generative AI (GenAI) application that analyzes customer service calls in real time and produces suggested responses for human customer service agents. The application must handle 500,000 concurrent calls during peak hours, with less than 200 ms end-to-end latency for every suggestion. The company uses an existing architecture to transcribe customer-call audio streams. The application must stay within a predefined monthly compute budget and retain auto-scaling capabilities.

Which solution meets these requirements?

Explanation

A low-latency, real-time optimized Amazon Bedrock model is suited to interactive response generation. Provisioned throughput provides dedicated, predictable inference capacity for sustained high concurrency, and automatic scaling policies support changing demand while helping manage capacity and compute costs. Batch processing cannot meet a sub-200 ms per-suggestion latency target.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!