QuestionQ75
VerifiedOperational Efficiency and Optimization for GenAI ApplicationsA company uses Amazon Bedrock and Anthropic Claude 3 Haiku to build an AI assistant. The AI assistant typically handles 10,000 requests per hour, but during peak-use periods it experiences surges of up to 30,000 requests per hour. The AI assistant must respond within 2 seconds while operating across multiple AWS Regions.
The company finds that, during peak-use periods, the AI assistant has throughput bottlenecks that result in increased latency and occasional request timeouts. The company must address these performance issues.
Which solution will meet this requirement?
Community Discussion