QuestionQ46
VerifiedOperational Efficiency and Optimization for GenAI ApplicationsA company is building a generative AI (GenAI) application that analyzes customer service calls in real time and produces suggested responses for human customer service agents. The application must handle 500,000 concurrent calls during peak hours, with less than 200 ms end-to-end latency for every suggestion. The company uses an existing architecture to transcribe customer-call audio streams. The application must stay within a predefined monthly compute budget and retain auto-scaling capabilities.
Which solution meets these requirements?
Community Discussion