QuestionQ97

VerifiedTesting, Validation, and Troubleshooting

A company operates a generative AI (GenAI) application that uses Amazon Bedrock to deliver real-time responses to customer queries. The company has observed intermittent API-call failures to foundation models (FMs) during periods of peak traffic.

The company requires a solution that handles transient errors and delivers detailed observability into FM performance. The solution must prevent cascading failures during throttling events and provide distributed tracing across service boundaries to identify contributors to latency. The solution must also support correlating performance issues with specific FM characteristics.

Which solution satisfies these requirements?

Explanation

AWS SDK standard retry mode retries transient and throttling errors with exponential backoff and jitter, and its retry quota limits retry traffic when failures persist, reducing the risk of cascading failures. AWS X-Ray traces requests across service boundaries, while annotations provide indexed fields that can be used to filter and correlate latency or error behavior with foundation-model characteristics. CloudTrail records API activity for auditing; it is not a distributed tracing service.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!