QuestionQ102

VerifiedOperational Efficiency and Optimization for GenAI Applications

A media company is building an AI-powered content moderation system using Amazon Bedrock. The system first classifies text using a small, low-latency model, then escalates requests whose confidence score is below 0.65 to a larger, more expensive model.

The solution must meet the following requirements:

  • Respond in near real time for high-confidence results.
  • Process low-confidence requests asynchronously.
  • Scale to handle sudden spikes in demand.
  • Optimize costs by invoking the larger model only when required.
  • Use decoupled components to achieve high resiliency.

Which solution will meet these requirements?

Explanation

Sending requests to an Amazon SQS queue and processing them with AWS Fargate decouples the ingestion layer from the compute layer, enabling the system to absorb sudden traffic spikes and scale the processing tier independently and resiliently. Invoking the small, low-latency model first and only forwarding low-confidence results (score below 0.65) to a second SQS queue for asynchronous processing by the larger model ensures the expensive model is invoked only when necessary, directly optimizing cost while still allowing high-confidence results to be handled quickly. This queue-based, decoupled architecture (SQS + Fargate) satisfies the asynchronous processing, scalability, cost-optimization, and resiliency requirements simultaneously, unlike synchronous API Gateway calls, always-invoking both models in parallel, or static EC2 instances with custom routing heuristics.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!