AWS Certified Generative AI Developer - Professional AIP-C01

By Amazon · Question Mode

QuestionQ10

Testing, Validation, and Troubleshooting

A company upgraded its Amazon Bedrock-powered foundation model (FM) that supports a multilingual customer service assistant. Following the upgrade, the assistant showed inconsistent behavior between languages. It started producing different responses in some languages for identical questions.

The company needs a solution to identify and remediate similar issues in future updates. The evaluation must:

  • Complete within 45 minutes for every supported language.
  • Process at least 15,000 test conversations concurrently.
  • Be fully automated and integrated into the CI/CD pipeline.
  • Prevent deployment when quality thresholds are not met.

Which solution meets these requirements?

Explanation

Amazon Bedrock automatic model evaluation jobs can run programmatically against a custom prompt dataset and produce computed quality metrics. Standardized conversations with the same meaning in each supported language make inconsistent multilingual responses measurable; similarity and hallucination thresholds provide objective release criteria. Invoking the evaluation from CI/CD and failing the pipeline when those criteria are not met creates the required automated deployment gate. AWS: Evaluate the performance of Amazon Bedrock resources

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!