QuestionQ122

VerifiedTesting, Validation, and Troubleshooting

A company deploys a GenAI application that uses an Amazon Bedrock foundation model (FM) with custom prompts. The company must implement an automated quality-assurance system to validate FM outputs against expected responses and identify regressions over time.

The solution must compare outputs with more than 500 golden examples by using semantic-similarity scoring. The solution must support regularly scheduled test rules. It must be able to scale to more than 2,000 test cases within 6 months without requiring architecture changes. The company wants to produce visualizations for quality metrics. The company must receive alerts when the solution detects quality degradation.

Which solution meets these requirements?

Explanation

AWS Lambda can invoke the Amazon Bedrock FM for each test case and implement semantic-similarity comparison against expected outputs. Amazon EventBridge scheduled rules can run the regression suite on a recurring schedule. Publishing custom quality metrics to Amazon CloudWatch provides metric visualizations and supports CloudWatch alarms when those metrics indicate quality degradation. Lambda scales automatically for a growing test suite.

Community Discussion

No comments yet. Be the first to start the discussion!