QuestionQ123

VerifiedTesting, Validation, and Troubleshooting

A healthcare company is deploying a clinical knowledge-base application on Amazon Bedrock that supplies medical information to doctors. During testing, the quality team finds that the generative AI (GenAI) model sometimes invents medical treatment recommendations that are not derived from the approved clinical guidelines.

Before the application is released, the quality team must implement a solution to detect these hallucinations. The solution must compare model responses with verified medical information and identify semantic inconsistencies when users phrase similar questions differently.

Which solution meets these requirements?

Explanation

Amazon Bedrock model evaluation can use custom prompt datasets containing validated prompts and ground-truth responses to assess response correctness. Its robustness evaluation tests whether outputs change when prompts undergo semantic-preserving variations, enabling identification of inconsistent responses to similarly intended questions. A validated clinical question-and-answer dataset provides the approved reference information needed to identify ungrounded treatment recommendations.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!