QuestionQ11
Solution DesignYou are testing an agent that generates natural-language responses to user queries, such as providing explanations or answering open-ended questions. The outputs must be evaluated for quality, semantic correctness, and reasoning. Which evaluator type is most appropriate for this scenario?
- A JSON similarity, as it assesses the structural similarity of outputs to ensure they align accurately.
- B LLM-as-a-Judge, as it provides flexible evaluation of semantic correctness and reasoning beyond exact matching.
- C Deterministic evaluators, as they guarantee all outputs strictly match predefined rules for correctness.
- D Exact match, as it ensures the agent output matches the expected response word-for-word.
Community Discussion