QuestionQ20

VerifiedApplications of Foundation Models

A company is introducing a mobile app that helps users learn foreign languages. The app makes text more coherent by calling a large language model (LLM). The company collected a diverse dataset of text and supplemented the dataset with examples of more readable versions. The company wants the LLM output to resemble the provided examples.

Which metric should the company use to assess whether the LLM meets these requirements?

Explanation

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is specifically designed to measure how well generated text matches reference examples by calculating n-gram overlap, precision, recall, and F1 scores. Since the company has reference examples of readable text and wants the model's output to resemble them, ROUGE directly evaluates this alignment. The loss function measures training performance, semantic robustness measures resilience, and latency measures speed—none assess output similarity to examples.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!