Certified Generative AI Engineer AssociateDemo

By Databricks · Question Mode

QuestionQ60

Evaluation and Monitoring

A generative AI engineer is deploying an AI agent, authored with MLflow's ChatAgent interface, for a retail company's customer-support system on Databricks using the Mosaic AI Agent Framework. The agent must process thousands of inquiries each day, and the engineer must monitor its performance and quality in real time to ensure it satisfies service-level agreements.

Which metrics are captured automatically by default and available for monitoring when the agent is deployed through the Mosaic AI Agent Framework?

  • A Operational metrics like request volume, latency, and errors
  • B Quality metrics like correctness and guideline adherence
  • C Both operational and quality metrics
  • D No metrics are automatically captured
Explanation

Databricks Model Serving makes endpoint health metrics available by default, including request rate, latency, and error rate. Agent-quality metrics require monitor configuration and selection of quality evaluations for sampled requests, so correctness and guideline adherence are not default-captured metrics.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!