QuestionQ49

VerifiedEvaluating and deploying agentic workflows

You are designing a continuous evaluation pipeline for an autonomous agent deployed on Agent Runtime. The agent uses multiple valid, non-linear sequences of function calls to resolve user requests rather than hardcoded golden trajectories. You want to continuously assess the semantic validity of tool selection and parameter generation without using hardcoded reference files.

What should you do?

Explanation

The managed types.RubricMetric.TOOL_USE_QUALITY metric uses adaptive rubrics to evaluate whether an agent selects appropriate tools and uses their parameters correctly. It accepts the prompt, agent instructions, tool declarations, and intermediate events without requiring a hardcoded reference trajectory, allowing valid alternative call sequences to be assessed. (docs.cloud.google.com)

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!