AI-103: Developing AI Apps and Agents on Azure

By Microsoft · Question Mode

QuestionQ41

VerifiedImplement generative AI and agentic solutions

You have a Microsoft Foundry project that contains a high-traffic agent. Following a recent update, operational costs rise significantly, even though monitoring confirms that the volume of user traffic to the agent has stayed the same.

You suspect that changes to the request or response characteristics are behind the increase, and you need to determine whether the added cost comes from the model input size, the model output size, or expanded tool usage.

Which observability capability should you use?

Explanation

Token usage telemetry breaks down consumption into prompt (input) tokens, completion (output) tokens, and the additional tokens consumed by tool or function calls on each run, so it can directly attribute a cost increase to whichever of those components grew — larger inputs, longer outputs, or heavier tool invocation — even when the number of requests stays constant. Latency measures response time, evaluation metrics assess response quality, and run success or failure rate measures reliability; none of these decompose cost by input size, output size, or tool usage the way token usage telemetry does.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!