QuestionQ35

VerifiedEvaluation, Testing & Optimization

A revenue-projection assistant is 38 percent above its monthly cost target. Profiling identifies three contributors: a 6,000-token policy preamble repeated on every call (45 percent of cost), retrieval of historical sales chunks averaging 3,000 tokens per call (30 percent), and inference on a flagship-tier model (25 percent). Stakeholders require that projection accuracy remain unchanged.

Which two optimizations should you sequence first to reduce cost without affecting accuracy? Each correct answer represents part of the solution. (Select two.)

Choose two
Explanation

Prompt caching lets recurring requests reuse an identical stable prefix, such as a policy preamble or frequently reused context, at a much lower input-processing cost while preserving the exact content supplied to the model. Caching the static policy and commonly reused historical-sales chunks therefore reduces the two largest repeatable prompt-related costs without changing the projection inputs or model tier.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!