QuestionQ15

VerifiedClaude Models, Prompting & Context Engineering

A document-analysis service handles legal filings that average 80,000 tokens each. Attorneys query every filing an average of 14 times during a case. The current architecture sends the entire filing with every query. The CFO has asked you to lower per-query costs while maintaining response quality. The security officer requires that filing content not be stored outside Fabrikam's tenancy.

Which optimization approach should you recommend?

Explanation

Prompt-prefix caching reuses the processed representation of the unchanged filing across the repeated queries, reducing input-token cost and latency while preserving the full-document context and response quality. Azure OpenAI prompt caching is designed for identical long prefixes, does not alter the response content, and does not share prompt caches between Azure subscriptions.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!