QuestionQ128
VerifiedOperational Efficiency and Optimization for GenAI ApplicationsA legal research company is building an AI-powered chat assistant using Amazon Bedrock and a knowledge base that holds millions of legal documents. When attorneys search for legal case precedents, the initial retrieval produces many documents. The company requires the chat assistant to generate every response within 3 seconds.
The foundation model (FM) used by the chat assistant has a context-window constraint that permits effective processing of only 10 documents per query. The current retrieval configuration obtains 50 documents through semantic search. Attorneys report that highly relevant precedents often occur in result positions 15–40.
The company needs a solution that improves query-result relevance while meeting its latency and context-window constraints.
Which solution meets these requirements?
Community Discussion