QuestionQ128

VerifiedOperational Efficiency and Optimization for GenAI Applications

A legal research company is building an AI-powered chat assistant using Amazon Bedrock and a knowledge base that holds millions of legal documents. When attorneys search for legal case precedents, the initial retrieval produces many documents. The company requires the chat assistant to generate every response within 3 seconds.

The foundation model (FM) used by the chat assistant has a context-window constraint that permits effective processing of only 10 documents per query. The current retrieval configuration obtains 50 documents through semantic search. Attorneys report that highly relevant precedents often occur in result positions 15–40.

The company needs a solution that improves query-result relevance while meeting its latency and context-window constraints.

Which solution meets these requirements?

Explanation

Amazon Bedrock Knowledge Bases can apply a reranker to an initially retrieved set and return a specified smaller number of reranked results. A cross-encoder reranker scores each query-document pair more precisely than the initial semantic retrieval ranking, so retrieving 50 candidates and sending the top 10 reranked documents to the FM can surface relevant precedents originally ranked 15–40 while staying within the 10-document context limit. Reducing the context supplied to the FM also helps control generation latency. Improve the relevance of query responses with a reranker model in Amazon Bedrock

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!