QuestionQ94
VerifiedOperational Efficiency and Optimization for GenAI ApplicationsA legal research company operates a Retrieval Augmented Generation (RAG) application that uses Amazon Bedrock and Amazon OpenSearch Service. The application stores 768-dimensional vector embeddings for 15 million legal documents, including statutes, court rulings, and case summaries.
The company’s current chunking strategy divides text into fixed-length blocks of 500 tokens. This strategy frequently separates contextually connected information, such as legal arguments, court opinions, or statute references, into different chunks. Researchers report that generated outputs often leave out critical context or reference outdated legal information.
Recent application logs show a 40% rise in response times. The p95 latency metric is greater than 2 seconds. The company anticipates that the application’s storage requirements will increase from 90 GB to 360 GB within one year.
The company needs a solution that improves retrieval relevance and system performance at scale.
Which solution meets these requirements?
Community Discussion