Exam-style
A RAG assistant gets about one answer in five wrong, and the team cannot agree whether to spend the next sprint on retrieval tuning or on prompt work. What should they run to settle it?
Reveal the answer
D. Run a Bedrock Knowledge Bases RAG evaluation, which scores retrieval quality and response quality separately
The RAG evaluation in Bedrock Knowledge Bases reports retrieval quality and response quality as separate numbers, which is exactly the split the argument needs; a cheap recall@k check against labelled chunks isolates the retriever further. A single end-to-end correctness score cannot attribute the failures, so tuning the prompt against it is guesswork. Reading logs and counting grounding flags both describe symptoms without saying which half produced them, and changing top-k experiments on one half before knowing it is the broken one.
Q. You cannot tell if a wrong RAG answer is a retrieval or a generation problem. What evaluates each half?
A. Bedrock Knowledge Bases RAG evaluation scores retrieval quality and response quality (groundedness, relevance) separately. Add a cheap Recall (retrieval)The share of genuinely relevant passages a search actually returns – what you lose when you retrieve fewer chunks. check against labelled chunks to isolate the retriever.
Why? Separate the two failure surfaces or you will fix the wrong thing.