Lab: Stand Up a Bedrock Knowledge Base

This starts the managed track of the hands-on labs. The first ten build everything by hand against the model API. These two buy it instead, and the point of going second is that you already know what is being bought. The from-scratch lab made you write embed-compare-rank yourself; this one hands the same Greenbox documents to a Knowledge Base and gives you the two calls that query it. The full lab is in lab-11-knowledge-base.zip.

Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper, a standing backstop that auto-deletes any lab you forget to tear down after 24 hours. It reaches this lab’s Knowledge Base and its S3 Vectors store too.

The scenario

The support assistant works. Behind it are five documents held in memory, embedded on cold start, scored with a cosine function you wrote. That is fine for five documents. It falls over at five thousand: nothing re-embeds when a document changes, nothing splits a long document into pieces small enough to match precisely, and every cold start pays to embed the whole corpus again.

A Knowledge Base takes that job. It watches a bucket, chunksChunkingSplitting documents into retrievable pieces before embedding them – small enough to match precisely, big enough to still make sense. what it finds, embeds the chunks, keeps them in a vector index, and re-embeds only what changed when you sync it. The documents in this lab are the same five Greenbox topics, expanded until they are long enough that chunking matters, plus one internal support runbook that staff can read and subscribers must not.

What you’re given

CloudFormation builds the lot: a bucket for the documents, an S3 Vectors vector bucket and index, the Knowledge Base with its service role, an S3 data source with fixed-size chunking configured, and a query Lambda. Every piece is a native CloudFormation resource, so nothing is created out of band, and AWS::S3Vectors::VectorBucket plus AWS::S3Vectors::Index mean the vector store is torn down with the stack.

S3 Vectors rather than OpenSearch Serverless is a cost decision. A collection bills for capacity units whether or not you query it, and an orphaned one is the expensive mistake in this track. A vector bucket bills for what it holds and what you ask it, which for a few hundred vectors is nothing.

The docs/ directory carries a .metadata.json sidecar next to each document, tagging it with a topic and an audience. Those become filterable attributes on every chunk, which is what makes the last part of the lab work.

src/handler.py has the request parsing, the response helper, and a small function that builds the retrieval configuration. Two gaps are left.

Your task

First, the raw search. No model, no prose, just chunks and scores. retrieve() is one call to the client’s Retrieve operation: the Knowledge Base id, a retrieval query carrying the question text, and a retrieval configuration whose vectorSearchConfiguration comes from the helper below. The reply is a list of retrievalResults, each holding the chunk text under content, a score, the source document under location.s3Location.uri, and the sidecar attributes under metadata. Reshape each result into a dict of text, score, source and audience, in the order the service returned them.

Then the whole thing, search and generation together, with citations attached. answer() calls RetrieveAndGenerate: the question goes in as the input text, and the configuration is type KNOWLEDGE_BASE, naming the Knowledge Base id, the generation model ARN, and the same vector search configuration from the helper. The generated answer comes back under output.text, and each entry in citations links a span of that text to the retrievedReferences behind it. Walk those references, collect the S3 URIs de-duplicated in first-seen order, and return the answer text alongside that list. The module docstring in src/handler.py has the exact request and response shapes for both calls.

Both go through the same helper, which is where top-kTop-kHow many chunks a retrieval step returns per query – the dial that trades answer coverage against token cost. and the metadata filter live:

def _vector_search_config(k, audience):
    config = {"numberOfResults": k}
    if audience:
        config["filter"] = {"equals": {"key": "audience", "value": audience}}
    return config

Note the client. Retrieve and RetrieveAndGenerate are on bedrock-agent-runtime, not the bedrock-runtime every earlier lab used. Same account, same credentials, different service endpoint, and reaching for the wrong one is the first thing that goes wrong.

Deploy and prove it

cd lab-11-knowledge-base
./scripts/deploy.sh
./scripts/test.sh
./scripts/teardown.sh

The first deploy takes about five minutes, most of it the Knowledge Base and index coming up. After the stack, the script uploads docs/, uploads your handler, then starts an ingestion job and polls until it completes, printing how many documents were scanned and indexed.

The test script asks six questions. “When will my box arrive?” comes back with Tuesdays and Fridays, a citation pointing at delivery-days.txt, and the chunks it drew on with their scores. The card-declined question runs twice, once at three chunks and once at one, so you can watch the answer narrow. “How much goodwill credit can I get?” answers from the internal runbook when nothing is filtered and refuses once the query is pinned to audience = subscriber. The carbon-footprint question admits it does not know.

Then change a document. Edit docs/delivery-days.txt to add a Wednesday run, run ./scripts/deploy.sh again, and ask again. The answer moves, because the deploy script re-uploads and re-syncs, and Bedrock re-embeds only the document that changed.

When you want the reference answer, deploy it with SRC=solution ./scripts/deploy.sh, or unfold it here:

def retrieve(question, k=3, audience=None):
    response = _agent.retrieve(
        knowledgeBaseId=KNOWLEDGE_BASE_ID,
        retrievalQuery={"text": question},
        retrievalConfiguration={
            "vectorSearchConfiguration": _vector_search_config(k, audience)
        },
    )
    return [
        {
            "text": r.get("content", {}).get("text", ""),
            "score": r.get("score"),
            "source": r.get("location", {}).get("s3Location", {}).get("uri"),
            "audience": r.get("metadata", {}).get("audience"),
        }
        for r in response.get("retrievalResults", [])
    ]

def answer(question, k=3, audience=None):
    response = _agent.retrieve_and_generate(
        input={"text": question},
        retrieveAndGenerateConfiguration={
            "type": "KNOWLEDGE_BASE",
            "knowledgeBaseConfiguration": {
                "knowledgeBaseId": KNOWLEDGE_BASE_ID,
                "modelArn": GEN_MODEL_ARN,
                "retrievalConfiguration": {
                    "vectorSearchConfiguration": _vector_search_config(k, audience)
                },
            },
        },
    )
    citations = []
    for citation in response.get("citations", []):
        for reference in citation.get("retrievedReferences", []):
            uri = reference.get("location", {}).get("s3Location", {}).get("uri")
            if uri and uri not in citations:
                citations.append(uri)
    return response["output"]["text"], citations

What it’s actually doing

The managed service is not doing anything you have not already built. It is doing it on a schedule, at a scale, and with a set of seams worth knowing.

Chunking is a property of the data source. Not of the Knowledge Base, and not of the query. ChunkingStrategy: FIXED_SIZE with MaxTokens and OverlapPercentage sits on AWS::Bedrock::DataSource, and it cannot be changed after the data source exists. A different chunk size means replacing the data source and re-ingesting everything, which is why the choice is worth making deliberately the first time. The overlap is there so a sentence split across a boundary still appears whole in one of the two chunks.

Nothing happens until you sync. Creating the Knowledge Base and pointing it at a bucket indexes precisely zero documents. StartIngestionJob is what crawls the bucket, and it is an operation rather than a resource, so CloudFormation cannot do it for you. That is also the mechanism for keeping answers current: subsequent jobs crawl incrementally, re-embedding what changed and leaving the rest, so a re-sync after one edited document costs one document’s worth of embedding.

Retrieve and RetrieveAndGenerate answer different questions. Retrieve gives you chunks, scores, source URIs and metadata, with no model call and no model cost. It is what you want when you are debugging why an answer is wrong, when you want to rerank the results yourself, or when the retrieved text is going into a prompt you control. RetrieveAndGenerate does the search and writes the answer over the results, and hands back citations linking spans of the generated text to the chunks behind them. Returning both, as this lab does, is how you tell “retrieval found the wrong thing” from “retrieval was fine and the model wandered”.

The filter is doing access control. The audience key exists because a sidecar file said so, and filtering at retrieval time means the internal runbook chunks are never eligible to come back. Asking the model to ignore documents it can see is a request; not retrieving them is a guarantee. The same mechanic is how one Knowledge Base serves several tenants without leaking between them.

Two IAM roles are doing separate jobs. The Knowledge Base service role is assumed by Bedrock to read your bucket, call the embedding model, and write vectors into the index. The Lambda role calls bedrock:Retrieve on one Knowledge Base ARN and bedrock:RetrieveAndGenerate, which AWS documents unscoped. Confusing the two produces access-denied errors at completely different moments: one at ingestion, one at query.

What’s worth remembering

1. A Knowledge Base is chunk, embed, index, search, operated for you; the trade is sync and scale in exchange for control over exactly how it chunks and ranks.
2. Chunking config lives on the data source, is immutable after creation, and changing it means replacing the data source and re-ingesting.
3. Ingestion is an operation, not a resource: StartIngestionJob is what indexes anything, and re-running it is how a changed document reaches the index.
4. Incremental sync means a re-ingest costs only the documents that changed, which is what makes a daily-changing corpus affordable.
5. Retrieve returns chunks and scores with no model call; RetrieveAndGenerate returns an answer with citations. Use the first to see, the second to answer, and both when you need to tell a retrieval fault from a generation fault.
6. numberOfResults is top-k, and it trades coverage against tokens on every single query.
7. Metadata filters come from .metadata.json sidecars beside each document, and filtering at retrieval time is stronger than instructing the model to ignore what it can see.
8. S3 Vectors puts the vector store on the same bill as the documents, giving up hybrid search and the lowest latencies; for most internal retrieval that is the right trade, and it is one you can delete cleanly.