Keeping a Knowledge Base Fresh Without Re-Embedding Everything

The situation

A support-automation team runs an Amazon Bedrock Knowledge Base over roughly forty thousand documents in S3: product manuals, pricing sheets, policy pages, and a large archive of resolved support tickets. An agent retrieves the top passages for each customer question and grounds its answer in them. When the Knowledge Base was first built, every document was embedded once and written to the vector store, and retrieval has worked well since.

The problem is keeping it current. Pricing sheets change weekly, policy pages change a few times a month, and the manual archive barely moves. To stay fresh, the team wired a nightly job that re-ingests the entire data source. It works, but the embedding cost of pushing forty thousand documents through the model every night now dwarfs the cost of the queries the Knowledge Base actually answers, and the sync takes long enough that it eats into the morning. Worse, when a document is deleted at source, nobody is sure the old passages ever leave the index, so retrieval sometimes surfaces a policy that was retired months ago.

The team wants fresh answers without paying to re-embed a corpus that mostly did not change, and without stale passages lingering to outrank the current ones. Underneath the nightly-cost complaint are three separate questions: what has to be re-embedded, when the work should run, and which facts do not belong in a Knowledge Base at all.

What actually matters

Embedding is the expensive, slow part of ingestion, and it scales with how much text you push through the model, not with how much of it changed. Every document that gets re-processed is tokenised, chunked, embedded, and written to the vector store, and the embedding-model invocation is where both the money and the minutes go. Re-embedding forty thousand documents to reflect a change in forty of them means paying for the other thirty-nine thousand nine hundred and sixty for nothing. The first thing worth naming is that full re-ingestion of a large corpus is a cost you almost never actually need to pay.

Bedrock Knowledge Bases already know this. After the first successful sync of a data source, subsequent syncs are incremental: the managed connector compares the current state of the source against what it has already ingested and re-processes only the documents that were added, modified, or deleted. It detects the change for you rather than reprocessing everything, so a sync after forty documents moved does forty documents of embedding work, not forty thousand. The nightly full re-ingest the team built is fighting this instead of using it, most likely by clearing and rebuilding rather than letting the connector diff.

Given incremental sync exists, the real lever is when it runs. A sync is a discrete job, so freshness is a function of how often you trigger one against how much embedding you are willing to pay for. Two shapes of trigger exist. A schedule (say, an EventBridge rule that starts an ingestion job every few hours) is simple and predictable, and its cost is bounded by how much changed since the last run. An event-driven trigger (an S3 event notification firing a Lambda that starts a sync when an object lands) makes answers current within minutes of a document changing, at the cost of more, smaller sync jobs and the operational plumbing to debounce them. Faster freshness costs more sync overhead; the right point on that line depends on how stale an answer is allowed to be for each kind of document.

Deletes and updates are where staleness does real damage, because a wrong-but-confident passage is worse than a missing one. When a document changes, its old chunksChunkingSplitting documents into retrievable pieces before embedding them – small enough to match precisely, big enough to still make sense. have to leave the index, or the retriever can pull a stale passage that competes with the fresh one and sometimes wins. Incremental sync handles this when the source is the connector’s source of truth: modify a file and its old vectors are replaced, delete a file and its vectors are removed. The trap is deleting at the edges of that contract, dropping an object without letting a sync observe the deletion, or managing vectors by hand, which leaves orphaned chunks behind to haunt retrieval.

Metadata is the quiet lever that lets fresh content win even when old content still exists. Bedrock lets you attach a metadata file to each document and filter or scope retrieval on those fields at query time. A last_updated or effective_date field means a query can prefer recent content, or exclude anything past a cutoff, so the retriever is not left ranking a superseded 2024 policy against the 2026 one on text similarity alone. Metadata does not replace deleting stale chunks; it is how you bias toward freshness among the chunks that legitimately coexist.

And the boundary worth drawing hardest: some facts do not belong in a Knowledge Base at any refresh rate. A current account balance, today’s inventory count, a live order status, a price that changes intraday, these are not documents to embed, they are values to look up. Retrieval always answers as of the last sync, so for a fact that must be correct to the second, no sync cadence is fast enough and every sync is wasted embedding. That is a live data or tool call, where the agent queries the system of record at question time and reads the current value. Matching the mechanism to how the fact behaves matters more than tuning the sync.

What we’ll filter on

1. Change rate, how often the source documents actually change, and what fraction of the corpus moves per period.
2. Freshness requirement, how stale an answer is allowed to be before it is wrong, per document type.
3. Re-embedding budget, how much embedding cost and sync latency the corpus size implies per full pass.
4. Delete and update fidelity, whether retired content reliably leaves the index rather than lingering.
5. Trigger fit, whether a schedule or an event-driven sync matches the freshness requirement without over-syncing.
6. Fact volatility, whether the value is a document to retrieve or a live reading to look up at query time.

The freshness landscape

Full re-ingestion. Clear the vectors and re-embed the whole data source. The only case this earns its keep is a genuine reset: a new embedding model, a chunking-strategy change that invalidates every existing vector, or a first build. As a routine freshness mechanism on a large, slow-changing corpus it is the most expensive option by a wide margin, because it pays to re-embed everything to reflect a change in almost nothing.

Incremental sync (the default managed behaviour). After the first sync, the Bedrock data-source connector diffs the source and re-processes only added, modified, and deleted documents. This is the baseline you want to be on: the cost of a sync tracks the volume of change, not the size of the corpus. It handles deletes and updates as part of the diff, so retired content leaves the index when the connector observes the deletion. The work is to trigger it well, not to replace it.

Scheduled incremental sync. Start an ingestion job on a fixed cadence, commonly an EventBridge schedule invoking StartIngestionJob. Predictable, easy to reason about, and its cost is bounded by how much changed in the interval. Freshness is capped at the interval length, so a weekly pricing change on a daily schedule can be up to a day stale, which may be fine or may not, depending on the document.

Event-driven incremental sync. An S3 event notification fires a Lambda that starts a sync when an object is created, updated, or removed. Answers go current within minutes of a change, which suits documents that must not lag. The costs are operational: many small sync jobs, the need to debounce bursts of changes so you do not start overlapping ingestion jobs, and more moving parts to monitor. Best reserved for the subset of the corpus that genuinely needs minute-level freshness.

Metadata-filtered retrieval. Attach last_updated or effective_date metadata and filter or scope queries on it, so retrieval prefers recent content or excludes anything past a cutoff. This is a query-time lever, not an ingestion one; it does not remove stale chunks, it stops legitimately coexisting older chunks from outranking newer ones. Pairs with any of the sync options above rather than replacing them.

Live data or tool call. For volatile facts, skip retrieval and have the agent call the system of record at question time, reading the current balance, price, or status directly. Always correct to the moment, no embedding cost, no sync to keep current. It only fits facts that are values rather than passages of prose, and it needs the tool and permissions wired up, but for the truly real-time slice it is the only mechanism that is ever actually fresh.

Side by side

Reading the table against the corpus: the manual archive wants a slow scheduled incremental sync; the pricing and policy pages want event-driven incremental so a change is reflected in minutes; every document type benefits from last_updated metadata so retrieval prefers the current version; and any genuinely live fact (a customer’s current plan status, today’s price) belongs in a tool call, not the index at all. The nightly full re-ingest serves none of these well.

The picks in depth

Getting off full re-ingestion is the first and largest win, and it is mostly a matter of stopping the wrong thing. The nightly job is almost certainly rebuilding rather than diffing, either by recreating the data source or clearing vectors before ingesting. Let the managed connector do what it already does: run StartIngestionJob against the existing data source and it re-processes only what changed since the last successful sync. The same forty-document change that cost a full corpus of embedding overnight becomes forty documents of work, and the sync finishes in a fraction of the time. Nothing exotic is needed here; the fix is trusting the incremental behaviour instead of overriding it.

Choosing the trigger is where the freshness-versus-cost trade actually gets made, and it is worth making per document type rather than once for the whole Knowledge Base. The slow-moving manual archive does not justify event plumbing; a scheduled incremental sync every few hours, or even daily, keeps it current enough and keeps sync jobs few. The pricing and policy pages are the opposite: a stale price is a wrong answer with real consequences, so an S3 event notification firing a Lambda that starts a sync earns the extra machinery. The one thing to get right in the event-driven path is debouncing. A bulk update that rewrites two hundred objects should coalesce into one sync, not two hundred overlapping ingestion jobs, so buffer the events (a short SQS-backed window, for instance) and start a single job for the batch.

Deletes deserve explicit attention because they fail silently. As long as the S3 data source is the connector’s source of truth, removing an object and running a sync removes its vectors, and a modified object replaces its old chunks. The staleness bug the team is seeing (a retired policy still surfacing) points at deletions that never reached a sync, or vectors that were once managed outside the connector. The discipline is to treat the data source as authoritative: change content by changing the source and syncing, never by editing the vector store directly, so every add, update, and delete flows through the same diff. A stale passage that outranks the current one is worse than a gap, because the model will answer confidently from it.

Metadata is the cheap insurance on top. Attach a metadata file to each document carrying last_updated (and, for policies, an effective_date), and have retrieval filter or scope on it so a query prefers current content and can exclude anything past its effective window. This does not substitute for deleting stale chunks; it handles the legitimate case where several versions coexist and pure text similarity would otherwise let an older, wordier passage win. It costs a little ingestion-side structure and pays off every query.

The last pick is a boundary, not a mechanism. For facts that change faster than any reasonable sync (a customer’s live plan status, an intraday price, a current stock level) retrieval is the wrong tool, because it can only ever answer as of the last sync and every sync is embedding spent on a value that will be wrong again by lunchtime. Wire those as a tool the agent calls against the system of record at question time. The Knowledge Base then holds the durable prose (how cancellation works, what the tiers include) while the volatile numbers come from a live call, and neither mechanism is asked to do the other’s job. This mirrors the reasoning behind reaching for a tool call rather than the model’s own weights when the answer depends on current data.

A worked example: the pricing sheet that lagged a day

A pricing sheet is updated in S3 at 09:00 on a Tuesday. Under the nightly full re-ingest, the change is invisible until the next run at 02:00 Wednesday, so for seventeen hours the assistant quotes the old price, and the run that finally picks it up re-embeds all forty thousand documents to reflect one changed file.

Reworked, the same change flows differently. The pricing prefix in S3 has event notifications enabled; the 09:00 write fires a Lambda, which (after a short debounce window in case more sheets follow) calls StartIngestionJob against the existing data source. The connector diffs the source, sees one modified object, re-embeds that single sheet’s chunks, replaces the old vectors, and finishes in seconds. By 09:03 retrieval returns the new price. The sheet also carries metadata, so even in the brief window where a superseded chunk might still be reachable, the query prefers the one with the later last_updated.

# EventBridge / S3-notification-driven sync (conceptual)
on s3:ObjectCreated | s3:ObjectRemoved for prefix pricing/:
    buffer events for 60s          # debounce a burst into one job
    bedrock-agent StartIngestionJob \
        --knowledge-base-id ${KB_ID} \
        --data-source-id  ${DS_ID}
    # connector re-processes only changed/added/deleted objects

And the fact that should never have been a document in the first place, the customer’s current plan and next billing date, is not retrieved at all: the agent calls a billing tool at question time and reads it live. The pricing prose is fresh within minutes at the cost of one document’s embedding; the live account fact is correct to the second at the cost of nothing embedded. The seventeen-hour lag and the nightly forty-thousand-document bill are both gone, and neither the sync nor the tool call is doing work the other should own.

What’s worth remembering

1. Embedding cost scales with the volume of text re-processed, not with how much changed, so full re-ingestion of a large corpus is a bill you almost never need to pay.
2. After the first sync, Bedrock Knowledge Base syncs are incremental: only added, modified, and deleted documents are re-processed and re-embedded, and the connector detects the change for you.
3. A nightly full re-ingest usually means something is clearing or rebuilding instead of letting the connector diff; trusting the incremental behaviour is most of the fix.
4. Freshness is set by how often you trigger a sync against how much embedding you will pay for; choose the trigger per document type, not once for the whole Knowledge Base.
5. Scheduled syncs (EventBridge on a cadence) are simple and cap freshness at the interval; event-driven syncs (S3 event to Lambda) go current in minutes at the cost of more, smaller jobs.
6. Debounce event-driven syncs so a bulk change coalesces into one ingestion job rather than many overlapping ones.
7. Let the data source stay authoritative so deletes and updates flow through the diff; editing the vector store by hand leaves orphaned chunks that outrank fresh content.
8. A stale passage that outranks the current one is worse than a missing one, because the model answers confidently from it.
9. Attach last_updated metadata and filter on it at query time so recent content is preferred where old and new legitimately coexist; it complements deleting stale chunks rather than replacing it.
10. For facts that change faster than any sync (live status, intraday price, current balance) retrieval is the wrong tool; call the system of record at question time and keep only the durable prose in the index.