Exam-style
A support-triage application on Bedrock costs AUD$40,000 a month. Most calls are simple classification of a short ticket, they all go to the largest available model, and every request repeats the same 4,000-token policy document as context. Answer quality must not drop. What are the first levers?
Reveal the answer
E. Route the classification calls to a smaller model and cache the repeated policy context
Bedrock cost is mostly model choice multiplied by tokens, so matching model strength to the task and caching the context that repeats on every call attacks both terms at once without changing answers a capable smaller model already gets right. Provisioned Throughput suits steady high volume but commits spend to traffic that has not been right-sized yet. Fine-tuning adds training and hosting cost to solve a model-selection problem. Batch inference is cheaper but asynchronous, and a triage queue needs answers now; a Guardrail filters content rather than reducing the cost of the calls that remain.
Q. Your Bedrock bill is high but quality must hold. First levers?
A. Right-size the model per task (a smaller model where it suffices), cache repeated responses and context, trim prompt and output tokens, and batch where latency allows. Reserve Provisioned ThroughputReserved Bedrock capacity bought by the hour for a fixed term, paid for whether traffic fills it or not. only for steady load.
Why? Most GenAI cost is tokens and model choice; match model strength to the task.