Cheat Sheet: Model Selection and Inference

A fast revision sheet for model choice and inference options across Amazon Bedrock and self-hosted SageMaker.

Services at a glance

Decision rules

• If the task is text-only and high-volume, start at Nova Micro and only move up if quality falls short.
• If input includes images or video, you need a multimodal model (Nova Lite/Pro, Claude, others), not Micro or Titan text.
• If you need to generate images, reach for Stability; for video, Luma Ray 2. Nova Canvas and Nova Reel are legacy, reach end of life on 30 September 2026, and are not a choice for new work.
• If you need embeddings for RAG, reach for Titan Embeddings or Cohere Embed, not a chat model.
• If traffic is spiky and low commitment, use on-demand.
• If volume is steady and high, price out Provisioned ThroughputProvisioned ThroughputReserved Bedrock capacity bought by the hour for a fixed term, paid for whether traffic fills it or not. against on-demand.
• If you fine-tune or import a custom model, you must serve it on Provisioned Throughput.
• If the job is large and offline with no latency need, use batch inferenceBatch inferenceSubmitting a bulk job of model calls to run asynchronously at a lower per-token price, trading immediacy for cost. for the discount.
• If a single region’s throughput quota is the limit, enable a cross-region inferenceCross-region inferenceLetting a request be served from any of several regions, raising effective throughput and riding out pressure in one of them. profile.
• If data must stay in one geography, scope the inference profile to same-geography regions and check residency.
• If the model is not on Bedrock, host it on SageMaker.
• If a SageMaker endpoint sits idle between bursts, use serverless to scale to zero and accept cold starts.
• If payloads are large or processing is slow, use SageMaker asynchronous inference with its queue.
• If you score a whole dataset once with no live endpoint, use SageMaker batch transform.
• If you host many similar models cheaply behind one endpoint, use a multi-model endpoint.
• If requests vary in difficulty, put Intelligent Prompt Routing in front and let it pick the cheapest tier that clears the bar.

Traps

• Nova Micro is text-only; picking it for an image or video task is wrong.
• Nova Canvas and Nova Reel are on the legacy list; an account with no recent history of calling them is refused access, and both are withdrawn entirely on 30 September 2026, so do not design new work around either.
• Video generation is asynchronous: Ray 2 runs through StartAsyncInvoke and delivers to S3, so there is no synchronous response to wait on.
• Provisioned Throughput is not just a cost lever; it is required for custom and imported models, not optional.
• Batch inference is asynchronous and offline; it does not lower latency for live requests.
• Cross-region inference profiles move data across regions, which can breach a data-residency requirement.
• On-demand is quota-bound; hitting throttling means raising quota or moving to Provisioned Throughput, not retrying blindly.
• SageMaker serverless has cold starts; do not pick it when consistent low latency matters.
• Enabling model access in the console is a prerequisite; a model you have not enabled will not answer.
• Model availability differs by region; a model in one region may be absent in another.
• Bigger is not automatically better; the smallest model that clears the quality bar is usually the right pick on cost and latency.
• SageMaker batch transform has no persistent endpoint; do not use it for real-time serving.
• Asynchronous inference is for large payloads and long jobs, not a substitute for provisioned real-time throughput.
• Fine-tuning changes the model you must provision; you cannot serve a fine-tuned model on plain on-demand.

Say it in one line

1. Nova Micro is text-only; Lite and Pro are multimodal; Canvas and Reel are legacy, so images come from Stability AI and video from Luma Ray 2.
2. Titan and Cohere give you embeddings; use them for RAG and search, not chat.
3. On-demand charges per token with no commitment and is bound by service quotas.
4. Provisioned Throughput reserves model unitsModel unitThe billing block Provisioned Throughput is sold in – one unit delivers a fixed tokens-per-minute rate for a specific model. per hour and is mandatory for custom and imported models.
5. Batch inference is async S3-to-S3, roughly half price, and offline only.
6. Cross-region inference profiles raise throughput by spreading load and can carry data-residency implications.
7. Enabling model access and checking regional availability are prerequisites before any call.
8. SageMaker real-time serves low-latency traffic on an always-on endpoint.
9. SageMaker serverless scales to zero and trades cold starts for idle savings.
10. SageMaker asynchronous inference queues large payloads and long-running jobs.
11. SageMaker batch transform scores a dataset offline with no persistent endpoint.
12. Multi-model endpoints host many similar models cheaply behind one endpoint.
13. Intelligent Prompt Routing routes within a family to the cheapest tier that meets a quality bar.
14. Pick the smallest model that clears the quality bar; move up only when it does not.