Generative AI Inference Recommendation for Amazon SageMaker now available in the SageMaker AI Studio

Amazon SageMaker AI now offers Generative AI Inference Recommendations in SageMaker AI Studio, giving customers a guided, low-code, no-code path to find the best inference configuration for their workload. This builds on the API-based launch in April 2026, extending the same benchmarking infrastructure to teams that prefer a visual workflow over programmatic access.

Deploying generative AI models in production requires finding the right combination of instance type, serving container, and optimization strategy. Getting this right typically involves weeks of manual benchmarking, configuration tuning, and trial-and-error, with no easy way to know if the final setup is actually optimal. With the new experience, customers describe their workload and what matters most, whether that's latency, throughput, or cost, and SageMaker AI does the rest. It benchmarks multiple configurations on real GPU infrastructure using NVIDIA AIPerf, applies goal-aligned techniques like speculative decoding for throughput or kernel tuning for latency, and returns ranked, production-ready recommendations with measured performance data. Teams get to a validated configuration in hours instead of weeks, without needing to decide which techniques to apply or how to configure them.

With the new experience, customers describe their workload and what matters most, whether that's latency, throughput, or cost, and SageMaker AI does the rest. It benchmarks multiple configurations on real GPU infrastructure using NVIDIA AIPerf, applies goal-aligned techniques like speculative decoding for throughput or kernel tuning for latency, and returns ranked, production-ready recommendations with measured performance data. Teams get to a validated configuration in hours instead of weeks, without needing to decide which techniques to apply or how to configure them.

In SageMaker AI Studio under Jobs, Inference optimization, customers select a use-case profile (Interact, Generate, Summarize, or Custom), choose an optimization goal (minimize latency, maximize throughput, or minimize cost), and pick their model from JumpStart, S3, Model Registry, or an existing SageMaker model. Recommendations are ranked by TTFT, inter-token latency, throughput, and cost, and can be compared visually before deploying to a SageMaker real-time endpoint directly from Studio.

There is no additional cost for generating recommendations. Standard compute costs apply for optimization jobs and endpoints provisioned during benchmarking. This capability is available in US East (N. Virginia), US West (Oregon), US East (Ohio), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Singapore), Asia Pacific (Tokyo). To learn more, visit the blog post