Feature
Amazon EMR Serverless now supports larger worker sizes to run more compute and memory intensive workloads
Amazon EMR Serverless now offers larger worker configurations of 32 vCPUs with up to 244 GB of memory, allowing you to run more compute and memory-intensive workloads. Previously, the largest worker configuration available on EMR Serverless was 16 vCPUs with up to 120 GB of memory. Larger workers can help you improve the runtime performance as well as cost profiles for your workloads.
For shuffle-heavy workloads, larger workers reduce inefficient data transfers between executors. For jobs with data skew, larger workers reduce the chances of out-of-memory failures. For jobs that need to cache data, larger workers allow holding more data in memory, boosting job performance. To take advantage of these benefits, we recommend using larger workers for your compute and memory-intensive Spark and Hive workloads.
To learn more about different worker configurations, please visit EMR Serverless documentation. Larger workers are available in all AWS Regions where EMR Serverless is available.
Read the original announcement