AWS Announces EMR Managed Scaling

Amazon EMR Managed Scaling automatically resizes your cluster to the lowest possible cost.

Recently, AWS announced the release of Amazon EMR Managed Scaling, which is a new feature that automatically resizes your cluster for best performance at the lowest possible cost.
 
According to AWS with EMR Managed Scaling you will need to specify just the minimum and maximum compute limits for your clusters and Amazon EMR will automatically resize them for best performance and resource utilization.
 
The solution continuously samples key metrics associated with the workloads running on clusters and is supported for Apache Spark, Apache Hive and YARN-based workloads on Amazon EMR versions 5.30.1 and above.
 
Well, Amazon EMR offers two ways to scale your clusters "Amazon EMR’s support for Auto Scaling released in 2016" and "EMR Managed Scaling". If you want a completely managed experience while running Apache Spark, Apache Hive, or YARN-based applications AWS recommends using EMR Managed Scaling. In cases where you need to define custom rules involving custom metrics for applications running in the cluster, AWS recommends use of Auto Scaling.
 
EMR Managed Scaling also introduces support for Amazon EMR instance fleets. It allows you to seamlessly scale Spot Instances, On-Demand Instances, and instances that are part of a Savings Plan all within the same cluster.
 
 
Source: Amazon
 
EMR Managed Scaling enables you to constrain the minimum and maximum capacity that the cluster can scale up to. The parameters that let you control these are MinimumCapacityUnits, MaximumCapacityUnits, MaximumCoreCapacityUnits, and MaximumOnDemandCapacityUnits.