Scaling EC2 From Prometheus Metrics Instead of Separate CloudWatch Metrics

Ananya Desai

Most EC2 Auto Scaling setups start with CloudWatch metrics.

CPU utilization is the obvious example. If CPU goes above 70 percent, add an instance. If it drops below 30 percent, remove one.

That works well for many applications.

The problem appears when CPU is no longer a good representation of the workload.

A web application may have low CPU usage while the request queue is growing.

A background worker may have moderate CPU usage while thousands of jobs are waiting.

A custom application may expose a metric such as requests per second, active jobs, queue depth, or processing latency that tells you much more about when another EC2 instance is needed.

If the application already exposes those metrics through Prometheus, creating another copy of the same metric in CloudWatch just for Auto Scaling can add unnecessary configuration.

AWS now provides a way to use Prometheus-compatible metrics with CloudWatch and EC2 Auto Scaling. Amazon Managed Service for Prometheus can collect application metrics, while Amazon CloudWatch can use metrics from Amazon Managed Service for Prometheus through the CloudWatch agent and Metric Insights. EC2 Auto Scaling can then use the resulting CloudWatch metric for scaling decisions.

The important idea is simple:

Application
    |
    | Prometheus metrics
    v
Amazon Managed Service for Prometheus
    |
    | CloudWatch integration
    v
Amazon CloudWatch
    |
    | Scaling metric
    v
EC2 Auto Scaling

This lets the scaling signal come from the application workload rather than forcing every scaling decision to be based on infrastructure metrics.

Why CPU Is Not Always the Right Scaling Signal

Imagine an EC2-based order-processing service.

Each instance has four worker processes.

The application exposes:

orders_waiting
orders_processed_total
processing_latency
active_workers

Now imagine the queue grows like this:

10 jobs
25 jobs
80 jobs
200 jobs
500 jobs

But CPU remains around 45 percent.

If your Auto Scaling policy only watches CPU, it may decide that nothing needs to change.

The application sees a completely different situation.

The workers are falling behind.

A better scaling signal could be:

orders_waiting

For example:

0 to 20 jobs       1 instance
20 to 100 jobs     2 instances
100 to 250 jobs    3 instances
250+ jobs          4+ instances

The exact thresholds depend on the application.

The important point is that the metric represents the thing you are trying to control.

If your goal is to keep queue latency low, scaling on CPU alone may not be enough.

A Typical Architecture

A production architecture might look like this:

                   EC2 Application
                         |
                         |
                  Prometheus metrics
                         |
                         v
             Amazon Managed Service
                  for Prometheus
                         |
                         |
                  CloudWatch agent
                         |
                         v
                Amazon CloudWatch
                         |
                         |
                Scaling metric
                         |
                         v
                EC2 Auto Scaling
                         |
              +----------+----------+
              |          |          |
              v          v          v
            EC2        EC2        EC2

There are several separate responsibilities here.

The application is responsible for producing meaningful metrics.

Prometheus is responsible for collecting and storing those metrics.

The CloudWatch integration makes the required metric available to CloudWatch.

CloudWatch provides the metric that the Auto Scaling policy evaluates.

EC2 Auto Scaling changes the number of instances based on that metric.

Keeping those responsibilities separate makes the system easier to troubleshoot.

Start With a Metric That Represents Real Work

Before configuring Auto Scaling, choose the metric carefully.

This is probably the most important part of the entire design.

A bad metric produces bad scaling.

For a web application, useful metrics might include:

Requests per second
Active requests
Request latency
Requests waiting
Queue depth
Error rate

For background workers:

Jobs waiting
Jobs currently processing
Processing latency
Jobs processed per minute

For API services:

Requests per second
Concurrent requests
p95 latency
Work waiting in an internal queue

For image or video processing:

Files waiting
Processing duration
Active workers
Items completed per minute

Try to choose a metric that has a direct relationship with the amount of capacity your application needs.

For example, this is often more meaningful:

jobs_waiting

than:

cpu_utilization

when the workload is a queue processor.

Expose the Metric From the Application

A Prometheus metric should be generated by the application or its supporting infrastructure.

For a .NET application, a Prometheus-compatible metrics library can expose application measurements.

A simplified example might look like this:

public static readonly Gauge JobsWaiting =
    Metrics.CreateGauge(
        "orders_jobs_waiting",
        "Number of orders waiting to be processed");

When the application checks the queue:

JobsWaiting.Set(queueDepth);

The application can expose the metric through its Prometheus endpoint.

For example, the endpoint may return:

orders_jobs_waiting 42

The actual metric naming and instrumentation depend on the library and application architecture.

The important thing is that the metric represents the current workload.

Avoid Metrics That Only Look Useful

Not every application metric is suitable for Auto Scaling.

Consider a metric such as:

total_orders_processed

This is a counter.

It tells you how many orders have been processed since the process started.

It does not directly tell you how much work is waiting right now.

A better scaling signal may be:

orders_waiting

or a rate derived from the counter.

The difference matters because Auto Scaling needs a metric that helps it decide whether additional capacity is required.

Think about the question the metric answers.

Bad question:
How many jobs have we processed since startup?

Better question:
How much work is waiting right now?

Prometheus Metric Types Matter

Prometheus commonly uses metrics such as counters, gauges, histograms, and summaries.

For scaling, a gauge is often convenient for values such as:

queue_depth
active_jobs
connected_users

A counter is useful for values such as:

requests_total
jobs_processed_total
errors_total

A counter can become a useful scaling signal when you calculate a rate.

For example, you might use a PromQL expression that calculates requests per second.

Conceptually:

rate(http_requests_total[5m])

This turns a monotonically increasing counter into a rate.

That can be much more useful for capacity decisions.

Why Use Prometheus Instead of Creating a New CloudWatch Metric?

If your application already uses Prometheus, you may already have the metrics you need.

Without an integration path, teams often create another publishing mechanism:

Application
   |
   +-- Prometheus metric
   |
   +-- CloudWatch metric

Now the application has two metric pipelines.

That can create problems.

The metric names may drift.

The labels may be different.

The aggregation rules may differ.

One system may update faster than the other.

Developers may fix one metric and forget the other.

Using Prometheus as the application metric source reduces some of this duplication.

The overall model becomes:

Application
      |
      v
Prometheus instrumentation
      |
      +----------------+
      |                |
      v                v
Prometheus          CloudWatch
analysis             scaling

Prometheus remains useful for observability.

CloudWatch provides the AWS-native integration needed by Auto Scaling.

Use Labels Carefully

Prometheus labels are powerful, but they can create problems if you use too many unique values.

For example:

orders_waiting{instance="i-123456"}

is reasonable.

But a metric such as:

request_count{user_id="123456"}

can produce extremely high cardinality if the application has millions of users.

This is especially important when metrics are being transferred between monitoring systems.

For scaling, keep the metric focused.

A good metric might be:

orders_waiting

with a small number of dimensions such as:

environment
service
queue

Avoid putting request IDs, user IDs, transaction IDs, or other highly unique values into scaling metrics.

Create a PromQL Expression for Scaling

The next step is deciding what Prometheus should calculate.

Suppose your application exposes:

orders_waiting

You might not want to scale directly from the raw value.

Instead, you could aggregate across instances.

For example:

sum(orders_waiting)

Now the metric represents the total number of waiting jobs across the service.

This is often better than scaling independently from each EC2 instance.

Another example is request rate:

sum(rate(http_requests_total[5m]))

This gives you the total request rate across the service.

The exact PromQL expression depends on your application.

The key question is:

What single value should represent the workload that the Auto Scaling group needs to handle?

Be Careful With Per-Instance Metrics

Suppose you have three EC2 instances.

Their queue depths are:

Instance A    100
Instance B    20
Instance C    0

If you scale using an average, the result is around:

40 jobs per instance

But that can hide uneven distribution.

If the queue is shared, total queue depth may be a better metric:

sum(orders_waiting)

If each instance has an independent queue, the scaling design may need to consider the maximum or another workload-specific aggregation.

The correct aggregation depends on the architecture.

Do not blindly use average, sum, or maximum just because it is available.

Connect the Metric to CloudWatch

EC2 Auto Scaling policies operate using CloudWatch metrics.

That means the Prometheus metric needs to become available through CloudWatch before it can be used by the scaling policy.

The AWS architecture can use the CloudWatch agent to retrieve Prometheus metrics and publish the required metric into CloudWatch.

At a high level:

Prometheus-compatible endpoint
          |
          v
CloudWatch agent
          |
          v
CloudWatch metric
          |
          v
Auto Scaling policy

The CloudWatch agent configuration is where you define the Prometheus scraping and metric transformation behavior.

Keep this configuration focused.

Do not send every application metric to CloudWatch simply because Auto Scaling needs one of them.

If the application exposes hundreds of metrics, publish only the metrics needed for the AWS-side scaling and monitoring workflows.

Use a Dedicated Scaling Metric

It is a good practice to create a metric specifically designed for scaling.

For example:

orders_waiting_for_scaling

This is clearer than allowing several teams to independently decide which Prometheus metric should control the production Auto Scaling group.

A dedicated metric also makes the intent obvious during an incident.

Someone reviewing the Auto Scaling policy can immediately understand:

Scaling is based on waiting orders.

rather than:

Scaling is based on some application metric with five labels and an undocumented PromQL expression.

Choose a Target Carefully

Suppose you decide to scale based on queue depth.

You need to decide what queue depth should represent normal capacity.

For example:

Target:
50 waiting jobs

The idea is not necessarily:

If queue depth > 50, add one instance immediately.

Instead, the Auto Scaling policy should use the metric and target to continuously adjust capacity according to the configured scaling behavior.

The correct target depends on:

How quickly one instance processes jobs
How quickly jobs arrive
How long users can wait
How quickly EC2 instances become ready
How much spare capacity you want
How bursty the workload is

Do not choose the target simply because the number looks reasonable.

Measure it.

Account for Instance Startup Time

This is one of the most common mistakes with custom scaling metrics.

Suppose:

Queue begins growing
        |
        v
Auto Scaling detects growth
        |
        v
New EC2 instance starts
        |
        v
Application initializes
        |
        v
Instance becomes ready

That can take time.

If your workload grows very quickly, scaling only after the queue becomes large may be too late.

You may need:

A lower scaling target
More aggressive scale-out behavior
Warm instances
Faster application startup
Larger scale-out increments

The metric tells Auto Scaling that capacity is needed.

It does not remove the physical time required to create and initialize an EC2 instance.

Scale In Needs More Care

Scale-out and scale-in are not equally safe.

Suppose queue depth drops from:

to:

You may think it is immediately safe to remove instances.

But the queue could start growing again a few seconds later.

If you scale in too aggressively, you can create a cycle:

Scale out
    |
    v
Work finishes
    |
    v
Scale in
    |
    v
Work arrives
    |
    v
Scale out again

This is called scaling oscillation.

Use appropriate scale-in behavior, cooldowns, and stabilization.

The goal is not to maintain the smallest possible instance count every second.

The goal is to maintain enough capacity without repeatedly adding and removing instances.

Test the Metric Before Enabling Scaling

Before allowing the metric to control production capacity, watch it without attaching it to Auto Scaling.

For example:

Week 1
    |
    +-- Observe metric
    +-- Compare with CPU
    +-- Compare with latency
    +-- Compare with request rate
    +-- Measure workload behavior

You want to understand how the metric behaves during:

Normal traffic
Morning traffic spikes
Batch jobs
Failures
Deployment windows
Downstream service failures
Queue growth
Queue recovery

If the metric is noisy, Auto Scaling will see that noise too.

A Good Scaling Metric Should Be Predictable

Suppose your metric looks like this:

It may be noisy.

A metric like:

may be easier to use for a queue-based scaling decision.

This does not mean you should artificially smooth every metric.

Instead, understand the natural behavior of the workload and choose an appropriate aggregation or evaluation period.

Sometimes a five-minute rate is more useful than an instantaneous value.

Sometimes an average is better.

Sometimes you need a percentile.

The scaling metric should match the behavior you are trying to control.

Combine Prometheus Metrics With Existing CloudWatch Metrics

You do not have to replace every CloudWatch metric.

A production Auto Scaling design can use multiple signals.

For example:

Primary workload metric
    |
    +-- Queue depth

Safety metrics
    |
    +-- CPU
    +-- Memory
    +-- Error rate
    +-- Application health

The queue depth can drive scaling while CPU and application health remain useful for monitoring and protection.

This is often better than trying to make one metric answer every operational question.

Example: Scaling a .NET Background Worker

Consider a .NET worker application running on EC2.

It processes customer export jobs.

Each instance can process about 20 jobs per minute.

The application exposes:

export_jobs_waiting
export_jobs_processed_total

Suppose the system normally receives 10 to 15 jobs per minute.

One instance is enough.

During a traffic spike, the queue begins growing:

20 jobs
45 jobs
90 jobs
150 jobs

A Prometheus metric can expose the current queue size:

export_jobs_waiting 150

The scaling pipeline can then use an aggregated value:

sum(export_jobs_waiting)

The resulting CloudWatch metric can be used by the Auto Scaling policy.

The flow becomes:

Jobs arrive
    |
    v
Queue grows
    |
    v
Application exposes export_jobs_waiting
    |
    v
Prometheus collects metric
    |
    v
CloudWatch receives scaling metric
    |
    v
Auto Scaling increases EC2 capacity
    |
    v
More workers process jobs
    |
    v
Queue decreases
    |
    v
Auto Scaling eventually reduces capacity

This is much closer to the application's real workload than simply watching CPU.

Monitor the Scaling System Itself

A common mistake is to monitor the application but not the scaling pipeline.

You should be able to answer:

Is the application exposing the metric?

Is Prometheus receiving it?

Is the CloudWatch integration receiving it?

Is CloudWatch publishing the expected value?

Is Auto Scaling reading the expected metric?

Did Auto Scaling make the expected decision?

Did the new instance become healthy?

If the queue is growing but Auto Scaling is not reacting, the problem may not be the application.

It could be the metric pipeline.

For troubleshooting, trace the metric from beginning to end.

Common Problems

The Metric Stops Updating

If the application stops exposing the metric, CloudWatch may stop receiving fresh values.

Do not assume that a missing metric means zero workload.

A missing metric and a metric with value zero are different situations.

Your scaling design should account for missing data safely.

The Metric Has Too Many Labels

High-cardinality labels can make the metric expensive and difficult to manage.

Keep scaling metrics simple.

The Wrong Aggregation Is Used

Using an average when total queue depth is required can cause under-scaling.

Using a sum when each instance has an independent workload may produce misleading results.

Scale-In Happens Too Quickly

This can cause instances to repeatedly start and stop.

Give the workload enough time to stabilize.

Scaling Starts Too Late

If instances take several minutes to become ready, a high threshold may cause users to experience latency before capacity arrives.

The Metric Does Not Represent Capacity

A metric can be interesting without being useful for scaling.

Choose a metric that has a measurable relationship with required compute capacity.

When CloudWatch Metrics Are Still the Better Choice

Prometheus-based scaling is not automatically better.

CloudWatch is already deeply integrated with EC2 and Auto Scaling.

If CPU utilization, network throughput, or another standard CloudWatch metric accurately represents your workload, there is little reason to introduce another metric path.

Prometheus becomes more attractive when:

The application already exposes Prometheus metrics.
The workload is not well represented by CPU.
The scaling decision depends on application-level information.
You already use Amazon Managed Service for Prometheus.
You need PromQL to calculate the scaling signal.
You want the same application metric for observability and scaling.

The goal should not be to replace CloudWatch.

The goal is to use the right signal for the workload.

Prometheus Scaling vs Traditional CloudWatch Scaling

Area	Traditional CloudWatch metric	Prometheus-based metric
Typical source	EC2 or AWS service	Application or Prometheus exporter
Best for	Infrastructure signals	Application workload
Example	CPU utilization	Queue depth
Query language	CloudWatch metric/statistics	PromQL
Existing AWS integration	Very strong	Requires metric integration
Application awareness	Usually lower	Usually higher
Setup complexity	Lower for standard metrics	Higher
Best use case	Simple infrastructure scaling	Custom workload scaling

The right choice depends on what actually drives capacity.

Production Checklist

Before using a Prometheus-derived metric for EC2 Auto Scaling, verify:

The metric represents real workload pressure.
The metric is available consistently.
The metric does not have unnecessary high-cardinality labels.
The PromQL aggregation matches the application architecture.
The CloudWatch metric updates reliably.
Missing metrics are handled safely.
The target value is based on measured capacity.
Scale-out happens early enough for instance startup time.
Scale-in is conservative enough to avoid oscillation.
The application becomes healthy before receiving production traffic.
Existing CPU and health metrics remain available for troubleshooting.
The scaling pipeline itself is monitored.
The behavior has been tested under realistic traffic.

Summary

EC2 Auto Scaling does not have to depend entirely on CPU utilization or other standard infrastructure metrics.

If your application already exposes useful Prometheus metrics, those metrics can provide a much better picture of workload pressure.

A queue processor may need to scale from queue depth.

An API service may need to scale from request rate.

A worker platform may need to scale from active jobs.

A data-processing service may need to scale from pending work.

The important part is choosing a metric that describes the actual capacity problem.

The architecture can then connect that application metric to AWS Auto Scaling:

Application
    |
    v
Prometheus metric
    |
    v
Amazon Managed Service for Prometheus
    |
    v
CloudWatch integration
    |
    v
CloudWatch metric
    |
    v
EC2 Auto Scaling

This approach adds some configuration compared with a simple CPU-based policy, so it should not be introduced without a reason.

But when CPU does not represent the workload, using an application-level Prometheus metric can give Auto Scaling a much better signal and can eliminate the need to invent a separate application metric pipeline just for scaling.

AWS EC2 Auto ScalingPrometheus metricsAmazon CloudWatchAmazon Managed Service for Prometheuscustom scaling metricsqueue depth

Comments

Join the conversation! Your thoughts help the community grow.

Scaling EC2 From Prometheus Metrics Instead of Separate CloudWatch Metrics

Ananya Desai

24m100100Article

Most EC2 Auto Scaling setups start with CloudWatch metrics.

CPU utilization is the obvious example. If CPU goes above 70 percent, add an instance. If it drops below 30 percent, remove one.

That works well for many applications.

The problem appears when CPU is no longer a good representation of the workload.

A web application may have low CPU usage while the request queue is growing.

A background worker may have moderate CPU usage while thousands of jobs are waiting.

A custom application may expose a metric such as requests per second, active jobs, queue depth, or processing latency that tells you much more about when another EC2 instance is needed.

If the application already exposes those metrics through Prometheus, creating another copy of the same metric in CloudWatch just for Auto Scaling can add unnecessary configuration.

The important idea is simple:

Application
    |
    | Prometheus metrics
    v
Amazon Managed Service for Prometheus
    |
    | CloudWatch integration
    v
Amazon CloudWatch
    |
    | Scaling metric
    v
EC2 Auto Scaling

This lets the scaling signal come from the application workload rather than forcing every scaling decision to be based on infrastructure metrics.

Why CPU Is Not Always the Right Scaling Signal

Imagine an EC2-based order-processing service.

Each instance has four worker processes.

The application exposes:

orders_waiting
orders_processed_total
processing_latency
active_workers

Now imagine the queue grows like this:

10 jobs
25 jobs
80 jobs
200 jobs
500 jobs

But CPU remains around 45 percent.

If your Auto Scaling policy only watches CPU, it may decide that nothing needs to change.

The application sees a completely different situation.

The workers are falling behind.

A better scaling signal could be:

orders_waiting

For example:

0 to 20 jobs       1 instance
20 to 100 jobs     2 instances
100 to 250 jobs    3 instances
250+ jobs          4+ instances

The exact thresholds depend on the application.

The important point is that the metric represents the thing you are trying to control.

If your goal is to keep queue latency low, scaling on CPU alone may not be enough.

A Typical Architecture

A production architecture might look like this:

                   EC2 Application
                         |
                         |
                  Prometheus metrics
                         |
                         v
             Amazon Managed Service
                  for Prometheus
                         |
                         |
                  CloudWatch agent
                         |
                         v
                Amazon CloudWatch
                         |
                         |
                Scaling metric
                         |
                         v
                EC2 Auto Scaling
                         |
              +----------+----------+
              |          |          |
              v          v          v
            EC2        EC2        EC2

There are several separate responsibilities here.

The application is responsible for producing meaningful metrics.

Prometheus is responsible for collecting and storing those metrics.

The CloudWatch integration makes the required metric available to CloudWatch.

CloudWatch provides the metric that the Auto Scaling policy evaluates.

EC2 Auto Scaling changes the number of instances based on that metric.

Keeping those responsibilities separate makes the system easier to troubleshoot.

Start With a Metric That Represents Real Work

Before configuring Auto Scaling, choose the metric carefully.

This is probably the most important part of the entire design.

A bad metric produces bad scaling.

For a web application, useful metrics might include:

Requests per second
Active requests
Request latency
Requests waiting
Queue depth
Error rate

For background workers:

Jobs waiting
Jobs currently processing
Processing latency
Jobs processed per minute

For API services:

Requests per second
Concurrent requests
p95 latency
Work waiting in an internal queue

For image or video processing:

Files waiting
Processing duration
Active workers
Items completed per minute

Try to choose a metric that has a direct relationship with the amount of capacity your application needs.

For example, this is often more meaningful:

jobs_waiting

than:

cpu_utilization

when the workload is a queue processor.

Expose the Metric From the Application

A Prometheus metric should be generated by the application or its supporting infrastructure.

For a .NET application, a Prometheus-compatible metrics library can expose application measurements.

A simplified example might look like this:

public static readonly Gauge JobsWaiting =
    Metrics.CreateGauge(
        "orders_jobs_waiting",
        "Number of orders waiting to be processed");

When the application checks the queue:

JobsWaiting.Set(queueDepth);

The application can expose the metric through its Prometheus endpoint.

For example, the endpoint may return:

orders_jobs_waiting 42

The actual metric naming and instrumentation depend on the library and application architecture.

The important thing is that the metric represents the current workload.

Avoid Metrics That Only Look Useful

Not every application metric is suitable for Auto Scaling.

Consider a metric such as:

total_orders_processed

This is a counter.

It tells you how many orders have been processed since the process started.

It does not directly tell you how much work is waiting right now.

A better scaling signal may be:

orders_waiting

or a rate derived from the counter.

The difference matters because Auto Scaling needs a metric that helps it decide whether additional capacity is required.

Think about the question the metric answers.

Bad question:
How many jobs have we processed since startup?

Better question:
How much work is waiting right now?

Prometheus Metric Types Matter

Prometheus commonly uses metrics such as counters, gauges, histograms, and summaries.

For scaling, a gauge is often convenient for values such as:

queue_depth
active_jobs
connected_users

A counter is useful for values such as:

requests_total
jobs_processed_total
errors_total

A counter can become a useful scaling signal when you calculate a rate.

For example, you might use a PromQL expression that calculates requests per second.

Conceptually:

rate(http_requests_total[5m])

This turns a monotonically increasing counter into a rate.

That can be much more useful for capacity decisions.

Why Use Prometheus Instead of Creating a New CloudWatch Metric?

If your application already uses Prometheus, you may already have the metrics you need.

Without an integration path, teams often create another publishing mechanism:

Application
   |
   +-- Prometheus metric
   |
   +-- CloudWatch metric

Now the application has two metric pipelines.

That can create problems.

The metric names may drift.

The labels may be different.

The aggregation rules may differ.

One system may update faster than the other.

Developers may fix one metric and forget the other.

Using Prometheus as the application metric source reduces some of this duplication.

The overall model becomes:

Application
      |
      v
Prometheus instrumentation
      |
      +----------------+
      |                |
      v                v
Prometheus          CloudWatch
analysis             scaling

Prometheus remains useful for observability.

CloudWatch provides the AWS-native integration needed by Auto Scaling.

Use Labels Carefully

Prometheus labels are powerful, but they can create problems if you use too many unique values.

For example:

orders_waiting{instance="i-123456"}

is reasonable.

But a metric such as:

request_count{user_id="123456"}

can produce extremely high cardinality if the application has millions of users.

This is especially important when metrics are being transferred between monitoring systems.

For scaling, keep the metric focused.

A good metric might be:

orders_waiting

with a small number of dimensions such as:

environment
service
queue

Avoid putting request IDs, user IDs, transaction IDs, or other highly unique values into scaling metrics.

Create a PromQL Expression for Scaling

The next step is deciding what Prometheus should calculate.

Suppose your application exposes:

orders_waiting

You might not want to scale directly from the raw value.

Instead, you could aggregate across instances.

For example:

sum(orders_waiting)

Now the metric represents the total number of waiting jobs across the service.

This is often better than scaling independently from each EC2 instance.

Another example is request rate:

sum(rate(http_requests_total[5m]))

This gives you the total request rate across the service.

The exact PromQL expression depends on your application.

The key question is:

What single value should represent the workload that the Auto Scaling group needs to handle?

Be Careful With Per-Instance Metrics

Suppose you have three EC2 instances.

Their queue depths are:

Instance A    100
Instance B    20
Instance C    0

If you scale using an average, the result is around:

40 jobs per instance

But that can hide uneven distribution.

If the queue is shared, total queue depth may be a better metric:

sum(orders_waiting)

If each instance has an independent queue, the scaling design may need to consider the maximum or another workload-specific aggregation.

The correct aggregation depends on the architecture.

Do not blindly use average, sum, or maximum just because it is available.

Connect the Metric to CloudWatch

EC2 Auto Scaling policies operate using CloudWatch metrics.

That means the Prometheus metric needs to become available through CloudWatch before it can be used by the scaling policy.

The AWS architecture can use the CloudWatch agent to retrieve Prometheus metrics and publish the required metric into CloudWatch.

At a high level:

Prometheus-compatible endpoint
          |
          v
CloudWatch agent
          |
          v
CloudWatch metric
          |
          v
Auto Scaling policy

The CloudWatch agent configuration is where you define the Prometheus scraping and metric transformation behavior.

Keep this configuration focused.

Do not send every application metric to CloudWatch simply because Auto Scaling needs one of them.

If the application exposes hundreds of metrics, publish only the metrics needed for the AWS-side scaling and monitoring workflows.

Use a Dedicated Scaling Metric

It is a good practice to create a metric specifically designed for scaling.

For example:

orders_waiting_for_scaling

This is clearer than allowing several teams to independently decide which Prometheus metric should control the production Auto Scaling group.

A dedicated metric also makes the intent obvious during an incident.

Someone reviewing the Auto Scaling policy can immediately understand:

Scaling is based on waiting orders.

rather than:

Scaling is based on some application metric with five labels and an undocumented PromQL expression.

Choose a Target Carefully

Suppose you decide to scale based on queue depth.

You need to decide what queue depth should represent normal capacity.

For example:

Target:
50 waiting jobs

The idea is not necessarily:

If queue depth > 50, add one instance immediately.

Instead, the Auto Scaling policy should use the metric and target to continuously adjust capacity according to the configured scaling behavior.

The correct target depends on:

How quickly one instance processes jobs
How quickly jobs arrive
How long users can wait
How quickly EC2 instances become ready
How much spare capacity you want
How bursty the workload is

Do not choose the target simply because the number looks reasonable.

Measure it.

Account for Instance Startup Time

This is one of the most common mistakes with custom scaling metrics.

Suppose:

Queue begins growing
        |
        v
Auto Scaling detects growth
        |
        v
New EC2 instance starts
        |
        v
Application initializes
        |
        v
Instance becomes ready

That can take time.

If your workload grows very quickly, scaling only after the queue becomes large may be too late.

You may need:

A lower scaling target
More aggressive scale-out behavior
Warm instances
Faster application startup
Larger scale-out increments

The metric tells Auto Scaling that capacity is needed.

It does not remove the physical time required to create and initialize an EC2 instance.

Scale In Needs More Care

Scale-out and scale-in are not equally safe.

Suppose queue depth drops from:

to:

You may think it is immediately safe to remove instances.

But the queue could start growing again a few seconds later.

If you scale in too aggressively, you can create a cycle:

Scale out
    |
    v
Work finishes
    |
    v
Scale in
    |
    v
Work arrives
    |
    v
Scale out again

This is called scaling oscillation.

Use appropriate scale-in behavior, cooldowns, and stabilization.

The goal is not to maintain the smallest possible instance count every second.

The goal is to maintain enough capacity without repeatedly adding and removing instances.

Test the Metric Before Enabling Scaling

Before allowing the metric to control production capacity, watch it without attaching it to Auto Scaling.

For example:

Week 1
    |
    +-- Observe metric
    +-- Compare with CPU
    +-- Compare with latency
    +-- Compare with request rate
    +-- Measure workload behavior

You want to understand how the metric behaves during:

Normal traffic
Morning traffic spikes
Batch jobs
Failures
Deployment windows
Downstream service failures
Queue growth
Queue recovery

If the metric is noisy, Auto Scaling will see that noise too.

A Good Scaling Metric Should Be Predictable

Suppose your metric looks like this:

It may be noisy.

A metric like:

may be easier to use for a queue-based scaling decision.

This does not mean you should artificially smooth every metric.

Instead, understand the natural behavior of the workload and choose an appropriate aggregation or evaluation period.

Sometimes a five-minute rate is more useful than an instantaneous value.

Sometimes an average is better.

Sometimes you need a percentile.

The scaling metric should match the behavior you are trying to control.

Combine Prometheus Metrics With Existing CloudWatch Metrics

You do not have to replace every CloudWatch metric.

A production Auto Scaling design can use multiple signals.

For example:

Primary workload metric
    |
    +-- Queue depth

Safety metrics
    |
    +-- CPU
    +-- Memory
    +-- Error rate
    +-- Application health

The queue depth can drive scaling while CPU and application health remain useful for monitoring and protection.

This is often better than trying to make one metric answer every operational question.

Example: Scaling a .NET Background Worker

Consider a .NET worker application running on EC2.

It processes customer export jobs.

Each instance can process about 20 jobs per minute.

The application exposes:

export_jobs_waiting
export_jobs_processed_total

Suppose the system normally receives 10 to 15 jobs per minute.

One instance is enough.

During a traffic spike, the queue begins growing:

20 jobs
45 jobs
90 jobs
150 jobs

A Prometheus metric can expose the current queue size:

export_jobs_waiting 150

The scaling pipeline can then use an aggregated value:

sum(export_jobs_waiting)

The resulting CloudWatch metric can be used by the Auto Scaling policy.

The flow becomes:

Jobs arrive
    |
    v
Queue grows
    |
    v
Application exposes export_jobs_waiting
    |
    v
Prometheus collects metric
    |
    v
CloudWatch receives scaling metric
    |
    v
Auto Scaling increases EC2 capacity
    |
    v
More workers process jobs
    |
    v
Queue decreases
    |
    v
Auto Scaling eventually reduces capacity

This is much closer to the application's real workload than simply watching CPU.

Monitor the Scaling System Itself

A common mistake is to monitor the application but not the scaling pipeline.

You should be able to answer:

Is the application exposing the metric?

Is Prometheus receiving it?

Is the CloudWatch integration receiving it?

Is CloudWatch publishing the expected value?

Is Auto Scaling reading the expected metric?

Did Auto Scaling make the expected decision?

Did the new instance become healthy?

If the queue is growing but Auto Scaling is not reacting, the problem may not be the application.

It could be the metric pipeline.

For troubleshooting, trace the metric from beginning to end.

Common Problems

The Metric Stops Updating

If the application stops exposing the metric, CloudWatch may stop receiving fresh values.

Do not assume that a missing metric means zero workload.

A missing metric and a metric with value zero are different situations.

Your scaling design should account for missing data safely.

The Metric Has Too Many Labels

High-cardinality labels can make the metric expensive and difficult to manage.

Keep scaling metrics simple.

The Wrong Aggregation Is Used

Using an average when total queue depth is required can cause under-scaling.

Using a sum when each instance has an independent workload may produce misleading results.

Scale-In Happens Too Quickly

This can cause instances to repeatedly start and stop.

Give the workload enough time to stabilize.

Scaling Starts Too Late

If instances take several minutes to become ready, a high threshold may cause users to experience latency before capacity arrives.

The Metric Does Not Represent Capacity

A metric can be interesting without being useful for scaling.

Choose a metric that has a measurable relationship with required compute capacity.

When CloudWatch Metrics Are Still the Better Choice

Prometheus-based scaling is not automatically better.

CloudWatch is already deeply integrated with EC2 and Auto Scaling.

If CPU utilization, network throughput, or another standard CloudWatch metric accurately represents your workload, there is little reason to introduce another metric path.

Prometheus becomes more attractive when:

The application already exposes Prometheus metrics.
The workload is not well represented by CPU.
The scaling decision depends on application-level information.
You already use Amazon Managed Service for Prometheus.
You need PromQL to calculate the scaling signal.
You want the same application metric for observability and scaling.

The goal should not be to replace CloudWatch.

The goal is to use the right signal for the workload.

Prometheus Scaling vs Traditional CloudWatch Scaling

Area	Traditional CloudWatch metric	Prometheus-based metric
Typical source	EC2 or AWS service	Application or Prometheus exporter
Best for	Infrastructure signals	Application workload
Example	CPU utilization	Queue depth
Query language	CloudWatch metric/statistics	PromQL
Existing AWS integration	Very strong	Requires metric integration
Application awareness	Usually lower	Usually higher
Setup complexity	Lower for standard metrics	Higher
Best use case	Simple infrastructure scaling	Custom workload scaling

The right choice depends on what actually drives capacity.

Production Checklist

Before using a Prometheus-derived metric for EC2 Auto Scaling, verify:

The metric represents real workload pressure.
The metric is available consistently.
The metric does not have unnecessary high-cardinality labels.
The PromQL aggregation matches the application architecture.
The CloudWatch metric updates reliably.
Missing metrics are handled safely.
The target value is based on measured capacity.
Scale-out happens early enough for instance startup time.
Scale-in is conservative enough to avoid oscillation.
The application becomes healthy before receiving production traffic.
Existing CPU and health metrics remain available for troubleshooting.
The scaling pipeline itself is monitored.
The behavior has been tested under realistic traffic.

Summary

EC2 Auto Scaling does not have to depend entirely on CPU utilization or other standard infrastructure metrics.

If your application already exposes useful Prometheus metrics, those metrics can provide a much better picture of workload pressure.

A queue processor may need to scale from queue depth.

An API service may need to scale from request rate.

A worker platform may need to scale from active jobs.

A data-processing service may need to scale from pending work.

The important part is choosing a metric that describes the actual capacity problem.

The architecture can then connect that application metric to AWS Auto Scaling:

Application
    |
    v
Prometheus metric
    |
    v
Amazon Managed Service for Prometheus
    |
    v
CloudWatch integration
    |
    v
CloudWatch metric
    |
    v
EC2 Auto Scaling

This approach adds some configuration compared with a simple CPU-based policy, so it should not be introduced without a reason.

AWS EC2 Auto ScalingPrometheus metricsAmazon CloudWatchAmazon Managed Service for Prometheuscustom scaling metricsqueue depth

Comments

Join the conversation! Your thoughts help the community grow.