Temporal helps developers build reliable applications that execute long-running workflows, coordinate background tasks, and recover from failures. It is useful for order processing, payment workflows, document generation, scheduled operations, and other business processes that may take seconds, minutes, or even days to complete.
However, running Temporal Workers introduces an infrastructure challenge. Workers poll Temporal Task Queues and execute the tasks assigned to them. If the application has no pending work, an always-running worker fleet may consume resources without providing much value.
Google Cloud Run offers several ways to run containerized applications, including services and worker pools. With the right architecture, you can scale Temporal Workers according to workload demand instead of maintaining a large, permanently running fleet.
The important part is choosing the correct Cloud Run resource, understanding how worker scaling works, and ensuring that workflows continue safely when workers start, stop, or restart.
In this article, you will learn how Temporal Workers operate, which Cloud Run deployment options are available, how to configure scaling, and what to consider before using scale-to-zero in production.
Understanding Temporal Workers and Task Queues
Temporal separates workflow execution from the application code that performs individual tasks.
A Temporal Worker is a process that connects to a Temporal service, polls a Task Queue, and executes registered Workflows and Activities.
A Workflow defines the orchestration logic. An Activity performs an operation such as calling an external API, generating a document, or updating a database.
For example, consider an order-processing application:
- A customer places an order.
- A Temporal Workflow starts the order-processing process.
- An Activity validates the order.
- Another Activity processes the payment.
- A final Activity sends a confirmation.
The workflow coordinates these operations and records its execution history through Temporal.
Workers execute the application code, but they do not own the workflow's durable execution history. If a worker stops unexpectedly, another compatible worker can resume processing tasks for the workflow, subject to the application's deployment and versioning configuration.
This distinction makes it possible to scale worker compute independently of workflow orchestration.
Why Workers Do Not Behave Like Normal Web Services
A typical web service receives HTTP requests. Cloud Run can observe incoming requests and adjust the number of instances according to demand.
A Temporal Worker behaves differently. It usually maintains a connection to Temporal and continuously polls a Task Queue for work.
When the queue is empty, the worker may still be connected and waiting for tasks. There may be no incoming HTTP request for Cloud Run to observe.
This creates a scaling challenge: the absence of HTTP traffic does not necessarily mean that the worker process has stopped working.
A suitable deployment strategy must account for background processing, task backlog, worker startup time, and the time required to connect to Temporal.
Choosing the Right Cloud Run Deployment Model
Cloud Run provides different resource types for different application patterns. Selecting the correct one is important because their scaling behavior is different.
Deployment option | Typical use case | Scaling characteristics |
|---|
Cloud Run service | HTTP APIs and request-driven applications | Scales according to incoming requests and configured scaling behavior |
Cloud Run worker pool | Continuous background processing and pull-based workers | Supports manual scaling and external-metrics autoscaling |
Cloud Run job | Finite tasks that start, execute, and finish | Runs task executions to completion |
Dedicated compute or Kubernetes | Long-running workers requiring greater control | Scaling depends on the selected infrastructure and autoscaler |
A Cloud Run service can run a Temporal Worker, but it requires careful configuration because the worker does not naturally behave like a request-serving application.
Cloud Run worker pools are designed for continuous, non-HTTP, pull-based workloads. They are a natural fit for worker processes that do not need a public HTTP endpoint.
Worker pools do not automatically scale by default. Automatic scaling requires an appropriate external autoscaling configuration or a supported Temporal integration.
Cloud Run jobs are generally better suited to finite tasks that run to completion than to continuously polling Temporal Workers.
Running Temporal Workers Without Keeping Every Instance Active
There are two main approaches to reducing idle worker costs.
The first is to run workers on Cloud Run services with instance-based billing and a carefully configured minimum instance count. The second is to use Cloud Run worker pools with an appropriate scaling mechanism.
The choice depends on your current architecture, scaling requirements, and the features available in your environment.
Approach 1: Cloud Run Services With Instance-Based Billing
Cloud Run services can run background processes, but their default request-oriented behavior must be considered.
For a worker that needs CPU outside HTTP requests, instance-based billing ensures that CPU remains available throughout the instance's lifetime. A minimum instance count can keep a baseline number of workers running.
For example, you might deploy a worker service with one minimum instance and a maximum of five instances.
This arrangement can reduce idle capacity compared with maintaining a fixed fleet of five instances. However, the minimum instance continues to consume resources while it is running.
There is another important limitation: scaling from zero for a Cloud Run service requires an incoming request. A worker that is waiting for tasks cannot independently generate the HTTP request needed to wake a service that has already scaled to zero.
Therefore, a request-driven service should not be assumed to scale from zero just because tasks are waiting in a Temporal Task Queue.
A wake-up mechanism or another supported scaling integration is required if the design depends on scaling from zero.
Approach 2: Cloud Run Worker Pools
Cloud Run worker pools are designed for background workloads that do not need an HTTP endpoint.
A Temporal Worker can run as a container in a worker pool, connect to Temporal, and poll its assigned Task Queue.
Unlike a typical request-serving service, the worker pool does not require the application to listen on an HTTP port simply to keep the container running.
Worker pools support manual scaling and can be autoscaled using external metrics. Depending on the deployment and integration available, scaling can use signals related to Temporal Task Queue activity.
This architecture is useful when the goal is to adjust worker capacity according to actual processing demand.
Temporal has also introduced serverless worker capabilities for Cloud Run. The availability and configuration requirements depend on the specific Temporal deployment and supported integration.
Before choosing this approach, confirm whether your Temporal environment supports the relevant worker integration and whether it is generally available or subject to preview limitations.
Configuring a Temporal Worker for Cloud Run
A worker should be packaged as a container that starts the Temporal Worker process directly.
The application must connect to the correct Temporal endpoint, authenticate when required, register its Workflow and Activity implementations, and begin polling the appropriate Task Queue.
The following example illustrates a simplified .NET worker using the Temporal .NET SDK.
using Temporalio.Client;
using Temporalio.Worker;
public class Program
{
public static async Task Main()
{
var temporalAddress =
Environment.GetEnvironmentVariable("TEMPORAL_ADDRESS")
?? throw new InvalidOperationException(
"TEMPORAL_ADDRESS is required.");
var taskQueue =
Environment.GetEnvironmentVariable("TEMPORAL_TASK_QUEUE")
?? "order-processing";
var client = await TemporalClient.ConnectAsync(
new TemporalClientConnectOptions
{
TargetHost = temporalAddress
});
using var worker = new TemporalWorker(
client,
new TemporalWorkerOptions(taskQueue)
.AddWorkflow<OrderWorkflow>()
.AddAllActivities(new OrderActivities()));
await worker.ExecuteAsync();
}
}
This example assumes that OrderWorkflow and OrderActivities are implemented and registered appropriately for the installed Temporal .NET SDK version.
The connection configuration is intentionally minimal. A production deployment using Temporal Cloud or another secured Temporal endpoint must configure the appropriate TLS and authentication settings.
The worker should also have graceful shutdown handling so that container termination does not interrupt application cleanup unnecessarily.
Understanding the Worker Configuration
The TEMPORAL_ADDRESS environment variable allows the endpoint to change between development, staging, and production without modifying the application code.
The TEMPORAL_TASK_QUEUE variable identifies the queue that the worker polls.
The worker registers the Workflow and Activity implementations before beginning execution. If a worker polls a queue but does not register the required Workflow or Activity types, it may receive tasks that it cannot execute.
The ExecuteAsync call keeps the worker process running while it performs its work.
This is the key difference between a worker container and a finite Cloud Run job: the worker is designed to continue polling rather than exit after completing one task.
How to Scale Workers According to Task Queue Demand
The most useful scaling signal is often the amount of work waiting for available workers.
CPU utilization alone may not be sufficient. A worker can spend much of its time waiting for tasks while the queue grows because its concurrency settings or downstream dependencies limit throughput.
A more useful approach considers:
- Task Queue backlog.
- Schedule-to-Start latency, which measures how long a task waits before a worker starts it.
- Worker slot utilization.
- Activity execution duration.
- CPU and memory consumption.
- Downstream service capacity.
For example, suppose a Task Queue contains 30 pending tasks and each worker instance can handle approximately five tasks concurrently under the current workload.
A basic estimate would be:
Required instances = Ceiling(Pending tasks / Target tasks per instance)
Required instances = Ceiling(30 / 5)
Required instances = 6
This is an illustrative sizing calculation, not a universal Temporal scaling formula. Actual capacity depends on task duration, worker concurrency, resource usage, and the rate at which new tasks arrive.
An autoscaler can use an external metric to estimate the desired number of worker instances.
However, scaling should also consider how quickly new instances become ready. If workers take a significant amount of time to start, waiting until the queue becomes large can result in unacceptable processing delays.
Temporal's serverless worker integration can use task-queue signals to provision capacity in advance, reducing dependence on a purely reactive scaling approach.
Preventing Workflow Failures During Scale-Down
Reducing the worker count is not simply a matter of terminating idle containers.
A worker may be processing an Activity when Cloud Run decides to remove an instance. The application must be prepared to handle termination safely.
Handle Graceful Shutdown
When a container receives a termination signal, the worker should stop accepting new work where supported, allow in-flight work to finish within the available shutdown window, and release resources cleanly.
If an Activity cannot finish before termination, the system must handle the interruption without losing business data.
For example, an Activity that sends a payment request should not assume that a failed connection means the payment was not processed. The payment provider may have completed the operation before the worker lost its connection.
Use idempotency keys and appropriate retry policies for operations where duplicate execution could create financial or operational problems.
Design Activities to Be Retry-Safe
Temporal can retry Activities according to their configured retry policies.
If an Activity is interrupted, the operation may be attempted again. Therefore, Activities that interact with external systems should be designed to handle repeated execution safely.
For database updates, this might involve a unique operation identifier or a transaction that prevents duplicate records.
For external API calls, use the provider's idempotency mechanism when available.
Understand Scale-to-Zero Delays
When no workers are active, a newly scheduled task may need to wait for worker capacity to become available.
The time required to provision a container, initialize the application, connect to Temporal, and begin polling affects Schedule-to-Start latency.
Scale-to-zero is most suitable when occasional startup delays are acceptable.
For latency-sensitive workloads, keep a minimum amount of worker capacity available or use a supported scaling integration that provisions capacity ahead of demand.
Best Practices for Production Deployments
Set Minimum and Maximum Capacity Deliberately
Choose scaling limits based on workload characteristics rather than arbitrary values.
A low maximum can protect downstream systems from excessive concurrency, but it can also increase queue latency during traffic spikes.
A high maximum can increase cost and overwhelm databases or external APIs if every worker performs concurrent operations.
Test the limits with representative workloads before deployment.
Separate Task Queues by Workload
Consider using separate Task Queues for activities with different resource requirements.
For example, lightweight notification tasks should not necessarily share the same worker capacity as CPU-intensive document generation.
Separate queues make it easier to configure concurrency, measure performance, and scale each workload independently.
Monitor Backlog and Processing Latency
Track Task Queue backlog, Schedule-to-Start latency, Activity failures, retry counts, worker startup time, and instance utilization.
A queue that continues growing despite additional workers may indicate a downstream bottleneck or a task-processing problem rather than insufficient infrastructure alone.
Protect Credentials and Network Access
Use the appropriate service identity and authentication configuration for your Temporal deployment.
Store sensitive credentials in a supported secret-management system, restrict network access, and avoid exposing worker management endpoints publicly when they are not needed.
Test Deployment and Recovery Behavior
Test container termination, worker restarts, scaling from zero, temporary Temporal connectivity failures, and downstream API failures.
Verify that Workflows continue making progress and that Activities do not create duplicate business operations after retries.
These tests are important because scaling changes the timing and frequency of worker startup and shutdown.
Common Problems and Troubleshooting
Problem | Possible cause | Recommended action |
|---|
Workers never start | Incorrect container entry point or missing configuration | Verify environment variables, startup command, and logs |
Tasks remain queued | No workers are polling the queue or workers cannot execute the task types | Check worker connectivity and Workflow or Activity registration |
Service does not scale from zero | No incoming request triggers startup | Use a supported scaling integration or wake-up mechanism |
Queue latency increases during traffic spikes | Workers cannot scale quickly enough | Review scaling signals, startup time, and maximum capacity |
Costs remain high when idle | Minimum instances or worker capacity remain active | Review minimum capacity and scaling configuration |
Activities run more than once | Retry behavior or interrupted execution | Make external operations idempotent |
Scaling up does not improve throughput | Downstream services or concurrency settings limit processing | Inspect worker slots, task duration, and downstream capacity |
When troubleshooting, examine Temporal's task-queue metrics alongside Cloud Run instance metrics. Looking only at CPU utilization can hide a queue bottleneck.
Advantages and Disadvantages
Scaling Temporal Workers on Cloud Run can reduce the operational work involved in maintaining a dedicated worker fleet. It allows applications to use managed container infrastructure and adjust capacity as workload demand changes.
Cloud Run worker pools are particularly useful for background processes that do not need an HTTP endpoint.
However, scale-to-zero introduces startup delays, and autoscaling requires an appropriate scaling signal or supported integration. Worker shutdown must also be handled carefully to protect in-flight operations.
For workloads that require predictable low latency, a minimum worker capacity or a different deployment model may be more appropriate.
Summary
Temporal Workers can run on Google Cloud Run without maintaining a large, permanently active fleet. The correct design depends on whether you use Cloud Run services, worker pools, or a supported Temporal serverless worker integration.
Cloud Run services require special care because request-driven scaling does not automatically detect a Temporal Task Queue backlog. Worker pools are designed for background processing and can be autoscaled using external metrics.
For production deployments, measure queue backlog and processing latency, configure sensible capacity limits, make Activities retry-safe, and test graceful shutdown and recovery.
The goal is not simply to reduce the number of running containers. It is to match worker capacity to workload demand while keeping workflow execution reliable and processing delays within acceptable limits.
Join the conversation! Your thoughts help the community grow.