Keeping one or more worker Pods alive is often the simplest way to process a queue. It is also wasteful when the queue is empty. A video worker, GPU job, or batch processor can spend most of its time doing nothing while still reserving compute, memory, and sometimes an expensive accelerator.
Kubernetes 1.37 makes a more useful version of this trade-off available: Horizontal Pod Autoscaler (HPA) scale-to-zero is now Beta and enabled by default for object and external metrics. A workload can fall to zero replicas while idle, then grow again when a queue or other external signal shows work.
That does not turn every Deployment into a serverless function. The feature has a strict metric boundary and real cold-start costs. Used with the right workload, however, it can remove idle capacity without requiring a separate autoscaling controller.
Why CPU cannot wake a zero-Pod workload
An HPA is a control loop. It periodically reads metrics, calculates a desired replica count, and updates the scale subresource of a Deployment, StatefulSet, or another scalable object. Kubernetes documents a default HPA sync period of 15 seconds, although an operator can configure it differently.
CPU and memory utilization are measured from running Pods. When a workload has zero Pods, there is no per-Pod CPU or memory sample from which to decide that it should start. That is why scale-to-zero is limited to object and external metrics: the signal must remain meaningful when the target has no Pods.
Examples of suitable signals include:
- Messages ready in a queue.
- Pending jobs in a work scheduler.
- A count of unprocessed objects reported by an external system.
- A custom object metric representing work assigned to a service.
Kubernetes does not create these metrics for you. You need a metrics adapter that exposes the queue or application signal through an external or object metrics API. The adapter's naming, labels, authentication, and freshness rules are part of your production design.
A representative HPA
The following is an illustrative shape for a queue consumer. It was not applied to a cluster in this article. The metric name and label are adapter-specific, so replace them with the contract provided by your metrics system.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: video-worker
namespace: jobs
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: video-worker
minReplicas: 0
maxReplicas: 20
metrics:
- type: External
external:
metric:
name: queue_messages_ready
selector:
matchLabels:
queue: video
target:
type: Value
value: "10"
The intent is that the external metric reports how much work is waiting and the adapter translates that signal into an HPA decision. maxReplicas remains a safety limit. minReplicas: 0 permits the lower bound, but it does not guarantee that the workload will immediately reach zero or immediately return to full capacity.
Treat this manifest as a starting point, not a universal queue policy. A target of 10 messages might be reasonable for one worker and completely wrong for another. If one job takes five minutes, or if jobs have very different weights, a simple message count may not represent capacity. A weighted backlog, age of oldest message, or another application metric may be a better signal.
What happens when work arrives
The wake-up path has several independent steps:
- The queue or external system records work.
- The metrics adapter exposes a fresh value.
- The HPA controller observes the value during a sync cycle.
- The HPA updates the Deployment's desired replica count.
- The scheduler places new Pods.
- Images are pulled, containers start, and readiness checks pass.
- Workers connect to the queue and claim work.
The documented sync period only describes one part of that path. Image-pull time, node capacity, startup work, readiness probes, and queue-client behavior can dominate the actual delay. Do not promise users a 15-second wake-up time based on the HPA default.
The worker must also tolerate duplicate delivery or a retry after termination. Scale-down can stop a Pod between jobs, during graceful shutdown, or while a queue client is renewing a lease. Use the queue's acknowledgement and visibility mechanisms correctly, make handlers idempotent where possible, and give the worker enough termination grace time to finish or safely release work.
Design the metric before enabling zero
The most important question is not “How do I set minReplicas to zero?” It is “What signal remains trustworthy when there are no workers?”
Check these properties:
The metric has a zero-Pod data path
The adapter must be able to query the queue or external system without relying on a worker Pod to publish the value. If the metric disappears when the last worker exits, HPA cannot reliably decide to scale back up.
The metric is bounded and fresh
Stale backlog data can cause late wake-ups. A broken adapter can also report an apparently empty queue and scale down while work is waiting. Alert on missing, stale, or implausible values instead of treating every error as zero.
The target reflects capacity
Message count is only a proxy. Consider job duration, concurrency, item size, and the maximum safe load per worker. Start with a conservative target and compare the resulting queue age with your service objective.
There is enough node capacity
Scaling from zero still requires somewhere to place the new Pods. Cluster autoscaling, GPU capacity, quotas, taints, and image availability can all add delay or prevent a scale-up. Test the path under realistic capacity constraints.
Observe the state HPA records
When HPA itself scales a workload from one replica to zero, Kubernetes 1.37 records a ScaledToZero condition in HPA status. That status helps distinguish an autoscaled idle state from a workload that an operator manually forced to zero.
During rollout, inspect both the HPA and its target:
kubectl -n jobs describe hpa video-worker
kubectl -n jobs get deployment video-worker
kubectl -n jobs get events --sort-by=.lastTimestamp
These commands are examples and were not run for this article. In a real incident, also inspect the external metric adapter, the queue's backlog and delivery state, Pod scheduling events, image-pull failures, and readiness failures. A healthy HPA object does not prove that the worker can start.
Pay attention to the difference between:
- The metric being unavailable.
- The metric being available and reporting no work.
- The HPA deciding that zero is appropriate.
- The Deployment having zero desired replicas.
- The new Pod being scheduled but not Ready.
Those states require different fixes. Instrument queue age, scale decisions, Pod startup duration, and time from scale-up to first successful job. Without those measurements, a cost optimization can quietly become a latency regression.
A cautious adoption sequence
- Start above zero. Validate the queue metric, worker shutdown, and retry behavior with
minReplicas: 1. - Test scale-down. Confirm that an idle worker exits cleanly and that the queue does not retain an abandoned lease longer than intended.
- Test wake-up. Create work while the Deployment is idle and measure the complete time to a successful job.
- Exercise failures. Make the adapter unavailable, delay image pulls, and constrain node capacity in a non-production environment. Confirm that alerts distinguish these cases from an empty queue.
- Enable zero for a suitable workload. Use a small
maxReplicas, observe queue age and startup latency, and increase limits only after the behavior is understood. - Keep a rollback. Raising
minReplicasto 1 can restore a warm worker, but document who owns that change and how it is applied.
Kubernetes' HPA documentation also describes stabilization and readiness behavior. Those controls can smooth rapid changes, but they cannot remove cold-start time or repair a metric that reports the wrong backlog. Tune them after the basic wake-up path is correct.
When scale-to-zero is a poor fit
Do not use this feature as a blanket replacement for a warm service. It is a poor fit when:
- Requests require consistently low latency and cannot tolerate startup delay.
- Work cannot be safely retried after a worker is terminated.
- The only useful signal is CPU or memory utilization.
- The external metric is not available during a zero-Pod state.
- The queue or scheduler cannot handle a burst while new workers start.
- A single worker is inexpensive compared with the operational complexity.
The Kubernetes enhancement design explicitly treats request buffering as out of scope. HPA changes replica counts; it does not provide a queue, hold incoming HTTP requests, or guarantee that capacity appears before work arrives. For event-driven systems with strict activation semantics, a purpose-built event scaler may be a better match.
The practical takeaway
Kubernetes 1.37 HPA scale-to-zero is most compelling for interruptible, queue-backed work with expensive or frequently idle workers. The safe pattern is not simply minReplicas: 0; it is a complete design in which an external signal survives zero Pods, workers shut down safely, new Pods can start within the workload's latency budget, and operators can tell an empty queue from a broken metrics path.
Start with one non-critical worker, measure the full wake-up path, and keep a warm replica when the product cannot afford the cold start. Zero is a useful lower bound—not a promise that infrastructure is free or instant.