Kubernetes HPA vs KEDA: taming spiky mobile backend traffic
Meta description: Why CPU-based HPA fails during push notification campaigns, and how KEDA’s event-driven autoscaling cuts the cold-start penalty that wrecks p99 latency.
Tags: kubernetes backend mobile microservices devops
TL;DR
Horizontal Pod Autoscaler reacts to what already happened. KEDA reacts to what is happening. For mobile backends where traffic spikes from zero to thousands of RPS in seconds, that distinction is the difference between a smooth App Store featuring event and a p99 latency catastrophe. What follows covers how to configure KEDA with the right stabilization windows, scale-down policies, and replica floors to stay ahead of your users.
The core problem with HPA for mobile traffic
In my experience building production systems that serve mobile clients, the traffic profile is fundamentally unlike web or API traffic. Push notification campaigns, App Store featuring events, and viral social moments share one characteristic: they are step functions, not ramps.
HPA’s default signal chain looks like this:
Traffic spikes → CPU/memory climbs → HPA observes metric →
Scaling decision made → Pod scheduled → Container starts → ReadyThat chain takes anywhere from 60 to 180 seconds in a typical cluster. For a push notification campaign delivering 2 million notifications in under 30 seconds, your backend has already absorbed the full burst before the first new pod is ready to serve traffic.
CPU-based HPA operates on a 15-second scrape interval by default, with a 30-second evaluation window. By the time the autoscaler acts, your existing pods are already saturated.
KEDA’s event-driven approach
KEDA (Kubernetes Event-Driven Autoscaling) inverts this model. Instead of reacting to resource exhaustion, it scales on the leading indicators of load — queue depth, message lag, incoming RPS — before your pods are under stress.
Scaling on queue depth
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: mobile-notification-processor
spec:
scaleTargetRef:
name: notification-service
minReplicaCount: 3
maxReplicaCount: 50
triggers:
- type: rabbitmq
metadata:
queueName: push-notifications
value: "100" # target messages per replica
advanced:
stabilizationWindowSeconds: 30
horizontalPodAutoscalerConfig:
behavior:
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Pods
value: 10
periodSeconds: 15
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 10
periodSeconds: 60The asymmetry is intentional.
The three parameters that prevent p99 degradation
| Parameter | HPA Default | KEDA Recommendation | Why It Matters |
|---|---|---|---|
scaleUp.stabilizationWindowSeconds | 0 (but CPU lag adds ~60s) | 0 explicit | React immediately to queue growth |
scaleDown.stabilizationWindowSeconds | 300s | 300–600s | Prevent oscillation after burst |
minReplicaCount | 0 or 1 | 3+ | Eliminate cold-start entirely for warm floor |
scaleUp policy (Pods) | 4 pods/60s | 10 pods/15s | Match burst velocity of push campaigns |
scaleDown policy (Percent) | — | 10%/60s | Gradual drain prevents rebound spikes |
The replica floor is non-negotiable
Most teams treat minReplicaCount: 0 as a cost optimization, then wonder why their p99 spikes from 120ms to 4 seconds during the first minute of a campaign. That’s the cold-start penalty, and it’s entirely self-inflicted.
A replica floor of 3–5 pods, sized to handle your baseline traffic comfortably, eliminates it for the initial burst wave. The incremental cost is trivial compared to the user-facing latency impact.
Stabilization windows: asymmetric by design
Scale-up stabilization should be zero or near-zero. The cost of one unnecessary pod for 60 seconds is negligible. The cost of under-provisioning during a featuring event is a 1-star review wave.
Scale-down stabilization should be aggressive — 5 to 10 minutes minimum. Mobile traffic bursts are rarely clean step-down functions. App Store featuring events drive long tails of organic discovery traffic for hours after the initial spike.
How it fits together
For a typical mobile push campaign backend, KEDA watches SQS queue depth or Kafka consumer lag and makes a scaling decision within 5 seconds of queue growth. The 3 warm replicas on the floor absorb the initial burst with no latency penalty. The scale-up policy (10 pods per 15 seconds) matches the backend load curve of a 2M notification campaign. Scale-down is deliberately slow — 10% per 60 seconds — to prevent the oscillation that causes secondary latency spikes after the burst.
What to change
Replace CPU-based HPA with KEDA for any mobile backend that receives push-triggered traffic. Queue depth and consumer lag are leading indicators; CPU is a lagging one. The 60–90 second lag in CPU-based scaling is unacceptable for step-function mobile traffic.
Set minReplicaCount to at least 3 and size those pods to handle your P50 load. The cold-start penalty isn’t a Kubernetes problem — it’s a configuration choice. A warm replica floor eliminates it for the burst’s first wave.
Use asymmetric stabilization windows: zero on scale-up, 300+ seconds on scale-down. The oscillation pattern that wrecks p99 latency in the minutes after a burst comes from aggressive scale-down followed by a rebound. Conservative scale-down is the fix, and the cost is a few extra pods for a few extra minutes.