The Kubernetes autoscaling landscape is one where most decisions are made by convention rather than derivation. Teams use HPA because it ships with Kubernetes. They scale on CPU because that is what the tutorials show. They discover KEDA when they need a Kafka consumer to scale on queue depth. They encounter VPA in the docs and add it experimentally. The result, in most production environments, is an autoscaling configuration that was assembled from whatever felt appropriate at the time rather than derived from the workload's actual scaling requirements.
This matters operationally. An incorrect autoscaling strategy does not fail immediately and visibly — it fails slowly and ambiguously, through slightly elevated latency during load spikes, through replica counts that oscillate rather than stabilise, through JVM services that scale to the correct number of replicas but still degrade because each replica is under-resourced. The failure modes of incorrect autoscaling strategy selection are almost always attributed to the application rather than to the scaling configuration — which means they persist.
This post derives the selection framework from first principles. The decision criteria are expressed as measurable thresholds, not as preferences or rules of thumb. The goal is not to produce a chart that tells you which tool to use, but to produce a reasoning process that tells you why.
The Three Autoscaling Dimensions
Before comparing the tools, the problem space must be defined. Autoscaling in Kubernetes addresses three independent dimensions of the capacity problem. Each dimension has a different control variable, a different control loop timescale, and a different set of appropriate tools.
────────────────────────────────────────────────────────────────────────────
THE THREE AUTOSCALING DIMENSIONS
DIMENSION 1: HORIZONTAL (Scale Out/In)
Control variable: replica count
Control loop timescale: seconds to minutes
SLO impact: affects capacity ceiling (SOT × replica_count)
Primary tool: HPA (built-in), KEDA (extended triggers)
Appropriate when: workload is stateless or horizontally shardable;
request rate varies with load patterns
DIMENSION 2: VERTICAL (Scale Up/Down)
Control variable: CPU and memory resource requests/limits per pod
Control loop timescale: minutes to hours (requires pod restart)
SLO impact: affects per-replica performance (SOT per replica)
Primary tool: VPA (Vertical Pod Autoscaler)
Appropriate when: workload has variable resource consumption;
right-sizing initial resource requests is difficult;
workload is NOT horizontally scalable
DIMENSION 3: EXTERNAL EVENT-DRIVEN (Scale to Zero / Scale on Queue)
Control variable: replica count, driven by external event sources
Control loop timescale: seconds (event-driven)
SLO impact: affects scale-to-zero and burst-from-zero latency
Primary tool: KEDA
Appropriate when: workload has predictable burst patterns (scheduled);
workload scales on upstream queue depth;
non-production workloads should scale to zero
────────────────────────────────────────────────────────────────────────────
KEY POINT: The three dimensions are NOT alternatives — they address
different aspects of the capacity problem and can be combined.
The selection question is not "HPA OR KEDA OR VPA" but:
"Which combination of tools addresses each dimension of this workload?"
The common error: treating HPA, KEDA, and VPA as interchangeable
alternatives and picking one, when the workload may need all three.
────────────────────────────────────────────────────────────────────────────
Decision Dimension 1: Horizontal Scaling — HPA vs KEDA
Both HPA and KEDA control replica count horizontally. The selection criterion is not which tool is better — it is which metric source and trigger model is appropriate for the workload's demand signal.
────────────────────────────────────────────────────────────────────────────
HPA vs KEDA SELECTION CRITERIA
USE HPA WHEN:
Trigger 1: Request-rate-based scaling is sufficient
Condition: workload receives HTTP/gRPC requests with a
stable relationship between request rate and resource usage
Metric: http_requests_per_second per replica (from Istio Envoy)
Target: SOT value derived from load testing
HPA limitation: only reads Kubernetes metrics API; limited to
metrics scraped into the cluster
Trigger 2: Single-dimension scaling signal
Condition: scaling decision can be made from one metric alone
HPA handles this cleanly with a single target metric
Trigger 3: Stateless services with predictable traffic
Condition: no off-hours scale-to-zero requirement;
no external queue depth dependency;
no scheduled burst preparation
USE KEDA WHEN:
Trigger 1: Multi-dimensional scaling required
Condition: scaling decision depends on MULTIPLE signals simultaneously
Example: scale on max(rps_per_replica, queue_depth_per_replica)
HPA cannot express this; KEDA ScaledObject handles multiple triggers
with OR logic (scale when ANY trigger exceeds threshold)
Trigger 2: External event source (queue, stream, database)
Condition: workload is a consumer of Kafka, RabbitMQ, SQS, Redis,
or any other external event source
KEDA has native scalers for 60+ external sources; HPA does not
Trigger 3: Scale-to-zero required
Condition: non-production workloads; batch workloads; scheduled jobs
KEDA supports minReplicaCount: 0; HPA minimum is 1
Scale-to-zero requires KEDA; this is a hard requirement
Trigger 4: Scheduled burst preparation
Condition: known traffic spikes at predictable times
(market open, batch processing windows, shift changes)
KEDA cron trigger provides pre-scheduled scaling;
HPA responds reactively and cannot pre-warm
────────────────────────────────────────────────────────────────────────────
QUANTITATIVE DECISION CRITERIA:
Question 1: Does the workload need to scale to zero?
YES → KEDA (mandatory; HPA cannot do this)
Question 2: Does the workload consume from an external queue/stream?
YES → KEDA (direct external scaler; cleaner than HPA custom metrics)
Question 3: Does the workload have known burst windows requiring pre-warm?
YES → KEDA cron trigger (or add to existing KEDA ScaledObject)
Question 4: Is the workload HTTP/gRPC with a stable RPS→resource relationship?
YES, simple → HPA with custom metric (Istio RPS)
YES, complex → KEDA with Prometheus trigger (more flexible)
Question 5: Is the scaling signal available as a Prometheus metric?
YES → Either HPA (via custom metrics adapter) or KEDA (native Prometheus)
NO → KEDA (broader external source support)
────────────────────────────────────────────────────────────────────────────
# Decision Rule Applied: Payments API
# - HTTP workload: YES → consider HPA
# - Scale to zero: NO (production)
# - External queue: YES (Kafka payment requests) → KEDA required
# - Known burst window: YES (09:20 pre-market open) → KEDA cron
# Result: KEDA ScaledObject with three triggers (not HPA)
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: payments-api-scaler
namespace: production
annotations:
sre.internal/scaling-strategy: "keda-multidimensional"
sre.internal/sot-value: "3040"
sre.internal/selection-rationale: >
KEDA selected over HPA: (1) Kafka queue depth trigger required;
(2) scheduled pre-warm for market open required;
(3) multi-dimensional scaling (RPS + queue depth) required.
HPA cannot satisfy trigger 1 or 2.
spec:
scaleTargetRef:
name: payments-api
minReplicaCount: 5
maxReplicaCount: 80
cooldownPeriod: 60
triggers:
- type: prometheus
metadata:
serverAddress: http://prometheus.monitoring.svc:9090
metricName: rps_per_replica
query: |
sum(rate(istio_requests_total{
destination_service_name="payments-api",
reporter="destination"
}[2m]))
/ count(kube_pod_info{namespace="production",pod=~"payments-api-.*"})
threshold: "3040" # SOT-derived target
- type: prometheus
metadata:
serverAddress: http://prometheus.monitoring.svc:9090
metricName: kafka_queue_depth
query: |
sum(kafka_consumer_group_lag{
topic="payment-requests",
group="payments-api"
})
threshold: "500"
- type: cron
metadata:
timezone: "America/New_York"
start: "20 9 * * 1-5"
end: "0 10 * * 1-5"
desiredReplicas: "25"
Decision Dimension 2: Should VPA Be Added?
VPA addresses the vertical dimension — the right-sizing of CPU and memory resource requests per replica. The selection question is whether the workload has a vertical right-sizing problem that HPA/KEDA cannot address.
────────────────────────────────────────────────────────────────────────────
VPA SELECTION CRITERIA
USE VPA WHEN (in Recommendation mode, not Auto mode):
Criterion 1: JVM workload with difficult-to-estimate heap requirements
Signal: pods OOMKilling periodically despite "sufficient" memory limits
OR: pods have very high memory request headroom (allocated >> used)
VPA recommendation mode tells you what resource requests should be
based on actual usage — removes the guesswork from JVM right-sizing
Criterion 2: Batch workloads with variable resource consumption
Signal: batch job CPU/memory usage varies significantly between runs
VPA in recommendation mode surfaces the right-sizing range
over time as the workload profile is observed
Criterion 3: Initial deployment right-sizing
Signal: new service with no historical usage data
VPA recommendation mode in the first 2-4 weeks provides the
data-driven request/limit values for the HPA/KEDA configuration
DO NOT USE VPA Auto mode with HPA/KEDA:
CRITICAL CONFLICT: VPA Auto and HPA/KEDA targeting the same metric
VPA Auto changes resource requests → triggers pod restarts
Pod restarts temporarily reduce replica availability
HPA/KEDA sees capacity drop → scales OUT to compensate
VPA sees new pods with new resource profile → adjusts again
Result: oscillation between VPA vertical adjustments and
HPA/KEDA horizontal adjustments
SAFE COMBINATION:
VPA in Recommendation mode: produces recommendations only, no changes
SRE reviews recommendations quarterly
Manual resource request updates applied as planned changes
HPA/KEDA manages horizontal scaling continuously
VPA Auto mode is ONLY safe when:
No HPA or KEDA is configured for the same deployment
The workload is not latency-sensitive (VPA Auto causes pod restarts)
The workload is a batch job or non-production service
────────────────────────────────────────────────────────────────────────────
VPA RECOMMENDATION MODE CONFIGURATION (safe with HPA/KEDA):
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: payments-api-vpa
namespace: production
annotations:
sre.internal/mode: "recommendation-only"
sre.internal/review-cadence: "quarterly"
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: payments-api
updatePolicy:
updateMode: "Off" # CRITICAL: Off = recommendation only, no automatic changes
resourcePolicy:
containerPolicies:
- containerName: payments-api
minAllowed:
cpu: "500m"
memory: "1Gi"
maxAllowed:
cpu: "4"
memory: "8Gi"
controlledResources: ["cpu", "memory"]
────────────────────────────────────────────────────────────────────────────
The Autoscaling Strategy Decision Framework
────────────────────────────────────────────────────────────────────────────
AUTOSCALING STRATEGY SELECTION FRAMEWORK
(Work through each question in order)
────────────────────────────────────────────────────────────────────────────
STEP 1: HORIZONTAL SCALING TRIGGER SELECTION
Q: Does the workload need to scale to zero in any environment?
YES → KEDA required (proceed to Step 2 with KEDA)
NO → continue
Q: Does the workload consume from an external event source?
YES → KEDA required
NO → continue
Q: Does the workload have predictable burst windows needing pre-warm?
YES → KEDA cron trigger (can combine with other triggers)
NO → continue
Q: Is a single Prometheus metric sufficient as the scaling signal?
YES, and it's RPS-based → HPA with custom metric adapter
(simpler configuration, standard Kubernetes API)
YES, but multiple signals needed → KEDA with Prometheus triggers
NO → KEDA with appropriate native scaler
STEP 2: SCALING METRIC SELECTION
Q: Is the workload request-rate-sensitive (web, API, gRPC)?
YES → Use RPS per replica (from Istio Envoy telemetry)
Target = SOT value (derived from load testing)
NEVER use CPU% for this workload type
Q: Is the workload queue-depth-sensitive (consumer, worker)?
YES → Use queue lag per replica
Target = max acceptable lag before processing falls behind
Q: Is the workload CPU-bound with predictable CPU-per-request ratio?
YES → CPU% MAY be acceptable
Verify: CPU utilisation at SOT threshold from load test
Verify: no JVM GC pause CPU suppression effect
If either fails → switch to RPS-based
STEP 3: VERTICAL SCALING DECISION
Q: Is the workload a JVM application?
YES → Deploy VPA in Recommendation mode
Review VPA recommendations before initial production sizing
Update resource requests based on VPA data quarterly
Q: Is the service new with no historical usage data?
YES → Deploy VPA in Recommendation mode for first 4 weeks
Set initial requests conservatively high
Update to VPA-recommended values after observation period
Q: Is VPA Auto mode being considered for production?
→ If HPA or KEDA is also configured: NO — use Recommendation mode
→ If no horizontal autoscaling: VPA Auto acceptable for batch only
STEP 4: SCALE BEHAVIOUR TUNING
Q: What is the cold-start latency for new replicas?
JVM (typical): 30–90 seconds
Container without JVM: 5–15 seconds
scaleUp.stabilizationWindowSeconds should be ≥ cold-start latency
Q: What is the SOT safety margin target?
Standard: 80% of empirical SLO threshold
High consequence: 70% (more conservative headroom)
Scale trigger = SOT × safety_margin
────────────────────────────────────────────────────────────────────────────
STRATEGY PATTERNS BY WORKLOAD TYPE:
Stateless HTTP API (JVM):
→ KEDA (multi-trigger) + VPA Recommendation
→ Triggers: RPS (Prometheus) + cron (if burst windows exist)
→ Target: SOT value from load test
Stateless HTTP API (non-JVM):
→ HPA (if single trigger sufficient) or KEDA (if multi-trigger needed)
→ VPA Recommendation optional (less critical for non-JVM)
Queue Consumer:
→ KEDA with queue depth trigger + optional cron for pre-warm
→ Scale to zero in non-production
Batch Processing Job:
→ KEDA with cron trigger + scale to zero between runs
→ VPA Auto acceptable (no HPA/KEDA horizontal conflict)
Stateful Workload (database, cache):
→ NOT recommended for HPA/KEDA horizontal scaling
→ VPA Recommendation mode for right-sizing
→ Manual scaling with database-specific considerations
────────────────────────────────────────────────────────────────────────────
Scaling Behaviour Configuration: The Parameters That Determine SLO Impact
The scaling strategy selection determines which tool manages replica count. The scaling behaviour configuration determines how aggressively and how smoothly the tool responds to demand changes. Both matter for SLO compliance.
# Scaling Behaviour: Optimised for SLO Protection
# Fast scale-up to prevent SLO breach; slow scale-down to prevent oscillation
# For KEDA ScaledObject (applied via ScaledObject.spec.advanced):
advanced:
horizontalPodAutoscalerConfig:
behavior:
scaleUp:
stabilizationWindowSeconds: 30 # Fast: respond to load in 30s
selectPolicy: Max # Use the largest scale-up recommendation
policies:
- type: Percent
value: 100 # Can double replica count per period
periodSeconds: 15 # Aggressive: every 15 seconds
- type: Pods
value: 5 # Or add 5 pods per period
periodSeconds: 15
scaleDown:
stabilizationWindowSeconds: 300 # Slow: 5 minutes before scaling down
selectPolicy: Min # Use the smallest scale-down recommendation
policies:
- type: Percent
value: 10 # Remove at most 10% of replicas per period
periodSeconds: 60 # Every 60 seconds
# Scale-up calibration rationale:
# Cold-start latency for JVM: ~45 seconds
# stabilizationWindowSeconds: 30s means scale-up fires before cold-start completes
# This is intentional: we want replicas initialising before we need them,
# not after we need them. The 30s window ensures overlap with JVM warm-up.
# Scale-down calibration rationale:
# 300s stabilisation: prevents scale-down during brief load valleys
# Traffic that drops for 2 minutes and recovers should NOT trigger scale-down
# Financial services: additional consideration for end-of-day settlement
# windows — scale-down should not fire during known high-activity periods
Istio STRICT mTLS: Scaling Metrics from the Right Source
# Scaling metric source matters in Istio STRICT mTLS environments
# reporter="destination" captures the full request rate including
# requests rejected at the mTLS layer before reaching the application
# CORRECT: Envoy proxy metric (reporter="destination")
- type: prometheus
metadata:
query: |
sum(rate(istio_requests_total{
destination_service_name="{{ service_name }}",
reporter="destination"
}[2m]))
/ count(kube_pod_info{
namespace="{{ namespace }}",
pod=~"{{ service_name }}-.*"
})
threshold: "{{ sot_value }}"
# WHY NOT reporter="source":
# Source metrics miss mTLS handshake failures
# A certificate rotation event fails connections at the destination sidecar
# These failures do NOT appear in source metrics
# → Scaling trigger misses a significant load signal during policy events
# WHY NOT application-level metrics:
# Application only sees requests that passed the sidecar
# Same gap as source metrics for mTLS-layer failures
# Additional gap: requests rejected by circuit breaker at sidecar
# → Scaling trigger systematically undercounts effective request rate
Common Antipatterns
The CPU Default antipattern → Configuring HPA with
type: Resource, resource: cpubecause it is the Kubernetes documentation default. CPU is a lagging indicator for web/API workloads, a misleading indicator for JVM workloads during GC pauses, and irrelevant for queue consumers. The only workloads where CPU% is the correct scaling metric are CPU-bound batch jobs where CPU utilisation has a stable, predictable relationship with throughput — which is a small minority of production services.The VPA Auto + HPA Conflict antipattern → Running VPA in Auto mode on the same deployment as HPA or KEDA. VPA Auto triggers pod restarts to apply new resource requests. Each restart removes a replica from the healthy pool, which HPA reads as reduced capacity and responds to with horizontal scale-out. The result is a control loop oscillation between VPA's vertical adjustments and HPA's horizontal compensations. Use VPA in Recommendation mode exclusively when HPA or KEDA is active.
The ScaleDown Too Fast antipattern → Setting
scaleDown.stabilizationWindowSecondsto 0 or to a very short value in the belief that fast scale-down reduces costs. Fast scale-down causes two SLO problems: replicas are removed during brief load valleys and then must be re-added when load recovers, introducing cold-start latency during the re-scaling period; and the oscillation between scale-down and scale-up events creates an elevated error rate from connections to terminating pods. Keep scale-down stabilisation at 5 minutes minimum.The Single-Trigger KEDA antipattern → Using KEDA for a workload that only needs one scaling trigger, when HPA would be simpler. KEDA's operational overhead (additional controller, ScaledObject CRD, external scaler connections) is justified by its extended trigger capabilities. For a simple request-rate-based scaler on a production service with no scale-to-zero requirement and no external queue dependency, HPA with a custom metric adapter is simpler and has fewer failure modes.
The SOT-Free Configuration antipattern → Setting HPA or KEDA targets without deriving them from SOT load testing. A target that is set "by feel" or copied from a similar service will be wrong in one of two directions: too aggressive (service degrades before scaling completes) or too conservative (service scales out unnecessarily, wasting capacity). SOT derivation from load testing is the mechanism that produces a principled target.
Maturity Progression
────────────────────────────────────────────────────────────────────────────
STAGE AUTOSCALING MATURITY NORTH STAR SIGNAL
────────────────────────────────────────────────────────────────────────────
Reactive CPU-based HPA everywhere. Capacity incidents after
Manual scaling for known the fact. Replicas scale
events. VPA unused. too late. JVM services
OOMKill under load.
Defined Selection framework documented. First SOT-derived
SOT derived for critical target deployed. KEDA
services. KEDA adopted for adopted for queue
queue consumers. consumers. VPA
Recommendation active.
Measured All production HPA/KEDA No SOT boundary events
targets SOT-derived. VPA in last 30 days.
recommendations reviewed Scale-up fires before
quarterly. CPU-based HPA SLO breach. Oscillation
eliminated from production. eliminated.
Optimised KEDA multi-trigger for all Pre-warm triggers
services with known burst active for all known
patterns. VPA right-sizing burst windows. Cold-
cycle automated. Scaling start latency measured
metrics from Istio Envoy. and accounted for.
Generative Autoscaling strategy part New services cannot
of service design review. launch without
SOT requirement blocks autoscaling strategy
deployment without load review and SOT
test baseline. derivation complete.
────────────────────────────────────────────────────────────────────────────
Five Action Items for This Week
Audit every HPA in your production cluster and identify which are using CPU% as the scaling metric. For each CPU-based HPA, assess whether the workload is actually CPU-bound or whether it is a web/API/queue service where CPU is a misleading signal. This audit produces your list of HPA configurations to replace with request-rate-based targets.
Identify every production service that needs to respond to an external event source (Kafka, RabbitMQ, database queue) and is currently using HPA. These are mandatory KEDA migration candidates. HPA cannot read external queue depth; KEDA can. The queue depth trigger is the correct scaling signal for consumer workloads.
Deploy VPA in Recommendation mode for your top three JVM services and observe the recommendations for one week. Do the VPA-recommended resource requests differ significantly from your current configuration? A large difference indicates that your current right-sizing is incorrect — either wastefully high (costing money) or insufficiently generous (causing OOMKills under load).
Verify that your HPA/KEDA targets are sourced from Istio Envoy metrics (reporter="destination") rather than application-level metrics. Query both sources for a production service and compare. Any difference is load that your current scaling trigger is missing — typically mTLS-layer failures and circuit-breaker rejections that the application never sees but that reduce effective capacity.
Set scaleDown.stabilizationWindowSeconds to 300 (5 minutes) on every HPA and KEDA ScaledObject that currently has a shorter window or no stabilisation configured. This single change eliminates most autoscaling oscillation in production. The cost is slightly slower capacity reclamation; the benefit is elimination of the latency spikes caused by replica removal during brief load valleys.
"The autoscaling question is not which tool to use. It is what the scaling signal should be, how quickly the system should respond to it, and what the correct capacity target is relative to the SLO. HPA, KEDA, and VPA are each optimal for a specific subset of these questions. The practitioner who understands the questions can select the tools. The practitioner who only knows the tools will select them by convention — and convention in autoscaling defaults to CPU-based HPA, which is wrong for most production web services."
