그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그
2 minute read
Auto-Scaling in Shoot Clusters
There are two parts that relate to auto-scaling in Kubernetes clusters in general:
- Horizontal node auto-scaling, i.e., dynamically adding and removing worker nodes.
- Vertical pod auto-scaling, i.e., dynamically raising or shrinking the resource requests/limits of pods.
This document provides an overview of both scenarios.
Horizontal Node Auto-Scaling
Every shoot cluster that has at least one worker pool with minimum < maximum
nodes configuration will get a cluster-autoscaler
deployment.
Gardener is leveraging the upstream community Kubernetes cluster-autoscaler
component.
We have forked it to gardener/autoscaler so that it supports the way how Gardener manages the worker nodes (leveraging gardener/machine-controller-manager).
However, we have not touched the logic how it performs auto-scaling decisions.
Consequently, please refer to the official documentation for this component.
The Shoot
API allows to configure a few flags of the cluster-autoscaler
:
There are general options for cluster-autoscaler
, and these values will be used for all worker groups except for those overwriting them. Additionally, there are some cluster-autoscaler
flags to be set per worker pool. They override any general value such as those specified in the general flags above.
Only some
cluster-autoscaler
flags can be configured per worker pool, and is limited by NodeGroupAutoscalingOptions of the upstream community Kubernetes repository. This list can be found here.
Vertical Pod Auto-Scaling
This form of auto-scaling is not enabled by default and must be explicitly enabled in the Shoot
by setting .spec.kubernetes.verticalPodAutoscaler.enabled=true
.
The reason is that it was only introduced lately, and some end-users might have already deployed their own VPA into their clusters, i.e., enabling it by default would interfere with such custom deployments and lead to issues, eventually.
Gardener is also leveraging an upstream community tool, i.e., the Kubernetes vertical-pod-autoscaler
component.
If enabled, Gardener will deploy it as part of the control plane into the seed cluster.
It will also be used for the vertical autoscaling of Gardener’s system components deployed into the kube-system
namespace of shoot clusters, for example, kube-proxy
or metrics-server
.
You might want to refer to the official documentation for this component to get more information how to use it.
The Shoot
API allows to configure a few flags of the vertical-pod-autoscaler
.
⚠️ Please note that if you disable the VPA again, then the related CustomResourceDefinition
s will remain in your shoot cluster (although, nobody will act on them).
This will also keep all existing VerticalPodAutoscaler
objects in the system, including those that might be created by you. You can delete the CustomResourceDefinition
s yourself using kubectl delete crd
if you want to get rid of them.