7 minute read  

Feature Gates in Gardener

This page contains an overview of the various feature gates an administrator can specify on different Gardener components.

Overview

Feature gates are a set of key=value pairs that describe Gardener features. You can turn these features on or off using the a component configuration file for a specific component.

Each Gardener component lets you enable or disable a set of feature gates that are relevant to that component. For example this is the configuration of the gardenlet component.

The following tables are a summary of the feature gates that you can set on different Gardener components.

  • The “Since” column contains the Gardener release when a feature is introduced or its release stage is changed.
  • The “Until” column, if not empty, contains the last Gardener release in which you can still use a feature gate.
  • If a feature is in the Alpha or Beta state, you can find the feature listed in the Alpha/Beta feature gate table.
  • If a feature is stable you can find all stages for that feature listed in the Graduated/Deprecated feature gate table.
  • The Graduated/Deprecated feature gate table also lists deprecated and withdrawn features.

Feature gates for Alpha or Beta features

FeatureDefaultStageSinceUntil
HVPAfalseAlpha0.31
HVPAForShootedSeedfalseAlpha0.32
ManagedIstiofalseAlpha1.51.18
ManagedIstiotrueBeta1.19
APIServerSNIfalseAlpha1.71.18
APIServerSNItrueBeta1.19
SeedChangefalseAlpha1.12
SeedKubeSchedulerfalseAlpha1.15
ReversedVPNfalseAlpha1.221.41
ReversedVPNtrueBeta1.42
RotateSSHKeypairOnMaintenancefalseAlpha1.281.44
RotateSSHKeypairOnMaintenancetrueBeta1.45
WorkerPoolKubernetesVersionfalseAlpha1.351.45
WorkerPoolKubernetesVersiontrueBeta1.46
CopyEtcdBackupsDuringControlPlaneMigrationfalseAlpha1.37
SecretBindingProviderValidationfalseAlpha1.38
ForceRestorefalseAlpha1.39
DisableDNSProviderManagementfalseAlpha1.41
ShootCARotationfalseAlpha1.42
ShootMaxTokenExpirationOverwritefalseAlpha1.431.44
ShootMaxTokenExpirationOverwritetrueBeta1.45
ShootMaxTokenExpirationValidationfalseAlpha1.431.45
ShootMaxTokenExpirationValidationtrueBeta1.46

Feature gates for graduated or deprecated features

FeatureDefaultStageSinceUntil
NodeLocalDNSfalseAlpha1.7
NodeLocalDNSRemoved1.26
KonnectivityTunnelfalseAlpha1.6
KonnectivityTunnelRemoved1.27
MountHostCADirectoriesfalseAlpha1.111.25
MountHostCADirectoriestrueBeta1.261.27
MountHostCADirectoriestrueGA1.27
MountHostCADirectoriesRemoved1.30
DisallowKubeconfigRotationForShootInDeletionfalseAlpha1.281.31
DisallowKubeconfigRotationForShootInDeletiontrueBeta1.321.35
DisallowKubeconfigRotationForShootInDeletiontrueGA1.36
DisallowKubeconfigRotationForShootInDeletionRemoved1.38
LoggingfalseAlpha0.131.40
LoggingfalseRemoved1.41
AdminKubeconfigRequestfalseAlpha1.241.38
AdminKubeconfigRequesttrueBeta1.391.41
AdminKubeconfigRequesttrueGA1.42
UseDNSRecordsfalseAlpha1.271.38
UseDNSRecordstrueBeta1.391.43
UseDNSRecordstrueGA1.44
CachedRuntimeClientsfalseAlpha1.71.33
CachedRuntimeClientstrueBeta1.341.44
CachedRuntimeClientstrueGA1.45
DenyInvalidExtensionResourcesfalseAlpha1.311.41
DenyInvalidExtensionResourcestrueBeta1.421.44
DenyInvalidExtensionResourcestrueGA1.45

Using a feature

A feature can be in Alpha, Beta or GA stage. An Alpha feature means:

  • Disabled by default.
  • Might be buggy. Enabling the feature may expose bugs.
  • Support for feature may be dropped at any time without notice.
  • The API may change in incompatible ways in a later software release without notice.
  • Recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support.

A Beta feature means:

  • Enabled by default.
  • The feature is well tested. Enabling the feature is considered safe.
  • Support for the overall feature will not be dropped, though details may change.
  • The schema and/or semantics of objects may change in incompatible ways in a subsequent beta or stable release. When this happens, we will provide instructions for migrating to the next version. This may require deleting, editing, and re-creating API objects. The editing process may require some thought. This may require downtime for applications that rely on the feature.
  • Recommended for only non-critical uses because of potential for incompatible changes in subsequent releases.

Please do try Beta features and give feedback on them! After they exit beta, it may not be practical for us to make more changes.

A General Availability (GA) feature is also referred to as a stable feature. It means:

  • The feature is always enabled; you cannot disable it.
  • The corresponding feature gate is no longer needed.
  • Stable versions of features will appear in released software for many subsequent versions.

List of Feature Gates

FeatureRelevant ComponentsDescription
HVPAgardenletEnables simultaneous horizontal and vertical scaling in Seed Clusters.
HVPAForShootedSeedgardenletEnables simultaneous horizontal and vertical scaling in managed seed (aka “shooted seed”) clusters.
ManagedIstiogardenletEnables a Gardener-tailored Istio in each Seed cluster. Disable this feature if Istio is already installed in the cluster. Istio is not automatically removed if this feature is disabled. See the detailed documentation for more information.
APIServerSNIgardenletEnables only one LoadBalancer to be used for every Shoot cluster API server in a Seed. Enable this feature when ManagedIstio is enabled or Istio is manually deployed in Seed cluster. See GEP-8 for more details.
CachedRuntimeClientsgardener-controller-manager, gardenletEnables a cache in the controller-runtime clients, that Gardener components use. The feature gate can be specified for gardenlet and gardener-controller-manager (and gardener-scheduler for the versions < 1.29).
SeedChangegardener-apiserverEnables updating the spec.seedName field during shoot validation from a non-empty value in order to trigger shoot control plane migration.
SeedKubeSchedulergardenletAdds custom kube-scheduler in gardener-kube-scheduler namespace. It schedules pods with scheduler name gardener-kube-scheduler on Nodes with higher resource utilization. It requires Seed cluster with kubernetes version 1.18 or higher.
ReversedVPNgardenletReverses the connection setup of the vpn tunnel between the Seed and the Shoot cluster(s). It allows Seed and Shoot clusters to be in different networks with only direct access in one direction (Shoot -> Seed). In addition to that, it reduces the amount of load balancers required, i.e. no load balancers are required for the vpn tunnel anymore. It requires APIServerSNI and kubernetes version 1.18 or higher to work. Details can be found in GEP-14.
AdminKubeconfigRequestgardener-apiserverEnables the AdminKubeconfigRequest endpoint on Shoot resources. See GEP-16 for more details.
UseDNSRecordsgardener-apiserver, gardener-controller-manager, gardenletEnables using DNSRecord resources for Gardener DNS records instead of DNSProvider, DNSEntry, and DNSOwner resources. See Contract: DNSRecord resources for more details.
RotateSSHKeypairOnMaintenancegardener-controller-managerEnables SSH keypair rotation in the maintenance controller of the gardener-controller-manager. Details can be found in GEP-15.
DenyInvalidExtensionResourcesgardenletCauses the seed-admission-controller to deny invalid extension resources, instead of just logging validation errors.
WorkerPoolKubernetesVersiongardener-apiserverAllows to overwrite the Kubernetes version used for shoot clusters per worker pool (see this document)
CopyEtcdBackupsDuringControlPlaneMigrationgardenletEnables the copy of etcd backups from the object store of the source seed to the object store of the destination seed during control plane migration.
SecretBindingProviderValidationgardener-apiserverEnables validations on Gardener API server that:
- requires the provider type of a SecretBinding to be set (on SecretBinding creation)
- requires the SecretBinding provider type to match the Shoot provider type (on Shoot creation)
- enforces immutability on the provider type of a SecretBinding
ForceRestoregardenletEnables forcing the shoot’s restoration to the destination seed during control plane migration if the preparation for migration in the source seed is not finished after a certain grace period and is considered unlikely to succeed (falling back to the control plane migration “bad case” scenario). If you enable this feature gate, make sure to also enable UseDNSRecords and CopyEtcdBackupsDuringControlPlaneMigration.
DisableDNSProviderManagementgardenletDisables management of dns.gardener.cloud/v1alpha1.DNSProvider resources. In this case, the shoot-dns-service extension will take this over if it is installed. This feature is only effective if the feature UseDNSRecords is true.
ShootCARotationgardener-apiserver, gardenletEnables the feature to trigger automated CA rotation for shoot clusters.
ShootMaxTokenExpirationOverwritegardener-apiserverMakes the Gardener API server overwriting values in the .spec.kubernetes.kubeAPIServer.serviceAccountConfig.maxTokenExpiration field of Shoot specifications to
- be at least 720h (30d) when the current value is lower
- be at most 2160h (90d) when the current value is higher
before persisting the object to etcd.
ShootMaxTokenExpirationValidationgardener-apiserverEnables validations on Gardener API server that enforce that the value of the .spec.kubernetes.kubeAPIServer.serviceAccountConfig.maxTokenExpiration field
- is at least 720h (30d).
- is at most 2160h (90d).
Only enable this after ShootMaxTokenExpirationOverwrite is enabled and all shoots got updated accordingly.