그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그
6 minute read
Gardener Configuration and Usage
Gardener automates the full lifecycle of Kubernetes clusters as a service. Additionally, it has several extension points allowing external controllers to plug-in to the lifecycle. As a consequence, there are several configuration options for the various custom resources that are partially required.
This document describes the:
- Configuration and usage of Gardener as operator/administrator.
- Configuration and usage of Gardener as end-user/stakeholder/customer.
Configuration and Usage of Gardener as Operator/Administrator
When we use the terms “operator/administrator”, we refer to both the people deploying and operating Gardener. Gardener consists of the following components:
gardener-apiserver, a Kubernetes-native API extension that serves custom resources in the Kubernetes-style (like
Shoots), and a component that contains multiple admission plugins.
gardener-admission-controller, an HTTP(S) server with several handlers to be used in a ValidatingWebhookConfiguration.
gardener-controller-manager, a component consisting of multiple controllers that implement reconciliation and deletion flows for some of the custom resources (e.g., it contains the logic for maintaining
gardener-scheduler, a component that assigns newly created
Shootclusters to appropriate
gardenlet, a component running in seed clusters and consisting out of multiple controllers that implement reconciliation and deletion flows for some of the custom resources (e.g., it contains the logic for reconciliation and deletion of
Each of these components have various configuration options.
gardener-apiserver uses the standard API server library maintained by the Kubernetes community, and as such it mainly supports command line flags.
Other components use so-called componentconfig files that describe their configuration in a Kubernetes-style versioned object.
Configuration File for Gardener Admission Controller
The Gardener admission controller only supports one command line flag, which should be a path to a valid admission-controller configuration file. Please take a look at this example configuration.
Configuration File for Gardener Controller Manager
The Gardener controller manager only supports one command line flag, which should be a path to a valid controller-manager configuration file. Please take a look at this example configuration.
Configuration File for Gardener Scheduler
The Gardener scheduler also only supports one command line flag, which should be a path to a valid scheduler configuration file. Please take a look at this example configuration. Information about the concepts of the Gardener scheduler can be found at Gardener Scheduler.
Configuration File for gardenlet
The gardenlet also only supports one command line flag, which should be a path to a valid gardenlet configuration file. Please take a look at this example configuration. Information about the concepts of the Gardenlet can be found at gardenlet.
After successful deployment of the four components, you need to setup the system.
Let’s first focus on some “static” configuration.
gardenlet starts, it scans the
garden namespace of the garden cluster for
Secrets that have influence on its reconciliation loops, mainly the
Internal domain secret - contains the DNS provider credentials (having appropriate privileges) which will be used to create/delete the so-called “internal” DNS records for the Shoot clusters, please see this yaml file for an example.
- This secret is used in order to establish a stable endpoint for shoot clusters, which is used internally by all control plane components.
- The DNS records are normal DNS records but called “internal” in our scenario because only the kubeconfigs for the control plane components use this endpoint when talking to the shoot clusters.
- It is forbidden to change the internal domain secret if there are existing shoot clusters.
Default domain secrets (optional) - contain the DNS provider credentials (having appropriate privileges) which will be used to create/delete DNS records for a default domain for shoots (e.g.,
example.com), please see this yaml file for an example.
- Not every end-user/stakeholder/customer has its own domain, however, Gardener needs to create a DNS record for every shoot cluster.
- As landscape operator you might want to define a default domain owned and controlled by you that is used for all shoot clusters that don’t specify their own domain.
- If you have multiple default domain secrets defined you can add a priority as an annotation (
dns.gardener.cloud/domain-default-priority) to select which domain should be used for new shoots during creation. The domain with the highest priority is selected during shoot creation. If there is no annotation defined, the default priority is
0, also all non integer values are considered as priority
Alerting secrets (optional) - contain the alerting configuration and credentials for the AlertManager to send email alerts. It is also possible to configure the monitoring stack to send alerts to an AlertManager not deployed by Gardener to handle alerting. Please see this yaml file for an example.
- If email alerting is configured:
- An AlertManager is deployed into each seed cluster that handles the alerting for all shoots on the seed cluster.
- Gardener will inject the SMTP credentials into the configuration of the AlertManager.
- The AlertManager will send emails to the configured email address in case any alerts are firing.
- If an external AlertManager is configured:
- Each shoot has a Prometheus responsible for monitoring components and sending out alerts. The alerts will be sent to a URL configured in the alerting secret.
- This external AlertManager is not managed by Gardener and can be configured however the operator sees fit.
- Supported authentication types are no authentication, basic, or mutual TLS.
- If email alerting is configured:
OpenVPN Diffie-Hellmann Key secret (optional) - contains the self-generated Diffie-Hellmann key used by OpenVPN in your landscape, please see this yaml file for an example.
- If you don’t specify a custom key, then a default key is used, but for productive landscapes it’s recommend to create a landscape-specific key and define it.
Global monitoring secrets (optional) - contains basic authentication credentials for the Prometheus aggregating metrics for all clusters.
- These secrets are synced to each seed cluster and used to gain access to the aggregate monitoring components.
Apart from this “static” configuration there are several custom resources extending the Kubernetes API and used by Gardener. As an operator/administrator, you have to configure some of them to make the system work.
Configuration and Usage of Gardener as End-User/Stakeholder/Customer
As an end-user/stakeholder/customer, you are using a Gardener landscape that has been setup for you by another team.
You don’t need to care about how Gardener itself has to be configured or how it has to be deployed.
Take a look at Gardener API Server - the topic describes which resources are offered by Gardener.
You may want to have a more detailed look for