2 minute read
Gardener Container Runtime Extension
At the lowest layers of a Kubernetes node is the software that, among other things, starts and stops containers. It is called “Container Runtime”. The most widely known container runtime is Docker, but it is not alone in this space. In fact, the container runtime space has been rapidly evolving.
Kubernetes supports different container runtimes using Container Runtime Interface (CRI) – a plugin interface which enables kubelet to use a wide variety of container runtimes.
Gardener supports creation of Worker machines using CRI, more information can be found here: CRI Support.
Prior to the
Container Runtime Extensibility concept, Gardener used Docker as the only
container runtime to use in shoot worker machines. Because of the wide variety of different container runtimes
offers multiple important features (for example enhanced security concepts) it is important to enable end users to use other container runtimes as well.
ContainerRuntime Extension Resource
Here is what a typical
ContainerRuntime resource would look-like:
--- apiVersion: extensions.gardener.cloud/v1alpha1 kind: ContainerRuntime metadata: name: my-container-runtime spec: binaryPath: /var/bin/containerruntimes type: gvisor workerPool: name: worker-ubuntu selector: matchLabels: worker.gardener.cloud/pool: worker-ubuntu
Gardener deploys one
ContainerRuntime resource per worker pool per CRI.
To exemplify this, consider a Shoot having two worker pools (
containerd as the CRI as well as
kata as enabled container runtimes.
Gardener would deploy four
ContainerRuntime resources. For
ContainerRuntime for type
gvisor and one for type
kata. The same resource are being deployed for
Supporting a new Container Runtime Provider
To add support for another container runtime (e.g., gvisor, kata-containers, etc.) a container runtime extension controller needs to be implemented. It should support Gardener’s supported CRI plugins.
The container runtime extension should install the necessary resources into the shoot cluster (e.g.,
RuntimeClasses), and it should copy the runtime binaries to the relevant worker machines in path:
Gardener labels the shoot nodes according to the CRI configured:
worker.gardener.cloud/cri-name=containerd) and multiple labels for each of the container runtimes configured for the shoot Worker machine:
The way to install the binaries is by creating a daemon set which copies the binaries from an image in a docker registry to the relevant labeled Worker’s nodes (avoid downloading binaries from internet to also cater with isolated environments).
For additional reference, please have a look at the runtime-gvsior provider extension, which provides more information on how to configure the necessary charts as well as the actuators required to reconcile container runtime inside the
Shoot cluster to the desired state.