그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그 그
3 minute read
Developer Docs for Gardener Extension Registry Cache
This document outlines how the Shoot reconciliation and deletion work for a Shoot with the registry-cache extension enabled.
Shoot reconciliation
This section outlines how the reconciliation works for a Shoot with the registry-cache extension enabled.
Extension enablement/reconciliation
This section outlines how the extension enablement/reconciliation works, e.g the extension has beeen added to the Shoot spec.
- As part of the Shoot reconciliation flow, gardenlet deploys the Extension resource.
- The registry-cache extension reconciles the Extension resource. pkg/controller/extension/actuator.go contains the implementation of the extension.Actuator interface. The reconciliation of an Extension of type
registry-cache
consists of the following steps:- The extension checks if a registry has been removed (by comparing the status and the spec of the Extension). If an upstream is being removed, then it deploys the
registry-cleaner
DaemonSet to the Shoot cluster to clean up the existing configuration for the upstream that has to be removed. - The registry-cache extension deploys resources to the Shoot cluster via ManagedResource. For every configured upstream it creates a StatefulSet (with PVC), Service and other resources.
- It lists all Services from the
kube-system
namespace having theupstream-host
label. It will return an error (and retry in exponential backoff) until the Services count matches the configured registries count. - When there is a Service created for each configured upstream registry, the registry-cache extension populates the Extension resource status. In the Extension status, for each upstream, it maintains an endpoint (in format
http://<cluster-ip>:5000
) which can be used to access the registry cache from within the Shoot cluster.<cluster-ip>
is the cluster IP of the registry cache Service. The cluster IP of a Service is assigned by the Kubernetes API server on Service creation.
- The extension checks if a registry has been removed (by comparing the status and the spec of the Extension). If an upstream is being removed, then it deploys the
- As part of the Shoot reconciliation flow, gardenlet deploys the OperatingSystemConfig resource.
- The registry-cache extension serves a webhook that mutates the OperatingSystemConfig resource for Shoots having the registry-cache extension enabled (the corresponding namespace gets labeled by gardenlet with
extensions.gardener.cloud/registry-cache=true
). pkg/webhook/operatingsystemconfig/ensurer.go contains implementation of the genericmutator.Ensurer interface.- The webhook appends the configure-containerd-registries.sh script to the OperatingSystemConfig files. The script accepts registries in the format
<upstream_host>,<registry_cache_endpoint>,<upstream_url>
separated by a space. For each given registry the script waits until the given registry is available (a request to the<registry_cache_endpoint>
succeeds). Then it creates ahosts.toml
file for the given<upstream_host>
. In short, thehosts.toml
file instructs containerd to first try to pull images for the given<upstream_host>
from the configured<registry_cache_endpoint>
. For more information about containerd registry configuration, see the containerd documentation. The motivation to introduce theconfigure-containerd-registries.sh
script is that we need to create thehosts.toml
file when the corresponding registry is available. For more details, see https://github.com/gardener/gardener-extension-registry-cache/pull/68. - The webhook appends the
configure-containerd-registries.service
unit to the OperatingSystemConfig units. The webhook fetches the Extension resource and then it configures the unit to invoke theconfigure-containerd-registries.sh
script with the registries from the Extension status.
- The webhook appends the configure-containerd-registries.sh script to the OperatingSystemConfig files. The script accepts registries in the format
Extension disablement
This section outlines how the extension disablement works, i.e the extension has be removed from the Shoot spec.
- As part of the Shoot reconciliation flow, gardenlet destroys the Extension resource because it is no longer needed.
- If the Extension resource contains registries in its status, the registry-cache extension deploys the
registry-cleaner
DaemonSet to the Shoot cluster to clean up the existing registry configuration. - The extension deletes the ManagedResource containing the registry cache resources.
- If the Extension resource contains registries in its status, the registry-cache extension deploys the
Shoot deletion
This section outlines how the deletion works for a Shoot with the registry-cache extension enabled.
- As part of the Shoot deletion flow, gardenlet destroys the Extension resource.
- In the Shoot deletion flow the Extension resource is deleted after the Worker resource. Hence, there is no need to deploy the
registry-cleaner
DaemonSet to the Shoot cluster to clean up the existing registry configuration. - The extension deletes the ManagedResource containing the registry cache resources.
- In the Shoot deletion flow the Extension resource is deleted after the Worker resource. Hence, there is no need to deploy the