Cleanup of Shoot Clusters in Deletion
When a shoot cluster is deleted then Gardener tries to gracefully remove most of the Kubernetes resources inside the cluster. This is to prevent that any infrastructure or other artifacts remain after the shoot deletion.
The cleanup is performed in five steps. Some resources are deleted with a grace period, and all resources are forcefully deleted (by removing blocking finalizers) after some time to not block the cluster deletion entirely.
Cleanup steps:
- All
ValidatingWebhookConfigurations,MutatingWebhookConfigurations, andValidatingAdmissionPolicyBindings are deleted with a5mgrace period. Forceful finalization happens after5m. - All
APIServices andCustomResourceDefinitions are deleted with a5mgrace period. Forceful finalization happens after1h. - All
CronJobs,DaemonSets,Deployments,Ingresss,Jobs,Pods,ReplicaSets,ReplicationControllers,Services,StatefulSets,PersistentVolumeClaims are deleted with a5mgrace period. Forceful finalization happens after5m.If the
Shootis annotated withshoot.gardener.cloud/skip-cleanup=true, then onlyServices,VolumeSnapshots andPersistentVolumeClaims are considered. - All
VolumeSnapshots andVolumeSnapshotContents are deleted with a5mgrace period. Forceful finalization happens after1h. - All
ManagedResourcesare deleted. Forceful finalization by Gardener Resource Manager happens after a5mgrace period.
It is possible to override the finalization grace periods via annotations on the Shoot:
shoot.gardener.cloud/cleanup-webhooks-finalize-grace-period-seconds(for the resources handled in step 1)shoot.gardener.cloud/cleanup-extended-apis-finalize-grace-period-seconds(for the resources handled in step 2)shoot.gardener.cloud/cleanup-kubernetes-resources-finalize-grace-period-seconds(for the resources handled in step 3)shoot.gardener.cloud/cleanup-managed-resources-finalize-grace-period-seconds(for the resources handled in step 5)
⚠️ If "0" is provided, then all resources are finalized immediately without waiting for any graceful deletion. Please be aware that this might lead to orphaned infrastructure artifacts.
Excluding Resources From Cleanup
Resources labeled with shoot.gardener.cloud/no-cleanup=true are excluded from the cleanup steps described above and are left untouched during shoot deletion. Use this label to prevent Gardener from deleting specific resources in the shoot cluster.
In some cases, even with the described cleanup logic, shoot deletion may still become stuck due to remaining resources in the shoot cluster. To address this, Gardener ignores any remaining resources in the following situations:
- Objects that belong to namespaces which have already been deleted (finalized).
- Objects that were created after the cleanup process began for the first time, plus the finalize grace period.
- Objects whose
resources.gardener.cloud/originannotation references an existingManagedResourcein the shoot's control-plane namespace in the seed cluster.
For objects whose origin annotation references an existing ManagedResource in the shoot’s control-plane namespace in the seed cluster, Gardener skips waiting for them to disappear during Kubernetes resource cleanup. Later in shoot deletion, Gardener (or the owning extension) requests deletion of the ManagedResource. The gardener-resource-manager deployed in the shoot’s control plane then deletes the objects it manages in the shoot cluster before allowing the ManagedResource itself to disappear.