This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Etcd Druid

A druid for etcd management in Gardener

ETCD Druid

Background

Etcd in the control plane of Kubernetes clusters which are managed by Gardener is deployed as a StatefulSet. The statefulset has replica of a pod containing two containers namely, etcd and backup-restore. The etcd container calls components in etcd-backup-restore via REST api to perform data validation before etcd is started. If this validation fails etcd data is restored from the latest snapshot stored in the cloud-provider’s object store. Once etcd has started, the etcd-backup-restore periodically creates full and delta snapshots. It also performs defragmentation of etcd data periodically.

The etcd-backup-restore needs as input the cloud-provider information comprising of security credentials to access the object store, the object store bucket name and prefix for the directory used to store snapshots. Currently, for operations like migration and validation, the bash script has to be updated to initiate the operation.

Goals

Deploy etcd and etcd-backup-restore using an etcd CRD.
Support more than one etcd replica.
Perform scheduled snapshots.
Support operations such as restores, defragmentation and scaling with zero-downtime.
Handle cloud-provider specific operation logic.
Trigger a full backup on request before volume deletion.
Offline compaction of full and delta snapshots stored in object store.

Proposal

The existing method of deploying etcd and backup-sidecar as a StatefulSet alleviates the pain of ensuring the pods are live and ready after node crashes. However, deploying etcd as a Statefulset introduces a plethora of challenges. The etcd controller should be smart enough to handle etcd statefulsets taking into account limitations imposed by statefulsets. The controller shall update the status regarding how to target the K8s objects it has created. This field in the status can be leveraged by HVPA to scale etcd resources eventually.

CRD specification

The etcd CRD should contain the information required to create the etcd and backup-restore sidecar in a pod/statefulset.

apiVersion: druid.gardener.cloud/v1alpha1
kind: Etcd
metadata:
  finalizers:
  - druid.gardener.cloud/etcd
  name: test
  namespace: demo
spec:
  annotations:
    app: etcd-statefulset
    gardener.cloud/role: controlplane
    networking.gardener.cloud/to-dns: allowed
    networking.gardener.cloud/to-private-networks: allowed
    networking.gardener.cloud/to-public-networks: allowed
    role: test
  backup:
    deltaSnapshotMemoryLimit: 1Gi
    deltaSnapshotPeriod: 300s
    fullSnapshotSchedule: 0 */24 * * *
    garbageCollectionPeriod: 43200s
    garbageCollectionPolicy: Exponential
    imageRepository: europe-docker.pkg.dev/gardener-project/public/gardener/etcdbrctl
    imageVersion: v0.25.0
    port: 8080
    resources:
      limits:
        cpu: 500m
        memory: 2Gi
      requests:
        cpu: 23m
        memory: 128Mi
    snapstoreTempDir: /var/etcd/data/temp
  etcd:
    Quota: 8Gi
    clientPort: 2379
    defragmentationSchedule: 0 */24 * * *
    enableTLS: false
    imageRepository: europe-docker.pkg.dev/gardener-project/public/gardener/etcd-wrapper
    imageVersion: v0.1.0
    initialClusterState: new
    initialClusterToken: new
    metrics: basic
    pullPolicy: IfNotPresent
    resources:
      limits:
        cpu: 2500m
        memory: 4Gi
      requests:
        cpu: 500m
        memory: 1000Mi
    serverPort: 2380
    storageCapacity: 80Gi
    storageClass: gardener.cloud-fast
  sharedConfig:
    autoCompactionMode: periodic
    autoCompactionRetention: 30m
  labels:
    app: etcd-statefulset
    gardener.cloud/role: controlplane
    networking.gardener.cloud/to-dns: allowed
    networking.gardener.cloud/to-private-networks: allowed
    networking.gardener.cloud/to-public-networks: allowed
    role: test
  pvcRetentionPolicy: DeleteAll
  replicas: 1
  storageCapacity: 80Gi
  storageClass: gardener.cloud-fast
  store:
    storageContainer: test
    storageProvider: S3
    storePrefix: etcd-test
    storeSecret: etcd-backup
  tlsClientSecret: etcd-client-tls
  tlsServerSecret: etcd-server-tls
status:
  etcd:
    apiVersion: apps/v1
    kind: StatefulSet
    name: etcd-test

Implementation Agenda

As first step implement defragmentation during maintenance windows. Subsequently, we will add zero-downtime upgrades and defragmentation.

Workflow

Deployment workflow

controller-diagram

Defragmentation workflow

defrag-diagram

Local Setup

To setup Etcd-druid locally as a pod running inside a kind cluster, follow this document

1 - API Reference

Packages:

druid.gardener.cloud/v1alpha1

druid.gardener.cloud/v1alpha1

Package v1alpha1 is the v1alpha1 version of the etcd-druid API.

Resource Types:

BackupSpec

(Appears on: EtcdSpec)

BackupSpec defines parameters associated with the full and delta snapshots of etcd.

Field	Description
`port` int32	(Optional) Port define the port on which etcd-backup-restore server will be exposed.
`tls` TLSConfig	(Optional)
`image` string	(Optional) Image defines the etcd container image and tag
`store` StoreSpec	(Optional) Store defines the specification of object store provider for storing backups.
`resources` Kubernetes core/v1.ResourceRequirements	(Optional) Resources defines compute Resources required by backup-restore container. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
`compactionResources` Kubernetes core/v1.ResourceRequirements	(Optional) CompactionResources defines compute Resources required by compaction job. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
`fullSnapshotSchedule` string	(Optional) FullSnapshotSchedule defines the cron standard schedule for full snapshots.
`garbageCollectionPolicy` GarbageCollectionPolicy	(Optional) GarbageCollectionPolicy defines the policy for garbage collecting old backups
`garbageCollectionPeriod` Kubernetes meta/v1.Duration	(Optional) GarbageCollectionPeriod defines the period for garbage collecting old backups
`deltaSnapshotPeriod` Kubernetes meta/v1.Duration	(Optional) DeltaSnapshotPeriod defines the period after which delta snapshots will be taken
`deltaSnapshotMemoryLimit` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) DeltaSnapshotMemoryLimit defines the memory limit after which delta snapshots will be taken
`compression` CompressionSpec	(Optional) SnapshotCompression defines the specification for compression of Snapshots.
`enableProfiling` bool	(Optional) EnableProfiling defines if profiling should be enabled for the etcd-backup-restore-sidecar
`etcdSnapshotTimeout` Kubernetes meta/v1.Duration	(Optional) EtcdSnapshotTimeout defines the timeout duration for etcd FullSnapshot operation
`leaderElection` LeaderElectionSpec	(Optional) LeaderElection defines parameters related to the LeaderElection configuration.

ClientService

(Appears on: EtcdConfig)

ClientService defines the parameters of the client service that a user can specify

Field	Description
`annotations` map[string]string	(Optional) Annotations specify the annotations that should be added to the client service
`labels` map[string]string	(Optional) Labels specify the labels that should be added to the client service

CompactionMode (`string` alias)

(Appears on: SharedConfig)

CompactionMode defines the auto-compaction-mode: ‘periodic’ or ‘revision’. ‘periodic’ for duration based retention and ‘revision’ for revision number based retention.

CompressionPolicy (`string` alias)

(Appears on: CompressionSpec)

CompressionPolicy defines the type of policy for compression of snapshots.

CompressionSpec

(Appears on: BackupSpec)

CompressionSpec defines parameters related to compression of Snapshots(full as well as delta).

Field	Description
`enabled` bool	(Optional)
`policy` CompressionPolicy	(Optional)

Condition

(Appears on: EtcdCopyBackupsTaskStatus, EtcdStatus)

Condition holds the information about the state of a resource.

Field	Description
`type` ConditionType	Type of the Etcd condition.
`status` ConditionStatus	Status of the condition, one of True, False, Unknown.
`lastTransitionTime` Kubernetes meta/v1.Time	Last time the condition transitioned from one status to another.
`lastUpdateTime` Kubernetes meta/v1.Time	Last time the condition was updated.
`reason` string	The reason for the condition’s last transition.
`message` string	A human-readable message indicating details about the transition.

ConditionStatus (`string` alias)

(Appears on: Condition)

ConditionStatus is the status of a condition.

ConditionType (`string` alias)

(Appears on: Condition)

ConditionType is the type of condition.

CrossVersionObjectReference

(Appears on: EtcdStatus)

CrossVersionObjectReference contains enough information to let you identify the referred resource.

Field	Description
`kind` string	Kind of the referent
`name` string	Name of the referent
`apiVersion` string	(Optional) API version of the referent

Etcd

Etcd is the Schema for the etcds API

Field Description

metadata
Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.

spec
EtcdSpec

`selector` Kubernetes meta/v1.LabelSelector	selector is a label query over pods that should match the replica count. It must match the pod template’s labels. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
`labels` map[string]string
`annotations` map[string]string	(Optional)
`etcd` EtcdConfig
`backup` BackupSpec
`sharedConfig` SharedConfig	(Optional)
`schedulingConstraints` SchedulingConstraints	(Optional)
`replicas` int32
`priorityClassName` string	(Optional) PriorityClassName is the name of a priority class that shall be used for the etcd pods.
`storageClass` string	(Optional) StorageClass defines the name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1
`storageCapacity` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) StorageCapacity defines the size of persistent volume.
`volumeClaimTemplate` string	(Optional) VolumeClaimTemplate defines the volume claim template to be created

status
EtcdStatus

EtcdConfig

(Appears on: EtcdSpec)

EtcdConfig defines parameters associated etcd deployed

Field	Description
`quota` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) Quota defines the etcd DB quota.
`defragmentationSchedule` string	(Optional) DefragmentationSchedule defines the cron standard schedule for defragmentation of etcd.
`serverPort` int32	(Optional)
`clientPort` int32	(Optional)
`image` string	(Optional) Image defines the etcd container image and tag
`authSecretRef` Kubernetes core/v1.SecretReference	(Optional)
`metrics` MetricsLevel	(Optional) Metrics defines the level of detail for exported metrics of etcd, specify ‘extensive’ to include histogram metrics.
`resources` Kubernetes core/v1.ResourceRequirements	(Optional) Resources defines the compute Resources required by etcd container. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
`clientUrlTls` TLSConfig	(Optional) ClientUrlTLS contains the ca, server TLS and client TLS secrets for client communication to ETCD cluster
`peerUrlTls` TLSConfig	(Optional) PeerUrlTLS contains the ca and server TLS secrets for peer communication within ETCD cluster Currently, PeerUrlTLS does not require client TLS secrets for gardener implementation of ETCD cluster.
`etcdDefragTimeout` Kubernetes meta/v1.Duration	(Optional) EtcdDefragTimeout defines the timeout duration for etcd defrag call
`heartbeatDuration` Kubernetes meta/v1.Duration	(Optional) HeartbeatDuration defines the duration for members to send heartbeats. The default value is 10s.
`clientService` ClientService	(Optional) ClientService defines the parameters of the client service that a user can specify

EtcdCopyBackupsTask

EtcdCopyBackupsTask is a task for copying etcd backups from a source to a target store.

Field Description

metadata
Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.

spec
EtcdCopyBackupsTaskSpec

`sourceStore` StoreSpec	SourceStore defines the specification of the source object store provider for storing backups.
`targetStore` StoreSpec	TargetStore defines the specification of the target object store provider for storing backups.
`maxBackupAge` uint32	(Optional) MaxBackupAge is the maximum age in days that a backup must have in order to be copied. By default all backups will be copied.
`maxBackups` uint32	(Optional) MaxBackups is the maximum number of backups that will be copied starting with the most recent ones.
`waitForFinalSnapshot` WaitForFinalSnapshotSpec	(Optional) WaitForFinalSnapshot defines the parameters for waiting for a final full snapshot before copying backups.

status
EtcdCopyBackupsTaskStatus

EtcdCopyBackupsTaskSpec

(Appears on: EtcdCopyBackupsTask)

EtcdCopyBackupsTaskSpec defines the parameters for the copy backups task.

Field	Description
`sourceStore` StoreSpec	SourceStore defines the specification of the source object store provider for storing backups.
`targetStore` StoreSpec	TargetStore defines the specification of the target object store provider for storing backups.
`maxBackupAge` uint32	(Optional) MaxBackupAge is the maximum age in days that a backup must have in order to be copied. By default all backups will be copied.
`maxBackups` uint32	(Optional) MaxBackups is the maximum number of backups that will be copied starting with the most recent ones.
`waitForFinalSnapshot` WaitForFinalSnapshotSpec	(Optional) WaitForFinalSnapshot defines the parameters for waiting for a final full snapshot before copying backups.

EtcdCopyBackupsTaskStatus

(Appears on: EtcdCopyBackupsTask)

EtcdCopyBackupsTaskStatus defines the observed state of the copy backups task.

Field	Description
`conditions` []Condition	(Optional) Conditions represents the latest available observations of an object’s current state.
`observedGeneration` int64	(Optional) ObservedGeneration is the most recent generation observed for this resource.
`lastError` string	(Optional) LastError represents the last occurred error.

EtcdMemberConditionStatus (`string` alias)

(Appears on: EtcdMemberStatus)

EtcdMemberConditionStatus is the status of an etcd cluster member.

EtcdMemberStatus

(Appears on: EtcdStatus)

EtcdMemberStatus holds information about a etcd cluster membership.

Field	Description
`name` string	Name is the name of the etcd member. It is the name of the backing `Pod`.
`id` string	(Optional) ID is the ID of the etcd member.
`role` EtcdRole	(Optional) Role is the role in the etcd cluster, either `Leader` or `Member`.
`status` EtcdMemberConditionStatus	Status of the condition, one of True, False, Unknown.
`reason` string	The reason for the condition’s last transition.
`lastTransitionTime` Kubernetes meta/v1.Time	LastTransitionTime is the last time the condition’s status changed.

EtcdRole (`string` alias)

(Appears on: EtcdMemberStatus)

EtcdRole is the role of an etcd cluster member.

EtcdSpec

(Appears on: Etcd)

EtcdSpec defines the desired state of Etcd

Field	Description
`selector` Kubernetes meta/v1.LabelSelector	selector is a label query over pods that should match the replica count. It must match the pod template’s labels. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
`labels` map[string]string
`annotations` map[string]string	(Optional)
`etcd` EtcdConfig
`backup` BackupSpec
`sharedConfig` SharedConfig	(Optional)
`schedulingConstraints` SchedulingConstraints	(Optional)
`replicas` int32
`priorityClassName` string	(Optional) PriorityClassName is the name of a priority class that shall be used for the etcd pods.
`storageClass` string	(Optional) StorageClass defines the name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1
`storageCapacity` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) StorageCapacity defines the size of persistent volume.
`volumeClaimTemplate` string	(Optional) VolumeClaimTemplate defines the volume claim template to be created

EtcdStatus

(Appears on: Etcd)

EtcdStatus defines the observed state of Etcd.

Field	Description
`observedGeneration` int64	(Optional) ObservedGeneration is the most recent generation observed for this resource.
`etcd` CrossVersionObjectReference	(Optional)
`conditions` []Condition	(Optional) Conditions represents the latest available observations of an etcd’s current state.
`serviceName` string	(Optional) ServiceName is the name of the etcd service.
`lastError` string	(Optional) LastError represents the last occurred error.
`clusterSize` int32	(Optional) Cluster size is the size of the etcd cluster.
`currentReplicas` int32	(Optional) CurrentReplicas is the current replica count for the etcd cluster.
`replicas` int32	(Optional) Replicas is the replica count of the etcd resource.
`readyReplicas` int32	(Optional) ReadyReplicas is the count of replicas being ready in the etcd cluster.
`ready` bool	(Optional) Ready is `true` if all etcd replicas are ready.
`updatedReplicas` int32	(Optional) UpdatedReplicas is the count of updated replicas in the etcd cluster.
`labelSelector` Kubernetes meta/v1.LabelSelector	(Optional) LabelSelector is a label query over pods that should match the replica count. It must match the pod template’s labels.
`members` []EtcdMemberStatus	(Optional) Members represents the members of the etcd cluster
`peerUrlTLSEnabled` bool	(Optional) PeerUrlTLSEnabled captures the state of peer url TLS being enabled for the etcd member(s)

GarbageCollectionPolicy (`string` alias)

(Appears on: BackupSpec)

GarbageCollectionPolicy defines the type of policy for snapshot garbage collection.

LeaderElectionSpec

(Appears on: BackupSpec)

LeaderElectionSpec defines parameters related to the LeaderElection configuration.

Field	Description
`reelectionPeriod` Kubernetes meta/v1.Duration	(Optional) ReelectionPeriod defines the Period after which leadership status of corresponding etcd is checked.
`etcdConnectionTimeout` Kubernetes meta/v1.Duration	(Optional) EtcdConnectionTimeout defines the timeout duration for etcd client connection during leader election.

MetricsLevel (`string` alias)

(Appears on: EtcdConfig)

MetricsLevel defines the level ‘basic’ or ‘extensive’.

SchedulingConstraints

(Appears on: EtcdSpec)

SchedulingConstraints defines the different scheduling constraints that must be applied to the pod spec in the etcd statefulset. Currently supported constraints are Affinity and TopologySpreadConstraints.

Field	Description
`affinity` Kubernetes core/v1.Affinity	(Optional) Affinity defines the various affinity and anti-affinity rules for a pod that are honoured by the kube-scheduler.
`topologySpreadConstraints` []Kubernetes core/v1.TopologySpreadConstraint	(Optional) TopologySpreadConstraints describes how a group of pods ought to spread across topology domains, that are honoured by the kube-scheduler.

SecretReference

(Appears on: TLSConfig)

SecretReference defines a reference to a secret.

Field	Description
`SecretReference` Kubernetes core/v1.SecretReference	(Members of `SecretReference` are embedded into this type.)
`dataKey` string	(Optional) DataKey is the name of the key in the data map containing the credentials.

SharedConfig

(Appears on: EtcdSpec)

SharedConfig defines parameters shared and used by Etcd as well as backup-restore sidecar.

Field	Description
`autoCompactionMode` CompactionMode	(Optional) AutoCompactionMode defines the auto-compaction-mode:‘periodic’ mode or ‘revision’ mode for etcd and embedded-Etcd of backup-restore sidecar.
`autoCompactionRetention` string	(Optional) AutoCompactionRetention defines the auto-compaction-retention length for etcd as well as for embedded-Etcd of backup-restore sidecar.

StorageProvider (`string` alias)

(Appears on: StoreSpec)

StorageProvider defines the type of object store provider for storing backups.

StoreSpec

(Appears on: BackupSpec, EtcdCopyBackupsTaskSpec)

StoreSpec defines parameters related to ObjectStore persisting backups

Field	Description
`container` string	(Optional) Container is the name of the container the backup is stored at.
`prefix` string	Prefix is the prefix used for the store.
`provider` StorageProvider	(Optional) Provider is the name of the backup provider.
`secretRef` Kubernetes core/v1.SecretReference	(Optional) SecretRef is the reference to the secret which used to connect to the backup store.

TLSConfig

(Appears on: BackupSpec, EtcdConfig)

TLSConfig hold the TLS configuration details.

Field	Description
`tlsCASecretRef` SecretReference
`serverTLSSecretRef` Kubernetes core/v1.SecretReference
`clientTLSSecretRef` Kubernetes core/v1.SecretReference	(Optional)

WaitForFinalSnapshotSpec

(Appears on: EtcdCopyBackupsTaskSpec)

WaitForFinalSnapshotSpec defines the parameters for waiting for a final full snapshot before copying backups.

Field	Description
`enabled` bool	Enabled specifies whether to wait for a final full snapshot before copying backups.
`timeout` Kubernetes meta/v1.Duration	(Optional) Timeout is the timeout for waiting for a final full snapshot. When this timeout expires, the copying of backups will be performed anyway. No timeout or 0 means wait forever.

Generated with gen-crd-api-reference-docs

2 - Druid

Packages:

druid.gardener.cloud/v1alpha1

druid.gardener.cloud/v1alpha1

Package v1alpha1 is the v1alpha1 version of the etcd-druid API.

Resource Types:

BackupSpec

(Appears on: EtcdSpec)

BackupSpec defines parameters associated with the full and delta snapshots of etcd.

Field	Description
`port` int32	(Optional) Port define the port on which etcd-backup-restore server will be exposed.
`tls` TLSConfig	(Optional)
`image` string	(Optional) Image defines the etcd container image and tag
`store` StoreSpec	(Optional) Store defines the specification of object store provider for storing backups.
`resources` Kubernetes core/v1.ResourceRequirements	(Optional) Resources defines compute Resources required by backup-restore container. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
`compactionResources` Kubernetes core/v1.ResourceRequirements	(Optional) CompactionResources defines compute Resources required by compaction job. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
`fullSnapshotSchedule` string	(Optional) FullSnapshotSchedule defines the cron standard schedule for full snapshots.
`garbageCollectionPolicy` GarbageCollectionPolicy	(Optional) GarbageCollectionPolicy defines the policy for garbage collecting old backups
`garbageCollectionPeriod` Kubernetes meta/v1.Duration	(Optional) GarbageCollectionPeriod defines the period for garbage collecting old backups
`deltaSnapshotPeriod` Kubernetes meta/v1.Duration	(Optional) DeltaSnapshotPeriod defines the period after which delta snapshots will be taken
`deltaSnapshotMemoryLimit` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) DeltaSnapshotMemoryLimit defines the memory limit after which delta snapshots will be taken
`compression` CompressionSpec	(Optional) SnapshotCompression defines the specification for compression of Snapshots.
`enableProfiling` bool	(Optional) EnableProfiling defines if profiling should be enabled for the etcd-backup-restore-sidecar
`etcdSnapshotTimeout` Kubernetes meta/v1.Duration	(Optional) EtcdSnapshotTimeout defines the timeout duration for etcd FullSnapshot operation
`leaderElection` LeaderElectionSpec	(Optional) LeaderElection defines parameters related to the LeaderElection configuration.

ClientService

(Appears on: EtcdConfig)

ClientService defines the parameters of the client service that a user can specify

Field	Description
`annotations` map[string]string	(Optional) Annotations specify the annotations that should be added to the client service
`labels` map[string]string	(Optional) Labels specify the labels that should be added to the client service

CompactionMode (`string` alias)

(Appears on: SharedConfig)

CompactionMode defines the auto-compaction-mode: ‘periodic’ or ‘revision’. ‘periodic’ for duration based retention and ‘revision’ for revision number based retention.

CompressionPolicy (`string` alias)

(Appears on: CompressionSpec)

CompressionPolicy defines the type of policy for compression of snapshots.

CompressionSpec

(Appears on: BackupSpec)

CompressionSpec defines parameters related to compression of Snapshots(full as well as delta).

Field	Description
`enabled` bool	(Optional)
`policy` CompressionPolicy	(Optional)

Condition

(Appears on: EtcdCopyBackupsTaskStatus, EtcdStatus)

Condition holds the information about the state of a resource.

Field	Description
`type` ConditionType	Type of the Etcd condition.
`status` ConditionStatus	Status of the condition, one of True, False, Unknown.
`lastTransitionTime` Kubernetes meta/v1.Time	Last time the condition transitioned from one status to another.
`lastUpdateTime` Kubernetes meta/v1.Time	Last time the condition was updated.
`reason` string	The reason for the condition’s last transition.
`message` string	A human-readable message indicating details about the transition.

ConditionStatus (`string` alias)

(Appears on: Condition)

ConditionStatus is the status of a condition.

ConditionType (`string` alias)

(Appears on: Condition)

ConditionType is the type of condition.

CrossVersionObjectReference

(Appears on: EtcdStatus)

CrossVersionObjectReference contains enough information to let you identify the referred resource.

Field	Description
`kind` string	Kind of the referent
`name` string	Name of the referent
`apiVersion` string	(Optional) API version of the referent

Etcd

Etcd is the Schema for the etcds API

Field Description

metadata
Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.

spec
EtcdSpec

`selector` Kubernetes meta/v1.LabelSelector	selector is a label query over pods that should match the replica count. It must match the pod template’s labels. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
`labels` map[string]string
`annotations` map[string]string	(Optional)
`etcd` EtcdConfig
`backup` BackupSpec
`sharedConfig` SharedConfig	(Optional)
`schedulingConstraints` SchedulingConstraints	(Optional)
`replicas` int32
`priorityClassName` string	(Optional) PriorityClassName is the name of a priority class that shall be used for the etcd pods.
`storageClass` string	(Optional) StorageClass defines the name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1
`storageCapacity` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) StorageCapacity defines the size of persistent volume.
`volumeClaimTemplate` string	(Optional) VolumeClaimTemplate defines the volume claim template to be created

status
EtcdStatus

EtcdConfig

(Appears on: EtcdSpec)

EtcdConfig defines parameters associated etcd deployed

Field	Description
`quota` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) Quota defines the etcd DB quota.
`defragmentationSchedule` string	(Optional) DefragmentationSchedule defines the cron standard schedule for defragmentation of etcd.
`serverPort` int32	(Optional)
`clientPort` int32	(Optional)
`image` string	(Optional) Image defines the etcd container image and tag
`authSecretRef` Kubernetes core/v1.SecretReference	(Optional)
`metrics` MetricsLevel	(Optional) Metrics defines the level of detail for exported metrics of etcd, specify ‘extensive’ to include histogram metrics.
`resources` Kubernetes core/v1.ResourceRequirements	(Optional) Resources defines the compute Resources required by etcd container. More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
`clientUrlTls` TLSConfig	(Optional) ClientUrlTLS contains the ca, server TLS and client TLS secrets for client communication to ETCD cluster
`peerUrlTls` TLSConfig	(Optional) PeerUrlTLS contains the ca and server TLS secrets for peer communication within ETCD cluster Currently, PeerUrlTLS does not require client TLS secrets for gardener implementation of ETCD cluster.
`etcdDefragTimeout` Kubernetes meta/v1.Duration	(Optional) EtcdDefragTimeout defines the timeout duration for etcd defrag call
`heartbeatDuration` Kubernetes meta/v1.Duration	(Optional) HeartbeatDuration defines the duration for members to send heartbeats. The default value is 10s.
`clientService` ClientService	(Optional) ClientService defines the parameters of the client service that a user can specify

EtcdCopyBackupsTask

EtcdCopyBackupsTask is a task for copying etcd backups from a source to a target store.

Field Description

metadata
Kubernetes meta/v1.ObjectMeta Refer to the Kubernetes API documentation for the fields of the metadata field.

spec
EtcdCopyBackupsTaskSpec

`sourceStore` StoreSpec	SourceStore defines the specification of the source object store provider for storing backups.
`targetStore` StoreSpec	TargetStore defines the specification of the target object store provider for storing backups.
`maxBackupAge` uint32	(Optional) MaxBackupAge is the maximum age in days that a backup must have in order to be copied. By default all backups will be copied.
`maxBackups` uint32	(Optional) MaxBackups is the maximum number of backups that will be copied starting with the most recent ones.
`waitForFinalSnapshot` WaitForFinalSnapshotSpec	(Optional) WaitForFinalSnapshot defines the parameters for waiting for a final full snapshot before copying backups.

status
EtcdCopyBackupsTaskStatus

EtcdCopyBackupsTaskSpec

(Appears on: EtcdCopyBackupsTask)

EtcdCopyBackupsTaskSpec defines the parameters for the copy backups task.

Field	Description
`sourceStore` StoreSpec	SourceStore defines the specification of the source object store provider for storing backups.
`targetStore` StoreSpec	TargetStore defines the specification of the target object store provider for storing backups.
`maxBackupAge` uint32	(Optional) MaxBackupAge is the maximum age in days that a backup must have in order to be copied. By default all backups will be copied.
`maxBackups` uint32	(Optional) MaxBackups is the maximum number of backups that will be copied starting with the most recent ones.
`waitForFinalSnapshot` WaitForFinalSnapshotSpec	(Optional) WaitForFinalSnapshot defines the parameters for waiting for a final full snapshot before copying backups.

EtcdCopyBackupsTaskStatus

(Appears on: EtcdCopyBackupsTask)

EtcdCopyBackupsTaskStatus defines the observed state of the copy backups task.

Field	Description
`conditions` []Condition	(Optional) Conditions represents the latest available observations of an object’s current state.
`observedGeneration` int64	(Optional) ObservedGeneration is the most recent generation observed for this resource.
`lastError` string	(Optional) LastError represents the last occurred error.

EtcdMemberConditionStatus (`string` alias)

(Appears on: EtcdMemberStatus)

EtcdMemberConditionStatus is the status of an etcd cluster member.

EtcdMemberStatus

(Appears on: EtcdStatus)

EtcdMemberStatus holds information about a etcd cluster membership.

Field	Description
`name` string	Name is the name of the etcd member. It is the name of the backing `Pod`.
`id` string	(Optional) ID is the ID of the etcd member.
`role` EtcdRole	(Optional) Role is the role in the etcd cluster, either `Leader` or `Member`.
`status` EtcdMemberConditionStatus	Status of the condition, one of True, False, Unknown.
`reason` string	The reason for the condition’s last transition.
`lastTransitionTime` Kubernetes meta/v1.Time	LastTransitionTime is the last time the condition’s status changed.

EtcdRole (`string` alias)

(Appears on: EtcdMemberStatus)

EtcdRole is the role of an etcd cluster member.

EtcdSpec

(Appears on: Etcd)

EtcdSpec defines the desired state of Etcd

Field	Description
`selector` Kubernetes meta/v1.LabelSelector	selector is a label query over pods that should match the replica count. It must match the pod template’s labels. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
`labels` map[string]string
`annotations` map[string]string	(Optional)
`etcd` EtcdConfig
`backup` BackupSpec
`sharedConfig` SharedConfig	(Optional)
`schedulingConstraints` SchedulingConstraints	(Optional)
`replicas` int32
`priorityClassName` string	(Optional) PriorityClassName is the name of a priority class that shall be used for the etcd pods.
`storageClass` string	(Optional) StorageClass defines the name of the StorageClass required by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1
`storageCapacity` k8s.io/apimachinery/pkg/api/resource.Quantity	(Optional) StorageCapacity defines the size of persistent volume.
`volumeClaimTemplate` string	(Optional) VolumeClaimTemplate defines the volume claim template to be created

EtcdStatus

(Appears on: Etcd)

EtcdStatus defines the observed state of Etcd.

Field	Description
`observedGeneration` int64	(Optional) ObservedGeneration is the most recent generation observed for this resource.
`etcd` CrossVersionObjectReference	(Optional)
`conditions` []Condition	(Optional) Conditions represents the latest available observations of an etcd’s current state.
`serviceName` string	(Optional) ServiceName is the name of the etcd service.
`lastError` string	(Optional) LastError represents the last occurred error.
`clusterSize` int32	(Optional) Cluster size is the size of the etcd cluster.
`currentReplicas` int32	(Optional) CurrentReplicas is the current replica count for the etcd cluster.
`replicas` int32	(Optional) Replicas is the replica count of the etcd resource.
`readyReplicas` int32	(Optional) ReadyReplicas is the count of replicas being ready in the etcd cluster.
`ready` bool	(Optional) Ready is `true` if all etcd replicas are ready.
`updatedReplicas` int32	(Optional) UpdatedReplicas is the count of updated replicas in the etcd cluster.
`labelSelector` Kubernetes meta/v1.LabelSelector	(Optional) LabelSelector is a label query over pods that should match the replica count. It must match the pod template’s labels.
`members` []EtcdMemberStatus	(Optional) Members represents the members of the etcd cluster
`peerUrlTLSEnabled` bool	(Optional) PeerUrlTLSEnabled captures the state of peer url TLS being enabled for the etcd member(s)

GarbageCollectionPolicy (`string` alias)

(Appears on: BackupSpec)

GarbageCollectionPolicy defines the type of policy for snapshot garbage collection.

LeaderElectionSpec

(Appears on: BackupSpec)

LeaderElectionSpec defines parameters related to the LeaderElection configuration.

Field	Description
`reelectionPeriod` Kubernetes meta/v1.Duration	(Optional) ReelectionPeriod defines the Period after which leadership status of corresponding etcd is checked.
`etcdConnectionTimeout` Kubernetes meta/v1.Duration	(Optional) EtcdConnectionTimeout defines the timeout duration for etcd client connection during leader election.

MetricsLevel (`string` alias)

(Appears on: EtcdConfig)

MetricsLevel defines the level ‘basic’ or ‘extensive’.

SchedulingConstraints

(Appears on: EtcdSpec)

Field	Description
`affinity` Kubernetes core/v1.Affinity	(Optional) Affinity defines the various affinity and anti-affinity rules for a pod that are honoured by the kube-scheduler.
`topologySpreadConstraints` []Kubernetes core/v1.TopologySpreadConstraint	(Optional) TopologySpreadConstraints describes how a group of pods ought to spread across topology domains, that are honoured by the kube-scheduler.

SecretReference

(Appears on: TLSConfig)

SecretReference defines a reference to a secret.

Field	Description
`SecretReference` Kubernetes core/v1.SecretReference	(Members of `SecretReference` are embedded into this type.)
`dataKey` string	(Optional) DataKey is the name of the key in the data map containing the credentials.

SharedConfig

(Appears on: EtcdSpec)

SharedConfig defines parameters shared and used by Etcd as well as backup-restore sidecar.

Field	Description
`autoCompactionMode` CompactionMode	(Optional) AutoCompactionMode defines the auto-compaction-mode:‘periodic’ mode or ‘revision’ mode for etcd and embedded-Etcd of backup-restore sidecar.
`autoCompactionRetention` string	(Optional) AutoCompactionRetention defines the auto-compaction-retention length for etcd as well as for embedded-Etcd of backup-restore sidecar.

StorageProvider (`string` alias)

(Appears on: StoreSpec)

StorageProvider defines the type of object store provider for storing backups.

StoreSpec

(Appears on: BackupSpec, EtcdCopyBackupsTaskSpec)

StoreSpec defines parameters related to ObjectStore persisting backups

Field	Description
`container` string	(Optional) Container is the name of the container the backup is stored at.
`prefix` string	Prefix is the prefix used for the store.
`provider` StorageProvider	(Optional) Provider is the name of the backup provider.
`secretRef` Kubernetes core/v1.SecretReference	(Optional) SecretRef is the reference to the secret which used to connect to the backup store.

TLSConfig

(Appears on: BackupSpec, EtcdConfig)

TLSConfig hold the TLS configuration details.

Field	Description
`tlsCASecretRef` SecretReference
`serverTLSSecretRef` Kubernetes core/v1.SecretReference
`clientTLSSecretRef` Kubernetes core/v1.SecretReference	(Optional)

WaitForFinalSnapshotSpec

(Appears on: EtcdCopyBackupsTaskSpec)

WaitForFinalSnapshotSpec defines the parameters for waiting for a final full snapshot before copying backups.

Field	Description
`enabled` bool	Enabled specifies whether to wait for a final full snapshot before copying backups.
`timeout` Kubernetes meta/v1.Duration	(Optional) Timeout is the timeout for waiting for a final full snapshot. When this timeout expires, the copying of backups will be performed anyway. No timeout or 0 means wait forever.

Generated with gen-crd-api-reference-docs

Etcd Druid

ETCD Druid

Background

Goals

Proposal

CRD specification

Implementation Agenda

Workflow

Deployment workflow

Defragmentation workflow

Local Setup

1 - API Reference

druid.gardener.cloud/v1alpha1

BackupSpec

ClientService

CompactionMode (string alias)

CompressionPolicy (string alias)

CompressionSpec

Condition

ConditionStatus (string alias)

ConditionType (string alias)

CrossVersionObjectReference

Etcd

EtcdConfig

EtcdCopyBackupsTask

EtcdCopyBackupsTaskSpec

EtcdCopyBackupsTaskStatus

EtcdMemberConditionStatus (string alias)

EtcdMemberStatus

EtcdRole (string alias)

EtcdSpec

EtcdStatus

GarbageCollectionPolicy (string alias)

LeaderElectionSpec

MetricsLevel (string alias)

SchedulingConstraints

SecretReference

SharedConfig

StorageProvider (string alias)

StoreSpec

TLSConfig

WaitForFinalSnapshotSpec

2 - Druid

druid.gardener.cloud/v1alpha1

BackupSpec

ClientService

CompactionMode (string alias)

CompressionPolicy (string alias)

CompressionSpec

Condition

ConditionStatus (string alias)

ConditionType (string alias)

CrossVersionObjectReference

Etcd

EtcdConfig

EtcdCopyBackupsTask

EtcdCopyBackupsTaskSpec

EtcdCopyBackupsTaskStatus

EtcdMemberConditionStatus (string alias)

EtcdMemberStatus

EtcdRole (string alias)

EtcdSpec

EtcdStatus

GarbageCollectionPolicy (string alias)

LeaderElectionSpec

MetricsLevel (string alias)

SchedulingConstraints

SecretReference

SharedConfig

StorageProvider (string alias)

StoreSpec

TLSConfig

WaitForFinalSnapshotSpec

CompactionMode (`string` alias)

CompressionPolicy (`string` alias)

ConditionStatus (`string` alias)

ConditionType (`string` alias)

EtcdMemberConditionStatus (`string` alias)

EtcdRole (`string` alias)

GarbageCollectionPolicy (`string` alias)

MetricsLevel (`string` alias)

StorageProvider (`string` alias)

CompactionMode (`string` alias)

CompressionPolicy (`string` alias)

ConditionStatus (`string` alias)

ConditionType (`string` alias)

EtcdMemberConditionStatus (`string` alias)

EtcdRole (`string` alias)

GarbageCollectionPolicy (`string` alias)

MetricsLevel (`string` alias)

StorageProvider (`string` alias)