11 minute read  

Releases, Features, Hotfixes

This document describes how to contribute features or hotfixes, and how new Gardener releases are usually scheduled, validated, etc.

• Releases, Features, Hotfixes
  • Releases
    • Release Responsible Plan
    • Release Validation
  • Contributing New Features or Fixes
    • TODO Statements
    • Deprecations and Backwards-Compatibility
  • Cherry Picks
    • Prerequisites
    • Initiate a Cherry Pick

Releases

The @gardener-maintainers are trying to provide a new release roughly every other week (depending on their capacity and the stability/robustness of the master branch).

Hotfixes are usually maintained for the latest three minor releases, though, there are no fixed release dates.

Release Responsible Plan

Apart from the release of the next version, the release responsible is also taking care of potential hotfix releases of the last three minor versions. The release responsible is the main contact person for coordinating new feature PRs for the next minor versions or cherry-pick PRs for the last three minor versions.

Release Validation

The release phase for a new minor version lasts two weeks. Typically, the first week is used for the validation of the release. This phase includes the following steps:

1. master (or latest release-* branch) is deployed to a development landscape that already hosts some existing seed and shoot clusters.
2. An extended test suite is triggered by the “release responsible” which:
   1. executes the Gardener integration tests for different Kubernetes versions, infrastructures, and Shoot settings.
   2. executes the Kubernetes conformance tests.
   3. executes further tests like Kubernetes/OS patch/minor version upgrades.
3. Additionally, every four hours (or on demand) more tests (e.g., including the Kubernetes e2e test suite) are executed for different infrastructures.
4. The “release responsible” is verifying new features or other notable changes (derived of the draft release notes) in this development system.

Usually, the new release is triggered in the beginning of the second week if all tests are green, all checks were successful, and if all of the planned verifications were performed by the release responsible.

Contributing New Features or Fixes

Please refer to the Gardener contributor guide. Besides a lot of general information, it also provides a checklist for newly created pull requests that may help you to prepare your changes for an efficient review process. If you are contributing a fix or major improvement, please take care to open cherry-pick PRs to all affected and still supported versions once the change is approved and merged in the master branch.

⚠️ Please ensure that your modifications pass the verification checks (linting, formatting, static code checks, tests, etc.) by executing

make verify

before filing your pull request.

The guide applies for both changes to the master and to any release-* branch. All changes must be submitted via a pull request and be reviewed and approved by at least one code owner.

TODO Statements

Sometimes, TODO statements are being introduced when one cannot follow up immediately with certain tasks or when temporary migration code is required. In order to properly follow-up with such TODOs and to prevent them from piling up without getting attention, the following rules should be followed:

• Each TODO statement should have an associated person and state when it can be removed. Example:

  // TODO(<github-username>): Remove this code after v1.75 has been released.
  

• When the task depends on a certain implementation, a GitHub issue should be opened and referenced in the statement. Example:

  // TODO(<github-username>): Remove this code after https://github.com/gardener/gardener/issues/<issue-number> has been implemented.
  

  The associated person should actively drive the implementation of the referenced issue (unless it cannot be done because of third-party dependencies or conditions) so that the TODO statement does not get stale.
• TODO statements without actionable tasks or those that are unlikely to ever be implemented (maybe because of very low priorities) should not be specified in the first place. If a TODO is specified, the associated person should make sure to actively follow-up.

Deprecations and Backwards-Compatibility

In case you have to remove functionality relevant to end-users (e.g., a field or default value in the Shoot API), please connect it with a Kubernetes minor version upgrade. This way, end-users are forced to actively adapt their manifests when they perform their Kubernetes upgrades. For example, the .spec.kubernetes.enableStaticTokenKubeconfig field in the Shoot API is no longer allowed to be set for Kubernetes versions >= 1.27.

In case you have to remove or change functionality which cannot be directly connected with a Kubernetes version upgrade, please consider introducing a feature gate. This way, landscape operators can announce the planned changes to their users and communicate a timeline when they plan to activate the feature gate. End-users can then prepare for it accordingly. For example, the fact that changes to kubelet.kubeReserved in the Shoot API will lead to a rolling update of the worker nodes (previously, these changes were updated in-place) is controlled via the NewWorkerPoolHash feature gate.

In case you have to remove functionality relevant to Gardener extensions, please deprecate it first, and add a TODO statement to remove it only after at least 9 releases. Do not forget to write a proper release note as part of your pull request. This gives extension developers enough time (~18 weeks) to adapt to the changes (and to release a new version of their extension) before Gardener finally removes the functionality. Examples are removing a field in the extensions.gardener.cloud/v1alpha1 API group, or removing a controller in the extensions library.

In case you have to run migration code (which is mostly internal), please add a TODO statement to remove it only after 3 releases. This way, we can ensure that the Gardener version skew policy is not violated. For example, the migration code for moving the Prometheus instances under management of prometheus-operator was running for three releases.

Tip

Please revisit the version skew policy.

Cherry Picks

This section explains how to initiate cherry picks on release branches within the gardener/gardener repository.

• Prerequisites
• Initiate a Cherry Pick

Prerequisites

Before you initiate a cherry pick, make sure that the following prerequisites are accomplished.

• A pull request merged against the master branch.
• The release branch exists (check in the branches section).
• Have the gardener/gardener repository cloned as follows:
  • the origin remote should point to your fork (alternatively this can be overwritten by passing FORK_REMOTE=<fork-remote>).
  • the upstream remote should point to the Gardener GitHub org (alternatively this can be overwritten by passing UPSTREAM_REMOTE=<upstream-remote>).
• Have hub installed, which is most easily installed via go get github.com/github/hub assuming you have a standard golang development environment.
• A GitHub token which has permissions to create a PR in an upstream branch.

Initiate a Cherry Pick

• Run the [cherry pick script][cherry-pick-script].

  This example applies a master branch PR #3632 to the remote branch upstream/release-v3.14:

  GITHUB_USER=<your-user> hack/cherry-pick-pull.sh upstream/release-v3.14 3632
  

  • Be aware the cherry pick script assumes you have a git remote called upstream that points at the Gardener GitHub org.

  • You will need to run the cherry pick script separately for each patch release you want to cherry pick to. Cherry picks should be applied to all active release branches where the fix is applicable.

  • When asked for your GitHub password, provide the created GitHub token rather than your actual GitHub password. Refer https://github.com/github/hub/issues/2655#issuecomment-735836048

• cherry-pick-script