Project

1 - NamespacedCloudProfiles

NamespacedCloudProfiles

NamespacedCloudProfiles are resources in Gardener that allow project-level customization of CloudProfiles. They enable project administrators to create and manage cloud profiles specific to their projects and reduce the operational burden on central Gardener operators. As opposed to CloudProfiles, NamespacedCloudProfiles are namespaced and thus limit configuration options for Shoots, such as special machine types, to the associated project only. These profiles inherit from a parent CloudProfile and can override or extend certain fields while maintaining backward compatibility.

Project viewers have the permission to see NamespacedCloudProfiles associated with a particular project. Project administrators can generally create, edit, or delete NamespacedCloudProfiles but with some exceptions (see the restrictions outlined below).

When creating or updating a Shoot, the cloud profile reference can be set to point to a NamespacedCloudProfile, allowing for more granular and project-specific configurations. The modification of a Shoot’s cloud profile reference is restricted to switching within the same profile hierarchy, i.e. from a CloudProfile to a descendant NamespacedCloudProfile, from a NamespacedCloudProfile to its parent CloudProfile and between NamespacedCloudProfiles having the same CloudProfile parent. Changing the reference from one CloudProfile or descendant NamespacedCloudProfile to another CloudProfile or descendant NamespacedCloudProfile is not allowed.

The usage of NamespacedCloudProfiles is currently subject to an alpha feature gate and is not enabled by default. It requires the enabled provider extensions to support the feature as well. The feature gate can be enabled by passing the --feature-gates=NamespacedCloudProfiles=true flag to the Gardener API server.

Please see this example manifest and GEP-25 for additional information.

Field Modification Restrictions

In order to make changes to specific fields in the NamespacedCloudProfile, a user must be granted custom RBAC verbs. Modifications of these fields need to be performed with caution and might require additional validation steps or accompanying changes. By default, only landscape operators have the permission to change these fields, as they are usually able to judge the implications.

Changing the following fields require the corresponding custom verbs:

• For changing the .spec.kubernetes field, the custom verb modify-spec-kubernetes is required.
• For changing the .spec.machineImages field, the custom verb modify-spec-machineimages is required.
• For changing the .spec.providerConfig field, the custom verb modify-spec-providerconfig is required.

The assignment of these custom verbs can be achieved by creating a ClusterRole and a RoleBinding like in the following example:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: namespacedcloudprofile-kubernetes
rules:
- apiGroups: ["core.gardener.cloud"] 
  resources: ["namespacedcloudprofiles"]
  verbs: ["modify-spec-kubernetes"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: edit-kubernetes
  namespace: dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: namespacedcloudprofile-kubernetes
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: alice.doe@example.com

2 - Projects

Projects

The Gardener API server supports a cluster-scoped Project resource which is used for data isolation between individual Gardener consumers. For example, each development team has its own project to manage its own shoot clusters.

Each Project is backed by a Kubernetes Namespace that contains the actual related Kubernetes resources, like Secrets or Shoots.

Example resource:

apiVersion: core.gardener.cloud/v1beta1
kind: Project
metadata:
  name: dev
spec:
  namespace: garden-dev
  description: "This is my first project"
  purpose: "Experimenting with Gardener"
  owner:
    apiGroup: rbac.authorization.k8s.io
    kind: User
    name: john.doe@example.com
  members:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: alice.doe@example.com
    role: admin
  # roles:
  # - viewer 
  # - uam
  # - serviceaccountmanager
  # - extension:foo
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: bob.doe@example.com
    role: viewer
# tolerations:
#   defaults:
#   - key: <some-key>
#   whitelist:
#   - key: <some-key>

The .spec.namespace field is optional and is initialized if unset. The name of the resulting namespace will be determined based on the Project name and UID, e.g., garden-dev-5aef3. It’s also possible to adopt existing namespaces by labeling them gardener.cloud/role=project and project.gardener.cloud/name=dev beforehand (otherwise, they cannot be adopted).

When deleting a Project resource, the corresponding namespace is also deleted. To keep a namespace after project deletion, an administrator/operator (not Project members!) can annotate the project-namespace with namespace.gardener.cloud/keep-after-project-deletion.

The spec.description and .spec.purpose fields can be used to describe to fellow team members and Gardener operators what this project is used for.

Each project has one dedicated owner, configured in .spec.owner using the rbac.authorization.k8s.io/v1.Subject type. The owner is the main contact person for Gardener operators. Please note that the .spec.owner field is deprecated and will be removed in future API versions in favor of the owner role, see below.

The list of members (again a list in .spec.members[] using the rbac.authorization.k8s.io/v1.Subject type) contains all the people that are associated with the project in any way. Each project member must have at least one role (currently described in .spec.members[].role, additional roles can be added to .spec.members[].roles[]). The following roles exist:

• admin: This allows to fully manage resources inside the project (e.g., secrets, shoots, configmaps, and similar). Mind that the admin role has read only access to service accounts.
• serviceaccountmanager: This allows to fully manage service accounts inside the project namespace and request tokens for them. The permissions of the created service accounts are instead managed by the admin role. Please refer to Service Account Manager.
• uam: This allows to add/modify/remove human users or groups to/from the project member list.
• viewer: This allows to read all resources inside the project except secrets.
• owner: This combines the admin, uam, and serviceaccountmanager roles.
• Extension roles (prefixed with extension:): Please refer to Extending Project Roles.

The project controller inside the Gardener Controller Manager is managing RBAC resources that grant the described privileges to the respective members.

There are three central ClusterRoles gardener.cloud:system:project-member, gardener.cloud:system:project-viewer, and gardener.cloud:system:project-serviceaccountmanager that grant the permissions for namespaced resources (e.g., Secrets, Shoots, ServiceAccounts). Via referring RoleBindings created in the respective namespace the project members get bound to these ClusterRoles and, thus, the needed permissions. There are also project-specific ClusterRoles granting the permissions for cluster-scoped resources, e.g., the Namespace or Project itself.
For each role, the following ClusterRoles, ClusterRoleBindings, and RoleBindings are created:

User Access Management

For Projects created before Gardener v1.8, all admins were allowed to manage other members. Beginning with v1.8, the new uam role is being introduced. It is backed by the manage-members custom RBAC verb which allows to add/modify/remove human users or groups to/from the project member list. Human users are subjects with kind=User and name!=system:serviceaccount:*, and groups are subjects with kind=Group. The management of service account subjects (kind=ServiceAccount or name=system:serviceaccount:*) is not controlled via the uam custom verb but with the standard update/patch verbs for projects.

All newly created projects will only bind the owner to the uam role. The owner can still grant the uam role to other members if desired. For projects created before Gardener v1.8, the Gardener Controller Manager will migrate all projects to also assign the uam role to all admin members (to not break existing use-cases). The corresponding migration logic is present in Gardener Controller Manager from v1.8 to v1.13. The project owner can gradually remove these roles if desired.

Stale Projects

When a project is not actively used for some period of time, it is marked as “stale”. This is done by a controller called “Stale Projects Reconciler”. Once the project is marked as stale, there is a time frame in which if not used it will be deleted by that controller.

Four-Eyes-Principle For Resource Deletion

In order to delete a Shoot, the deletion must be confirmed upfront with the confirmation.gardener.cloud/deletion=true annotation. Without this annotation being set, gardener-apiserver denies any DELETE request. Still, users sometimes accidentally shot themselves in the foot, meaning that they accidentally deleted a Shoot despite the confirmation requirement.

To prevent that (or make it harder, at least), the Project can be configured to apply the dual approval concept for Shoot deletion. This means that the subject confirming the deletion must not be the same as the subject sending the DELETE request.

Example:

spec:
  dualApprovalForDeletion:
  - resource: shoots
    selector:
      matchLabels: {}
    includeServiceAccounts: true

ℹ️ Note

As of today, core.gardener.cloud/v1beta1.Shoot is the only resource for which this concept is implemented.

As usual, .spec.dualApprovalForDeletion[].selector.matchLabels={} matches all resources, .spec.dualApprovalForDeletion[].selector.matchLabels=null matches none at all. It can also be decided to specify an individual label selector if this concept shall only apply to a subset of the Shoots in the project (e.g., CI/development clusters shall be excluded).

The includeServiceAccounts (default: true) controls whether the concept also applies when the Shoot deletion confirmation and actual deletion is triggered via ServiceAccounts. This is to prevent that CI jobs have to follow this concept as well, adding additional complexity/overhead. Alternatively, you could also use two ServiceAccounts, one for confirming the deletion, and another one for actually sending the DELETE request, if desired.

❗ Important

Project members can still change the labels of Shoots (or the selector itself) to circumvent the dual approval concept. This concern is intentionally excluded/ignored for now since the principle is not a “security feature” but shall just help preventing accidental deletion.

3 - Service Account Manager

Service Account Manager

Overview

With Gardener v1.47, a new role called serviceaccountmanager was introduced. This role allows to fully manage ServiceAccount’s in the project namespace and request tokens for them. This is the preferred way of managing the access to a project namespace, as it aims to replace the usage of the default ServiceAccount secrets that will no longer be generated automatically.

Actions

Once assigned the serviceaccountmanager role, a user can create/update/delete ServiceAccounts in the project namespace.

Create a Service Account

In order to create a ServiceAccount named “robot-user”, run the following kubectl command:

kubectl -n project-abc create sa robot-user

Request a Token for a Service Account

A token for the “robot-user” ServiceAccount can be requested via the TokenRequest API in several ways:

kubectl -n project-abc create token robot-user --duration=3600s

• directly calling the Kubernetes HTTP API

curl -X POST https://api.gardener/api/v1/namespaces/project-abc/serviceaccounts/robot-user/token \
    -H "Authorization: Bearer <auth-token>" \
    -H "Content-Type: application/json" \
    -d '{
        "apiVersion": "authentication.k8s.io/v1",
        "kind": "TokenRequest",
        "spec": {
          "expirationSeconds": 3600
        }
      }'

Mind that the returned token is not stored within the Kubernetes cluster, will be valid for 3600 seconds, and will be invalidated if the “robot-user” ServiceAccount is deleted. Although expirationSeconds can be modified depending on the needs, the returned token’s validity will not exceed the configured service-account-max-token-expiration duration for the garden cluster. It is advised that the actual expirationTimestamp is verified so that expectations are met. This can be done by asserting the expirationTimestamp in the TokenRequestStatus or the exp claim in the token itself.

Delete a Service Account

In order to delete the ServiceAccount named “robot-user”, run the following kubectl command:

kubectl -n project-abc delete sa robot-user

This will invalidate all existing tokens for the “robot-user” ServiceAccount.