3 minute read
Configuring the Logging stack via Gardenlet configurations
Enable the Logging
In order to install the Gardener logging stack the logging.enabled
configuration option has to be enabled in the Gardenlet configuration:
logging:
enabled: true
From now on each Seed is going to have a logging stack which will collect logs from all pods and some systemd services. Logs related to Shoots with testing
purpose are dropped in the fluent-bit
output plugin. Shoots with a purpose different than testing
have the same type of log aggregator (but different instance) as the Seed. The logs can be viewed in the Grafana in the garden
namespace for the Seed components and in the respective shoot control plane namespaces.
Enable logs from the Shoot’s node systemd services.
The logs from the systemd services on each node can be retrieved by enabling the logging.shootNodeLogging
option in the Gardenlet configuration:
logging:
enabled: true
shootNodeLogging:
shootPurposes:
- "evaluation"
- "deployment"
Under the shootPurpose
section just list all the shoot purposes for which the Shoot node logging feature will be enabled. Specifying the testing
purpose has no effect because this purpose prevents the logging stack installation.
Logs can be viewed in the operator Grafana!
The dedicated labels are unit
, syslog_identifier
and nodename
in the Explore
menu.
Configuring the log processor
Under logging.fluentBit
there is three optional sections.
input
: This overwrite the input configuration of the fluent-bit log processor.output
: This overwrite the output configuration of the fluent-bit log processor.service
: This overwrite the service configuration of the fluent-bit log processor.
logging:
enabled: true
fluentBit:
output: |-
[Output]
...
input: |-
[Input]
...
service: |-
[Service]
...
Configuring the Loki PriorityClass
The central Loki, which is in the garden
namespace, contains all the logs from the most important seed components. When the central Loki PriorityClass
is with low value then its pods can be preempted and often moved from one node to another while Kubernetes tries to free space for more important pods. The persistent volume will be detached/attached again as well. Based on the performance of the underlying infrastructure, this leads to great central Loki downtime. To give greater priority of the seed Loki you can use the logging.loki.garden.priority
option.
logging:
enabled: true
loki:
garden:
priority: 100
Configure central logging
For central logging, the output configuration of the fluent-bit log processor can be overwritten (logging.fluentBit.output
) and the Loki instances deployments in Garden and Shoot namespace can be enabled/disabled (logging.loki.enabled
), by default Loki is enabled.
logging:
enabled: true
fluentBit:
output: |-
[Output]
...
loki:
enabled: false
Configuring central Loki storage capacity
By default, the central Loki has 100Gi
of storage capacity.
To overwrite the current central Loki storage capacity, the logging.loki.garden.storage
setting in the gardenlet’s component configuration should be altered.
If you need to increase it you can do so without losing the current data by specifying higher capacity. Doing so, the Loki’s PersistentVolume
capacity will be increased instead of deleting the current PV.
However, if you specify less capacity then the PersistentVolume
will be deleted and with it the logs, too.
logging:
enabled: true
fluentBit:
output: |-
[Output]
...
loki:
garden:
storage: "200Gi"
Feedback
Was this page helpful?