Logging Stack

This document contains logging stack related how-tos and configuration options for developers.

Expose Logs for Component to User Plutono

Exposing logs for a new component to the User's Plutono is described in the How to Expose Logs to the Users section.

Configuration

Fluent-bit

The Fluent-bit configurations can be found on pkg/component/observability/logging/fluentoperator/customresources There are six different specifications:

• FluentBit: Defines the fluent-bit DaemonSet specifications
• ClusterFluentBitConfig: Defines the labelselectors of the resources which fluent-bit will match
• ClusterInput: Defines the location of the input stream of the logs
• ClusterOutput: Defines the location of the output source (Vali for example)
• ClusterFilter: Defines filters which match specific keys
• ClusterParser: Defines parsers which are used by the filters

Vali

The Vali configurations can be found on charts/seed-bootstrap/charts/vali/templates/vali-configmap.yaml

The main specifications there are:

• Index configuration: Currently the following one is used:

    schema_config:
      configs:
      - from: 2018-04-15
        store: boltdb
        object_store: filesystem
        schema: v11
        index:
          prefix: index_
          period: 24h

• from: Is the date from which logs collection is started. Using a date in the past is okay.
• store: The DB used for storing the index.
• object_store: Where the data is stored.
• schema: Schema version which should be used (v11 is currently recommended).
• index.prefix: The prefix for the index.
• index.period: The period for updating the indices.

Adding a new index happens with new config block definition. The from field should start from the current day + previous index.period and should not overlap with the current index. The prefix also should be different.

    schema_config:
      configs:
      - from: 2018-04-15
        store: boltdb
        object_store: filesystem
        schema: v11
        index:
          prefix: index_
          period: 24h
      - from: 2020-06-18
        store: boltdb
        object_store: filesystem
        schema: v11
        index:
          prefix: index_new_
          period: 24h

• chunk_store_config Configuration

    chunk_store_config:
      max_look_back_period: 336h

chunk_store_config.max_look_back_period should be the same as the retention_period

• table_manager Configuration

    table_manager:
      retention_deletes_enabled: true
      retention_period: 336h

table_manager.retention_period is the living time for each log message. Vali will keep messages for (table_manager.retention_period - index.period) time due to specification in the Vali implementation.

Plutono

This is the Vali configuration that Plutono uses:

    - name: vali
      type: vali
      access: proxy
      url: http://logging.{{ .Release.Namespace }}.svc:3100
      jsonData:
        maxLines: 5000

• name: Is the name of the datasource.
• type: Is the type of the datasource.
• access: Should be set to proxy.
• url: Vali's url
• svc: Vali's port
• jsonData.maxLines: The limit of the log messages which Plutono will show to the users.

Decrease this value if the browser works slowly!