Abstract

Documentation for the Prometheus Helm Chart values.

Prometheus Helm Chart Values

This folder contains a values file to deploy the Prometheus Community Helm Chart along with a JSON schema file generated from that values file.

This is a copy

Any documentation here was originally sourced from the Prometheus Community Helm Chart’s ArtifactHub. In the event of conflicts between this information and the source repository, the source repository should be considered the truth.

Schema Generation

If you need to update the JSON schema, follow these steps.

1. Make sure the Helm schema-gen plugin is installed.

   helm plugin install https://github.com/karuppiah7890/helm-schema-gen
   

2. Use the plugin to generate the new schema.

   helm schema-gen values.yaml
   

Usage

1. Add the repository to Helm.

   helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
   helm repo update
   

2. Edit the values.yaml as you need.

3. Deploy the chart.

   kubectl create ns prometheus
   helm -n prometheus upgrade --install prometheus prometheus-community/prometheus -f values.yaml
   

Values

imagePullSecrets

Prometheus Values

---

Default values for prometheus. This is a YAML-formatted file. Declare variables to be passed into your templates.

RBAC and PSP

---

Role based access control and pod security policy.

example pull secrets

imagePullSecrets:
- name: "image-pull-secret"

imagePullSecrets: []
podSecurityPolicy:
  enabled: false
rbac:
  create: true

serviceAccountsServer

Service Account

---

Define serviceAccount names for components. Defaults to component’s fully qualified name.

Token Automount

Opt out of automounting Kubernetes API credentials.

It will be overriden by server.automountServiceAccountToken value, if set.

automount token

automountServiceAccountToken: false

commonMetaLabels

Additional labels to attach to all resources

serviceAccountsServer:
  create: true
  name: prometheus
  annotations: {}
commonMetaLabels: {}

configmapReload

ConfigMap Reload

---

Monitors ConfigMap changes and POSTs to a URL. More information is available here.

env

env sets environment variables to pass to the container. Can be set as name/value pairs, read from secrets or configmaps.

env var example

env:
- name: SOMEVAR
  value: somevalue
- name: PASSWORD
  valueFrom:
    secretKeyRef:
      name: mysecret
      key: password
      optional: false

reloadUrl

URL for configmap-reload to use for reloads

extraVolumeMounts example

- name: prometheus-alerts
  mountPath: /etc/alerts.d
  subPath: ""
  configMap: prometheus-alerts
  readOnly: true

configmapReload:
  reloadUrl: ""
  env: []
  prometheus:
    enabled: true
    name: configmap-reload
    # configmap-reload container image
    image:
      repository: quay.io/prometheus-operator/prometheus-config-reloader
      tag: v0.76.0
      digest: ""
      pullPolicy: IfNotPresent
    containerPort: 8080
    containerPortName: metrics
    extraArgs: {}
    extraVolumeDirs: []
    extraVolumeMounts: []
    extraConfigmapMounts: []
    containerSecurityContext: {}
    livenessProbe:
      httpGet:
        path: /healthz
        port: metrics
        scheme: HTTP
      periodSeconds: 10
      initialDelaySeconds: 2
    readinessProbe:
      httpGet:
        path: /healthz
        port: metrics
        scheme: HTTP
      periodSeconds: 10
    startupProbe:
      enabled: false
      httpGet:
        path: /healthz
        port: metrics
        scheme: HTTP
      periodSeconds: 10
    resources: {}

server

Prometheus Server

---

Configuration for the Prometheus server.

releaseNamespace

Enable only the release namespace for monitoring. By default all namespaces are monitored. If releaseNamespace and namespaces are both set a merged list will be monitored.

SA Token Auto Mount

Opt out of automounting Kubernetes API credentials.

If set it will override serviceAccountsServer.automountServiceAccountToken value for ServiceAccount.

automountServiceAccountToken: false

ClusterRole/Binding

Use a ClusterRole (and ClusterRoleBinding)

• If set to false

  • we define a RoleBinding in the defined namespaces ONLY

Note

because we need a Role with nonResourceURL’s (“/metrics”) you must get someone with Cluster-admin privileges to define this role for you, before running with this setting enabled.

This makes prometheus work for users who do not have ClusterAdmin privs, but wants prometheus to operate on their own namespaces, instead of clusterwide.

You MUST also set namespaces to the ones you have access to and want monitored by Prometheus.

useExistingClusterRoleName: nameofclusterrole

If set it will override prometheus.server.fullname value for ClusterRole and ClusterRoleBinding

Namespaces

namespaces to monitor (instead of monitoring all clusterwide).

Needed if you want to run without Cluster-admin privileges.

namespaces:
  - yournamespace

sidecarContainers

add more containers to prometheus server

Key/Value where Key is the sidecar - name: <Key>

Example

  sidecarContainers:
     webServer:
       image: nginx

OR for adding OAuth authentication to Prometheus

OAuth Example

  sidecarContainers:
    oauth-proxy:
      image: quay.io/oauth2-proxy/oauth2-proxy:v7.1.2
      args:
      - --upstream=http://127.0.0.1:9090
      - --http-address=0.0.0.0:8081
      - ...
      ports:
      - containerPort: 8081
        name: oauth-proxy
        protocol: TCP
      resources: {}

sidecarTemplateValues - context to be used in template for sidecarContainers Example: sidecarTemplateValues: *your-custom-globals sidecarContainers: webserver: |- Template for webserver-container-template might looks like this: image: “:” …

EnableServiceLinks

indicates whether information about services should be injected into pod’s environment variables, matching the syntax of Docker links.

Warning

the field is unsupported and will be skipped in K8s prior to v1.13.0.

prefixURL

The URL prefix at which the container can be accessed. Useful in the case the `-web.external-url`` includes a slug so that the various internal URLs are still able to access as they are in the default case. (Optional) External URL which can access prometheus Maybe same with Ingress host name

env

Additional server container environment variables

You specify this manually like you would a raw deployment manifest. This means you can bind in environment variables from secrets.

e.g. static environment variable

 - name: DEMO_GREETING
   value: "Hello from the environment"

e.g. secret environment variable

- name: USERNAME
  valueFrom:
    secretKeyRef:
      name: mysecret
      key: username

defaultFlagsOverride

List of flags to override default parameters, e.g:

defaultFlagsOverride example

defaultFlagsOverride:
- --enable-feature=agent
- --storage.agent.retention.max-time=30m
- --config.file=/etc/config/prometheus.yml

extraFlags

extra flagss to pass to prometheus

• web.enable-admin-api flag controls access to the administrative HTTP API which includes functionality such as deleting time series. This is disabled by default.

• storage.tsdb.no-lockfile flag controls BD locking

• storage.tsdb.wal-compression flag enables compression of the write-ahead log (WAL)

extraFlags example

extraFlags:
- web.enable-admin-api
- storage.tsdb.no-lockfile
- storage.tsdb.wal-compression

probeHeaders

Custom HTTP headers for Liveness/Readiness/Startup Probe

Useful for providing HTTP Basic Auth to healthchecks

probeHeaders example

probeHeaders:
- name: "Authorization"
  value: "Bearer ABCDEabcde12345"

Anti Affinity

Pod anti-affinity can prevent the scheduler from placing Prometheus server replicas on the same node. The value “soft” means that the scheduler should prefer to not schedule two replica pods onto the same node but no guarantee is provided.

The value “hard” means that the scheduler is required to not schedule two replica pods onto the same node.

The default value “” will disable pod anti-affinity so that no anti-affinity rules will be configured (unless set in server.affinity).j

server:
  name: prometheus
  releaseNamespace: false
  sidecarContainers: {}
  sidecarTemplateValues: {}
  image:
    repository: quay.io/prometheus/prometheus
    # if not set appVersion field from
    # Chart.yaml is used
    tag: ""
    digest: ""
    pullPolicy: IfNotPresent
  command: []
  priorityClassName: ""
  enableServiceLinks: true
  prefixURL: ""
  baseURL: "prometheus.breeze-blocks.net"
  env: []
  extraFlags:
  - web.enable-lifecycle
  - web.enable-admin-api
  - web.external-url=prometheus.breeze-blocks.net
  - storage.tsdb.no-lockfile
  - storage.tsdb.wal-compression
  configPath: /etc/config/prometheus.yml
  storagePath: ""
  global:
    scrape_interval: 1m
    scrape_timeout: 10s
    evaluation_interval: 1m
  remoteWrite: []
  remoteRead: []
  tsdb: {}
  exemplars: {}
  probeHeaders: []
  extraArgs: {}
  extraInitContainers: []
  extraVolumeMounts: []
  extraVolumes: []
  extraHostPathMounts: []
  extraConfigmapMounts: []
  extraSecretMounts: []
  fullnameOverride: prometheus
  ingress:
    enabled: true
    ingressClassName: nginx
    annotations:
      nginx.ingress.kubernetes.io/default-backend: prometheus
    extraLabels: {}
    hosts:
    - 'prometheus.breeze-blocks.net'
    path: /
    pathType: Prefix
    tls: []
  strategy:
    type: Recreate
  hostAliases: []
  tolerations: []
  nodeSelector:
    kubernetes.io/hostname: k8s08.breeze-blocks.net
  affinity: {}
  podAntiAffinity: ""
  podAntiAffinityTopologyKey: kubernetes.io/hostname
  topologySpreadConstraints: []
  podDisruptionBudget:
    enabled: false
    maxUnavailable: 1
  persistentVolume:
    enabled: true
    statefulSetNameOverride: prometheus
    accessModes:
    - ReadWriteOnce
    labels: {}
    annotations: {}
    mountPath: /data
    size: 50Gi
    storageClass: csi-lvm-linear
    volumeBindingMode: Filesystem
    subPath: ""
  emptyDir:
    sizeLimit: ""
  podAnnotations: {}
  podLabels: {}
  alertmanagers: []
  podSecurityPolicy:
    annotations: {}
  replicaCount: 1
  revisionHistoryLimit: 10
  configMapAnnotations: {}
  deploymentAnnotations: {}
  statefulSet:
    enabled: true
    annotations: {}
    labels: {}
    podManagementPolicy: OrderedReady
    # Alertmanager headless service to use for the statefulset
    headless:
      annotations: {}
      labels: {}
      servicePort: 80
      gRPC:
        enabled: true
        servicePort: 9091
    pvcDeleteOnStsDelete: false
    pvcDeleteOnStsScale: false
  tcpSocketProbeEnabled: false
  probeScheme: HTTP
  readinessProbeInitialDelay: 30
  readinessProbePeriodSeconds: 5
  readinessProbeTimeout: 4
  readinessProbeFailureThreshold: 3
  readinessProbeSuccessThreshold: 1
  livenessProbeInitialDelay: 30
  livenessProbePeriodSeconds: 15
  livenessProbeTimeout: 10
  livenessProbeFailureThreshold: 3
  livenessProbeSuccessThreshold: 1
  startupProbe:
    enabled: false
    periodSeconds: 5
    failureThreshold: 30
    timeoutSeconds: 10
  resources:
    limits:
      cpu: 2
      memory: 2048Mi
    requests:
      cpu: 1
      memory: 512Mi
  hostNetwork: false
  dnsPolicy: ClusterFirst
  portName: ""
  securityContext:
    runAsUser: 0
    runAsNonRoot: false
    runAsGroup: 0
    fsGroup: 0
  containerSecurityContext: {}
  service:
    enabled: true
    annotations: {}
    labels: {}
    clusterIP: ""
    externalIPs:
    - 192.168.5.18
    loadBalancerIP: 192.168.5.5
    loadBalancerSourceRanges:
    - '192.168.5.0/24'
    - '127.0.0.0/8'
    servicePort: 80
    sessionAffinity: None
    type: LoadBalancer
    gRPC:
      enabled: true
      servicePort: 9091
    statefulsetReplica:
      enabled: true
      replica: 1
    additionalPorts:
    - name: authenticated
      port: 8081
      targetPort: 8081
  terminationGracePeriodSeconds: 300
  retention: "15d"

ruleFiles

Prometheus Config

---

Prometheus server ConfigMap entries for rule files (allow prometheus labels interpolation)

Prometheus server ConfigMap entries for scrape_config_files (allows scrape configs defined in additional files)

ruleFiles: {}
scrapeConfigFiles: []

serverFiles

Server ConfigMaps

---

Prometheus server ConfigMap entries

serverFiles:
  alerting_rules.yml: {}
  recording_rules.yml: {}
  prometheus.yml:
    rule_files:
    - /etc/config/recording_rules.yml
    - /etc/config/alerting_rules.yml
    scrape_configs:
    - job_name: prometheus
      static_configs:
      - targets:
        - prometheus.breeze-blocks.net:9090

prometheus.yml

scrape_configs

job_name

api servers scrape config

A scrape configuration for running Prometheus on a Kubernetes cluster. This uses separate scrape configs for cluster components (i.e. API server, node) and services to allow each to use different authentication configs.

Kubernetes labels will be added as Prometheus labels on metrics via the labelmap relabeling action.

Scrape config for API servers.

Kubernetes exposes API servers as endpoints to the default/kubernetes service so this uses endpoints role and uses relabelling to only keep the endpoints associated with the default/kubernetes service using the default named port https. This works for single API server deployments as well as HA API server deployments.

job_name

Scrape config for service endpoints

The relabeling allows the actual service scrape endpoint to be configured via the following annotations:

• prometheus.io/scrape: Only scrape services that have a value of true, except if prometheus.io/scrape-slow is set to true as well.

• prometheus.io/scheme: If the metrics endpoint is secured then you will need to set this to https & most likely set the tls_config of the scrape config.

• prometheus.io/path: If the metrics path is not /metrics override this.

• prometheus.io/port: If the metrics are exposed on a different port to the service then set this appropriately.

• prometheus.io/param_<parameter>: If the metrics endpoint uses parameters then you can set any parameter

job_name

Slow Service Scrape Config

---

Scrape config for slow service endpoints; same as above, but with a larger timeout and a larger interval

The relabeling allows the actual service scrape endpoint to be configured via the following annotations:

• prometheus.io/scrape-slow:: Only scrape services that have a value of true

• prometheus.io/scheme: If the metrics endpoint is secured then you will need to set this to https & most likely set the tls_config of the scrape config.

• prometheus.io/path: If the metrics path is not /metrics override this.

• prometheus.io/port: If the metrics are exposed on a different port to the service then set this appropriately.

• prometheus.io/param_<parameter>: If the metrics endpoint uses parameters then you can set any parameter

job_name

Service Scrape Config

---

Example scrape config for probing services via the Blackbox Exporter.

The relabeling allows the actual service scrape endpoint to be configured via the following annotations:

• prometheus.io/probe: Only probe services that have a value of true

job_name

Pod Scrape Config

---

Example scrape config for pods

The relabeling allows the actual pod scrape endpoint to be configured via the following annotations:

• prometheus.io/scrape: Only scrape pods that have a value of true, except if prometheus.io/scrape-slow is set to true as well.

• prometheus.io/scheme: If the metrics endpoint is secured then you will need to set this to https & most likely set the tls_config of the scrape config.

• prometheus.io/path: If the metrics path is not /metrics override this.

• prometheus.io/port: Scrape the pod on the indicated port instead of the default of 9102.

job_name

Scrape Slow Pods

---

Example Scrape config for pods which should be scraped slower. An useful example would be stackriver-exporter which queries an API on every scrape of the pod

The relabeling allows the actual pod scrape endpoint to be configured via the following annotations:

• prometheus.io/scrape-slow: Only scrape pods that have a value of true

• prometheus.io/scheme: If the metrics endpoint is secured then you will need to set this to https & most likely set the tls_config of the scrape config.

• prometheus.io/path: If the metrics path is not /metrics override this.

• prometheus.io/port: Scrape the pod on the indicated port instead of the default of 9102.

extraScrapeConfigs

Additional Promethes Config

---

adds additional scrape configs to prometheus.yml must be a string so you have to add a | after extraScrapeConfigs: example adds prometheus-blackbox-exporter scrape config

- job_name: 'prometheus-blackbox-exporter'
  metrics_path: /probe
  params:
    module: [http_2xx]
  static_configs:
    - targets:
      - https://example.com
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: prometheus-blackbox-exporter:9115

alertRelabelConfigs

Alert Relabel

---

Adds option to add alert_relabel_configs to avoid duplicate alerts in alertmanager useful in H/A prometheus with different external labels but the same alerts

alert_relabel_configs:
- source_labels: [dc]
  regex: (.+)\d+
  target_label: dc

networkPolicy

Network Policy

---

Enable creation of NetworkPolicy resources.

forceNamespace

Force Namespace

---

Force namespace of namespaced resources

extraManifests

Extra Manifests

---

Extra manifests to deploy as an array

- |
  apiVersion: v1
  kind: ConfigMap
  metadata:
  labels:
    name: prometheus-extra
  data:
    extra-data: "value"

alertmanager

Subchart configuration

---

Configuration of subcharts defined in Chart.yaml

Alert Manager

---

alertmanager sub-chart configurable values More information available here

If alertmanager.enabled: false, alertmanager will not be installed

kube-state-metrics

Kube State Metrics

---

kube-state-metrics sub-chart configurable values More information available here.

If kube-state-metrics.enabled: false, kube-state-metrics sub-chart will not be installed

prometheus-node-exporter

Prometheus Node Exporter

---

prometheus-node-exporter sub-chart configurable values More information available here.

If prometheus-node-exporter.enabled: false, node-exporter will not be installed

prometheus-pushgateway

Prometheus Push Gateway

---

prometheus-pushgateway sub-chart configurable values More information is available here

If `prometheus-pushgateway.enabled: false, pushgateway will not be installed