Configure Workload Monitoring

×

Apart from infrastructure metrics like CPU, memory, disk IO, etc., applications generate metrics based on their app logic. For example, DB applications can have several queries, several pending transactions, and so on; message broker applications can have several topics, several messages not delivered, and so on. These metrics are essential in debugging and finding out root-causing issues.

Kubernetes 2.0 provides a mechanism to display both application/workload metrics and the infrastructure metrics of pods.

Configure Workload Monitoring

Sample ConfigMap

Below is an example of how workloads can be configured in the opsramp-workload-metric-user-config ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: opsramp-workload-metric-user-config
  namespace: <agent-installed-namespace>
data:
  workloads: |
    mysql: 
    - name: mysql1 
....Mysql monitoring Config....
    - name: mysql2
....Mysql monitoring Config....
    nginx:
    - name: nginx1
....Nginx monitoring Config....
    - name: nginx2
....Nginx monitoring Config....

Common Configuration for All Workloads

This section outlines the common configuration settings that apply to all workloads.

Sample Configuration

  userName: 
    source: configmap 
    value: default/nginx-cm/user 
  password: 
    source: secret 
    value: default/nginx-secret/password 

Application-Specific Configuration

This section provides detailed configuration settings tailored for specific applications. Each application may have unique requirements and parameters that need to be adjusted for optimal performance and compatibility.

Nginx

NGINX is a high-performance web server and reverse proxy used for load balancing, caching, and media streaming. The OpenTelemetry NGINX Receiver helps monitor key metrics like request rates, response times, errors, and connections.

Sample Nginx Config (No Authentication)

nginx:
- name: nginx1
  maxConnsPerHost: 4
  idleConnTimeout: 5s
  disableKeepAlives: true
  http2ReadIdleTimeout: 1m
  collectionFrequency: 20m
  http2PingTimeout: 5s
  httpPath: stub_status
  auth: none
  port: 8080
  targetPodSelector: 
    matchLabels: 
      - key: app 
        operator: ==
        value: 
          - my-nginx

Sample Nginx Config (TLS Authentication)

- name: nginx-with-tls
  maxConnsPerHost: 4
  idleConnTimeout: 5s
  disableKeepAlives: true
  http2ReadIdleTimeout: 1m
  http2PingTimeout: 5s
  collectionFrequency: 10s
  httpPath: tls_stub_status
  auth: tls
  certFile:
    source: secret
    value: default/nginx-secret/tls.crt
  keyFile:
    source: secret
    value: default/nginx-secret/tls.key
  port: 443
  targetPodSelector: 
    matchLabels: 
      - key: app 
        operator: ==
        value: 
          - my-nginx-tls

HA Proxy

HAProxy is a high-performance load balancer and reverse proxy for TCP and HTTP. The OpenTelemetry HAProxy Receiver enables monitoring of key metrics like request rates, response times, errors, sessions, and backend health.

Sample HA Proxy (No Authentication)

  - name: haproxy
    maxConnsPerHost: 4
    idleConnTimeout: 5s
    disableKeepAlives: true
    http2ReadIdleTimeout: 1m
    collectionFrequency: 20m
    http2PingTimeout: 5s
    httpPath: stats
    auth: none
    port: 8080
    targetPodSelector:
      matchLabels:
        - key: app
          operator: ==
          value:
            - haproxy

Sample HA Proxy (TLS Authentication)

  - name: haproxy-with-tls
    maxConnsPerHost: 4
    idleConnTimeout: 5s
    disableKeepAlives: true
    http2ReadIdleTimeout: 1m
    http2PingTimeout: 5s
    collectionFrequency: 10s
    httpPath: stats
    auth: tls
    certFile:
      source: secret
      value: default/haproxy-secret/tls.crt
    keyFile:
      source: secret
      value: default/haproxy-secret/tls.key
    port: 443
    targetPodSelector:
      matchLabels:
        - key: app
          operator: ==
          value:
            - haproxy-tls

Redis

Redis is a high-performance in-memory data store used as a database, cache, and message broker. The OpenTelemetry Redis Receiver enables monitoring of key metrics like memory usage, latency, and keyspace operations.

Sample Redis (No Authentication)

    redis:
      - name: redis
        auth: none
        port: 6379
        collectionFrequency: 2m
        targetPodSelector:
          matchLabels:
            - key: app
              operator: ==
              value:
                - redis

Sample Redis (TLS Authentication)

      - name: redis-with-tls
        auth: tls
        certFile:
          source: secret
          value: default/redis-tls-client/redis-client.crt
        keyFile:
          source: secret
          value: default/redis-tls-client/redis-client.key
        port: 6379
        collectionFrequency: 2m
        targetPodSelector:
          matchLabels:
            - key: app
              operator: ==
              value:
                - redis-tls

Sample Redis (Credentials)

      - name: redis-with-credentials
        auth: credentials
        userName:
          source: value
          value: sri
        password:
          source: configmap
          value: default/redis-config/REDIS_PASSWORD
        port: 6379
        collectionFrequency: 2m
        targetPodSelector:
          matchLabels:
            - key: app
              operator: ==
              value:
                - redis-creds

MySQL

MySQL is a popular relational database known for its performance and reliability. The OpenTelemetry MySQL Receiver helps monitor key performance metrics efficiently.

Sample Mysql (No Authentication)

    mysql: 
    - name: mysql1 
      userName: 
        source: value 
        value: root 
      password: 
        source: value 
        value: password 
      targetPodSelector: 
        matchLabels: 
          - key: app 
            operator: == 
            value: ["mysql"]

Kafka

Apache Kafka is a distributed event streaming platform for real-time data pipelines and event-driven applications. The OpenTelemetry Kafka Receiver helps monitor key metrics like broker health, topic throughput, and consumer lag for optimal performance.

Sample Kafka (No Authentication)

  - name: kafka-plaintext
    collectionInterval: 60s
    protocolVersion: "2.0.0"
    scrapers:
      - brokers
      - topics
      - consumers
    auth: plaintext
    port: 9092
    targetPodSelector:
      matchLabels:
        - key: app.kubernetes.io/instance
          operator: ==
          value:
            - my-kafka-plain
        - key: app.kubernetes.io/name
          operator: ==
          value:
            - kafka

Sample Kafka (SASL Plain text)

  - name: kafka-sasl-plaintext
    collectionInterval: 60s
    protocolVersion: "2.0.0"
    scrapers:
      - brokers
      - topics
      - consumers
    auth: sasl_plaintext
    userName:
      value: "user1"
    password:
      value: "password1"
    port: 9092
    targetPodSelector:
      matchLabels:
        - key: app.kubernetes.io/instance
          operator: ==
          value:
            - my-kafka-plain
        - key: app.kubernetes.io/name
          operator: ==
          value:
            - kafka

Sample Kafka (TLS Authentication)

  - name: kafka-tls
    collectionInterval: 60s
    protocolVersion: "2.0.0"
    scrapers:
      - brokers
      - topics
      - consumers
    auth: tls
    certFile:
      source: secret
      value: default/kafka-client-tls-secret/client.pem
    keyFile:
      source: secret
      value: default/kafka-client-tls-secret/client-key.pem
    insecureSkipVerify: true
    port: 9093
    targetPodSelector:
      matchLabels:
        - key: app.kubernetes.io/instance
          operator: ==
          value:
            - my-kafka-plain
        - key: app.kubernetes.io/name
          operator: ==
          value:
            - kafka

End-to-End Guide: Monitoring a Redis Instance in Kubernetes

This example demonstrates how to monitor a Redis instance running in a Kubernetes cluster using a ConfigMap. The provided ConfigMap supports multiple authentication methods for seamless integration.

Step 1: Retrieve the ConfigMap

To check the existing workload configuration, use the following command:

kubectl get configmap workload-master -n <agent-installed-namespace>

Step 2: Create the ConfigMap

If this is your first time setting up workload monitoring, create a new ConfigMap named workload-master in the <agent-installed-namespace>. Use the following manifest as a template:

apiVersion: v1
kind: ConfigMap
metadata:
  name: workload-master
  namespace: opsramp-agent
data:
  workloads: |
    redis:
      - name: redis
        auth: none
        port: 6379
        collectionFrequency: 2m
        targetPodSelector:
          matchLabels:
            - key: app
              operator: ==
              value:
                - redis
      - name: redis-with-tls
        auth: tls
        certFile:
          source: secret
          value: default/redis-tls-client/redis-client.crt
        keyFile:
          source: secret
          value: default/redis-tls-client/redis-client.key
        port: 6379
        collectionFrequency: 2m
        targetPodSelector:
          matchLabels:
            - key: app
              operator: ==
              value:
                - redis-tls
      - name: redis-with-credentials
        auth: credentials
        userName:
          source: value
          value: sri
        password:
          source: configmap
          value: default/redis-config/REDIS_PASSWORD
        port: 6379
        collectionFrequency: 2m
        targetPodSelector:
          matchLabels:
            - key: app
              operator: ==
              value:
                - redis-creds

Step 3: Apply the ConfigMap

Once you have created or updated the manifest with all desired workloads, apply it using the following command:

kubectl apply -f <path-to-your-manifest-file>

Step 4: View Metrics in OpsRamp Portal

After a few minutes, you will be able to see the metrics in the OpsRamp portal under Dashboards > Dashboard.

Step 5: Create Alerts and Availability Rules

With metrics flowing into the OpsRamp portal, you can create alerts and availability rules based on these workload or application-specific metrics by editing the respective K8s ConfigMaps. This version provides clear steps and maintains a logical flow, making it easy for users to follow along.