Kubernetes - Deploy the Prometheus-Grafana Stack for Cluster Monitoring

Since I'm mostly working with OpenShift I'm used to the Monitoring Stack being already deployed. However, if you're rolling your own Kubernetes Stack, you'll have to take care of monitoring yourself and I'd like to stick to the Prometheus-Grafana Stack since I'm fairly familiar with it.

This is not intended to be a production-ready Deployment but more in the category of 'Proof-of-Concept'.

This setup will require a working Kubernetes Cluster with the following Features:

• default Storage Class ('managed-nfs' in my case)
• working Ingress Class (I'll be using 'nginx-ingress')

Additionally, access to the 'helm' binary on the workstation.

Since this is a demo, I'll also provide a self-signed Cert for the Ingress. This is also what we're starting with:

 [archy@workstation ~]$ DEPLOYMENT=grafana  
 [archy@workstation ~]$ KEY="${DEPLOYMENT}.key"  
 [archy@workstation ~]$ CRT="${DEPLOYMENT}.crt"  
 [archy@workstation ~]$ INGRESSHOST="${DEPLOYMENT}.apps.k8s.15knetworks.com"  
 [archy@workstation ~]$ openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -keyout ${KEY} -out ${CRT} -subj "/CN=${INGRESSHOST}/O=${INGRESSHOST}" -addext "subjectAltName = DNS:${INGRESSHOST}"  

Now, create the namespace. I'll be deploying the stack in a namespace called 'monitoring':

 [archy@workstation ~]$ kubectl create namespace monitoring  

Now that the namespace and certificates are present, create the secret for the ingress created by the helm chart later on:

 [archy@workstation ~]$ kubectl -n monitoring create secret tls ${DEPLOYMENT}-certs --key ${DEPLOYMENT}.key --cert ${DEPLOYMENT}.crt  

Next, add the helm repo:

 [archy@workstation ~]$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts  
 [archy@workstation ~]$ helm repo update  

The stock 'values.yaml' can be downloaded here. Some customizations that I'll need to for grafana are:

 grafana:  
   adminPassword: replace-with-your-password  
   ingress:  
     enabled: true  
     ingressClassName: nginx  
     annotations:  
       kubernetes.io/ingress.class: nginx  
       nginx.ingress.kubernetes.io/ssl-redirect: "true"  
       nginx.ingress.kubernetes.io/use-forwarded-headers: "true"  
       nginx.ingress.kubernetes.io/affinity: "cookie"  
       nginx.ingress.kubernetes.io/session-cookie-name: "selector"  
       nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"  
     hosts:  
       - grafana.apps.k8s.archyslife.lan  
     tls:  
       - secretName: grafana-certs  
         hosts:  
           - grafana.apps.k8s.archyslife.lan  
  persistence:  
    enabled: true  
    type: sts  
    storageClassName: "managed-nfs"  
    accessModes:  
      - ReadWriteOnce  
    size: 20Gi  
    finalizers:  
      - kubernetes.io/pvc-protection  

After that, install the stack using helm:

 [archy@workstation ~]$ helm install monitoring prometheus-community/kube-prometheus-stack -n monitoring -f values.yaml  

The initial rollout might take a few moments.

After all pods are up, the Grafana WebUI can be accessed using this url in my case: 'https://grafana.apps.k8s.archyslife.lan'. The Credentials are specified in the values.yaml mentioned above.

Feel free to comment and / or suggest a topic.