Grafana Kubernetes deployment

Deploy the Lightstep Incident Response Grafana plugin in the Kubernetes environment. The steps here are prepared using minikube in a development environment. Some of the steps might vary in an actual production environment.

Note: This is a sample script to help with the configuration. The steps could change depending on your environment and OS.

Create Docker image with Grafana image and LIR plugin

  1. Get the docker file from the Grafana github repository from here. The url is for alpine-linux. For ubuntu use a different dockerfile in the same folder.
  2. Save the contents of the above link into a file (Dockerfile).
  3. Build the docker image by running the following command. Note that there is a dot at the end:
    docker build \
      --build-arg "GRAFANA_VERSION=latest" \
      --build-arg "GF_INSTALL_PLUGINS=https://github.com/ServiceNow/lightstep-ir-grafana-metrics/archive/master.zip;lightstep-ir-grafana-metrics" \
      -t lir-grafana-image -f Dockerfile .
    

Run Grafana on Kubernetes environment

  1. Create a file named lir-grafana-pv.yaml using the following:
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: grafana-pv
    spec:
      capacity:
        storage: 1Gi
      accessModes:
      - ReadWriteOnce
      storageClassName: manual
      hostPath:
        path: "/mnt/data"
    
  2. Create a file named lir-grafana-pv-claim.yaml using the following:
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: grafana-pvc
    spec:
      storageClassName: manual
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi
    
  3. Create a file name lir-grafana.yaml using the following:
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: grafana
      name: grafana
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: grafana
      template:
        metadata:
          labels:
            app: grafana
        spec:
          securityContext:
            runAsUser: 0
          containers:
            - name: grafana
              image: lir-grafana-image
              imagePullPolicy: Never
              env:
              -  name: GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS
                 value: lightstep-ir-grafana-metrics
              ports:
                - containerPort: 3000
                  name: http-grafana
                  protocol: TCP
              readinessProbe:
                failureThreshold: 3
                httpGet:
                  path: /robots.txt
                  port: 3000
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 30
                successThreshold: 1
                timeoutSeconds: 2
              livenessProbe:
                failureThreshold: 3
                initialDelaySeconds: 30
                periodSeconds: 10
                successThreshold: 1
                tcpSocket:
                  port: 3000
                timeoutSeconds: 1
              volumeMounts:
                - mountPath: /var/lib/grafana
                  name: grafana-storage
          volumes:
            - name: grafana-storage
              persistentVolumeClaim:
                claimName: grafana-pvc
    
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: grafana
    spec:
      ports:
        - port: 3000
          protocol: TCP
          targetPort: http-grafana
      selector:
        app: grafana
      sessionAffinity: None
      type: LoadBalancer
    
  4. Run the following commands to create PersistentVolume, PersistentVolumeClaim, Deployment, Services, and Pods:
    kubectl create -f lir-grafana-pv.yaml
    kubectl create -f lir-grafana-pv-claim.yaml
    kubectl create -f lir-grafana.yaml
    
  5. Verify by running the following commands to see if the resources mentioned above are created and the status is shown as Bound:
    kubectl get pv grafana-pv 
    kubectl get pvc grafana-pvc
    
  6. Verify by running the following command to see if the status of the Pods are Running:
    kubectl get pods