Create a webhook endpoint for Grafana 7.5.7

Create a webhook endpoint so that Grafana can send automated real-time alerts to Incident Response.

Before you begin

Role required: Responder, Manager, or Administrator

Procedure

  1. Log in to Lightstep Incident Response.
  2. On the navigation pane, select Integrations.
    Integrations landing page.
  3. Click the Grafana integration card.
  4. Enter the name of the integration in the Name field.
  5. The Status field shows the status of the integration such as enabled or disabled.
    Note: You can modify this field only after the webhook is generated.
  6. Select the service that you want to associate with the integration. You have two options:
    • Specify the service path in the JSON alert payload, so that Grafana can send alerts on behalf of various services without having to create a separate webhook for each service.
      1. Select the Service defined in payload? check box.
      2. To associate the alert to the service that is defined in the alert payload, enter the JSON path where the service name can be found in the JSON path for the service field.
        • If this service name already exists in Incident Response, the alert is assigned to it.
        • If the service name doesn't exist in Incident Response, then a new service is created and the alert is assigned to this service.
        • The alert is assigned to the default service selected in the Service field if:
          • The JSON path specified for the service doesn't exist in the alert payload.
          • The JSON path for the service exists, but there's no value given for it in the alert payload.
          • You've crossed your subscription limit.
        • Grafana supports multiple alerts in a single payload, in the form of alert arrays. In this case, you can provide the array path for the service. For example, alerts[3].labels.alertname.
        For example, if the JSON alert payload contains
        {
            …
            "labels":{
        	"service_name":"Shopping Cart",
        	…
            }
            …
        }
        
        and the JSON path is labels.service_name, then Incident Response will assign the alert to the Shopping Cart service.Service not defined in payload.
    • Select a single service so that Grafana alerts will always be associated with that service.
      1. Leave the Service defined in payload? check box deselected.
      2. Select a service in the Service field.
      Service not defined in payload.
  7. Add a description for your integration in the Description field.
  8. Add the URL of the home page of the monitoring tool that sends alerts in the Integration URL field.
  9. Add tags that can help users search for the integration using the Tags field.
  10. Click Generate Webhook.

    A webhook URL is generated. Grafana sends messages to this secure webhook endpoint.

    A webhook URL is generated.
    Copy the webhook by clicking the copy icon (Copy the webhook URL.). You will need it when configuring Grafana.
    Note: Click Send sample alert to generate a sample alert from Incident Response and view it in Alerts. This allows you to experiment with automation rules, on-call scheduling, and escalation policies without having to set up the webhook with an external monitoring tool.
  11. Click Save.

What to do next

Configure a webhook in Grafana