Incident Response Automation

Automation contains response rules that are conditional triggers that execute response actions automatically, based on the contents of the alert.

An alert is created in Incident Response when a third-party monitoring tool has an event. Use response rules to automatically assign and respond to those alerts. Response rules are routing that helps you adapt your response to specific conditions. Each rule combines a set of filters and an outcome, usually setting who is assigned, priority, or creating email, Slack channels, or Zoom meetings.

A system default rule for automatic incident creation for P1 alerts is included when your system is provisioned. It is active by default.
Default alert rule

View the response rules by navigating to the Automation module from your Home page.

Create rules based on any criteria available from the condition filter fields. You can also set filter properties on the rules.
Note: Prior to creating response rules, you should have teams, schedules, and integrations in place.
Rules are run only once on an alert, however, depending on when the alert or the rule entered the system, different things happen.
  • If an alert is imported, and rule 1 is the only rule, it runs only once on that alert.
  • If you add rules 2 and 3 after that alert was imported, they will not run on it unless the alert is updated. Those rules were never run on the alert to begin with.
  • If you close and reopen the alert, all three rules run on it.
If you change an existing rule and an existing alert would have been captured by it, it won’t be. Whatever criteria or actions you have changed won’t be addressed unless the existing alert is closed and reopened. Then, whatever changed rule actions are will take place.
Note: If a Slack channel already exists, another one won't be created, however, any changes are applied to the existing channel.
Response rules run in the order chosen, from lowest to highest. Or you can use the Rule Order icon to reorder your rules. Enter a new number and click Update in the Response rules list view.
Note: Administrators can reorder any rule. Responders and managers can reorder only their own or their team's rules.
Renumbering the rule to reorder it
Note:

Order is important for alert rules because the lower-order rules run first. This can impact whether the conditions of higher-order rules are met, or will process. Lower-order rules can adjust field values or even stop processing.

The alert rule list view can be arranged in several ways.
Note: This feature does not change the order in which the rules are run.
Order in which the rules run
  • Order: Order they are in, ascending - Default.
  • Order: Order they are in, descending
  • Last Modified: Recent
  • Last Modified: Oldest
You can filter the listings using the Filters list menu in the header for:
  • System Generated Rules: Displays only rules that were created automatically when your account was provisioned.
  • My Team's Rules: Displays only those rules your team created.
  • Rules I created: Displays only those rules you created.
  • Active: Displays only active alerts.
  • Inactive: Displays only inactive alerts.

Name & Description represents the alert rule name and description.

Scope represents who or what is impacted by this rule. Whether the impact is Global or to a team or service.

Actions are denoted by the action icons.
Response rule action icons

Stop after run contains a pause icon Pause icon if you wanted processing to stop after that rule runs.

Whether your rules are Active or not is shown using the slider icon.
Active slider
Note: Managers and responders can activate or deactivate only their own or their team’s rules.

Only administrators can activate or deactivate any rule, including global rules.

Alert rule examples

For detailed information on alert rules and alert actions, see Create a response rule.

For example:
  • If an alert description contains 'critical' or 'outage', set the Priority group to 1.
    Figure 1. Example of filtering based on text in the description of this alert
    Filtering based on text in the alert description
    Figure 2. Example of setting the Priority Group for this alert
    Setting the Priority group for this alert
  • If the Metric Name field value in an alert is not empty, send an email to the Assignment team's manager.
    Figure 3. Example of setting the filter based on Metric Name value for this alert
    Metric Name is not empty. Set the filter for Metric Name
    Figure 4. Example of sending an email for this alert
    Sending an email for this alert
  • If an alert is P1-Critical and you want to create a Zoom meeting for it:

    Figure 5. Create a Zoom meeting for this alert
    Setting up a zoom meeting for the alert
    Note: This rule requires the Zoom application integration.
  • If an alert is P1-Critical and you want to create an incident for it.

    Figure 6. Create an incident for this alert
    Creating an incident from a P1 alert