Incident Response Automation

Automation contains response and grouping rules that are conditional triggers that execute response actions automatically, based on the contents of the alert.

An alert is created in Incident Response when a third-party monitoring tool has an event. The system analyzes incoming alerts, once a minute, and runs grouping and response rules accordingly.

Grouping rules

Use grouping rules to automatically group similar alerts and reduce the noise in you queue. Grouping rules are routing that helps you categorize by specific conditions. Each rule combines a set of filters and methods based on fields or condition matches.

A system default rule for automatic group creation based on Service and Metric name fields is included when your system is provisioned. It is active by default.

Default grouping rule

View the grouping rules by navigating to the Automation module from your Home page. Select the Grouping rules tab.

Create rules based on any criteria available from the field or condition fields.
Note: Prior to creating grouping rules, you should have teams, schedules, and integrations in place.
Rules are run only once on an alert, however, depending on when the alert or the rule entered the system, different things happen.
  • If an alert is created, and rule 1 is the only rule, it will run only once on that alert.
  • If you add rules 2 and 3 after that alert was created, they will not run on it unless the alert is updated. Those rules were never run on the alert to begin with.
  • If you close and reopen the alert, all three rules will run on it.

If you change an existing rule and an existing alert would have been captured by it, it won’t be. Whatever criteria or actions you have changed won’t be addressed unless the existing alert is closed and reopened. Then, whatever changed rule actions are will take place.

Grouping rules run in the order chosen, from lowest to highest. You can use the Rule Order icon to reorder your rules. Enter a new number and click Update in the Response rules list view.
Note: Administrators can reorder any rule. Responders and managers can reorder only their own or their team's rules.
Note:

Order is important for grouping rules because the lower-order rules run first. This can impact whether the conditions of higher-order rules are met, or will process. Lower-order rules can adjust field values or even stop processing.

The grouping rule list view can be arranged in several ways.
Note: This feature does not change the order in which the rules are run.
  • Order (asc): Order they are in, ascending - Default.
  • Order (desc): Order they are in, descending
  • Last Modified (recent): Recent
  • Last Modified (oldest): Oldest
You can filter the listings using the Filters list menu in the header for:
  • System Generated Rules: Displays only rules that were created automatically when your account was provisioned.
  • My Team's Rules: Displays only those rules your team created.
  • Rules I created: Displays only those rules you created.
  • Active: Displays only active alerts.
  • Inactive: Displays only inactive alerts.

Name & Description represents the group rule name and description.

Scope represents who or what is impacted by this rule. Whether the impact is Global or to a team or service.

Criteria represents which grouping method this rule uses. Either Field Match Field Match icon or Condition Match Condition Match.

Timeframe represents how long this rule should continue to run and group alerts after an initiating alert.

Whether your rules are Active or not is shown using the slider icon.
Active slider
The order of alert grouping is as follows:
  • Grouping rules run
  • Manually created groups are updated
  • Automated grouping runs
All grouping occurs before response rules run.

Alerts are grouped under the primary alert. The oldest of the highest severity alerts becomes the primary alert. The rest become related alerts. Grouping activity is recorded in Work notes..

A grouped, related alert contains a Grouped by field. This field indicates how the related alert was grouped. For more information, see Alert workspace.

For some examples of different response rules, see Grouping rules examples.
Note: Managers and responders can activate or deactivate only their own or their team’s rules.

Only administrators can activate or deactivate any rule, including global rules.

Response rules

Use response rules to automatically assign and respond to alerts. Response rules are routing that helps you adapt your response to specific conditions. Each rule combines a set of filters and an outcome, usually setting who is assigned, priority, or creating email, Slack channels, or Zoom meetings.

Watch this quick video to know what you can do with response rules.

A system default rule for automatic incident creation for P1 alerts is included when your system is provisioned. It is active by default.
Default response rule

View the response rules by navigating to the Automation module from your Home page.

Create rules based on any criteria available from the condition filter fields. You can also set filter properties on the rules.
Note: Prior to creating response rules, you should have teams, schedules, and integrations in place.
Rules are run only once on an alert, however, depending on when the alert or the rule entered the system, different things happen.
  • If an alert is created, and rule 1 is the only rule, it will run only once on that alert.
  • If you add rules 2 and 3 after that alert was created, they will not run on it unless the alert is updated. Those rules were never run on the alert to begin with.
  • If you close and reopen the alert, all three rules will run on it.

If you change an existing rule and an existing alert would have been captured by it, it won’t be. Whatever criteria or actions you have changed won’t be addressed unless the existing alert is closed and reopened. Then, whatever changed rule actions are will take place.

Response rules run in the order chosen, from lowest to highest. Or you can use the Rule Order icon to reorder your rules. Enter a new number and click Update in the Response rules list view.
Note: Administrators can reorder any rule. Responders and managers can reorder only their own or their team's rules.
Renumbering the rule to reorder it
Note:

Order is important for response rules because the lower-order rules run first. This can impact whether the conditions of higher-order rules are met, or will process. Lower-order rules can adjust field values or even stop processing.

The response rule list view can be arranged in several ways.
Note: This feature does not change the order in which the rules are run.
Order in which the rules run
  • Order: Order they are in, ascending - Default.
  • Order: Order they are in, descending
  • Last Modified: Recent
  • Last Modified: Oldest
You can filter the listings using the Filters list menu in the header for:
  • System Generated Rules: Displays only rules that were created automatically when your account was provisioned.
  • My Team's Rules: Displays only those rules your team created.
  • Rules I created: Displays only those rules you created.
  • Active: Displays only active alerts.
  • Inactive: Displays only inactive alerts.

Name & Description represents the alert rule name and description.

Scope represents who or what is impacted by this rule. Whether the impact is Global or to a team or service.

Actions are denoted by the action icons.
Response rule action icons

Stop after run contains a pause icon Pause icon if you wanted processing to stop after that rule runs.

Whether your rules are Active or not is shown using the slider icon.
Active slider

Response rule activity is recorded in Work notes.

For some examples of different response rules, see Response rule examples.
Note: Managers and responders can activate or deactivate only their own or their team’s rules.

Only administrators can activate or deactivate any rule, including global rules.