Loading

Create a failed docs rule

Serverless Stack 9.1.0

Note

The Editor role or higher is required to create a failed docs rule using custom threshold rule. To learn more, refer to Assign user roles and privileges.

Create a failed docs rule using the custom threshold rule to alert when the number of failed documents in your data stream reaches or exceeds a given value.

Create failed docs rule using the custom threshold rule type

When creating a failed docs rule, the process depends on your deployment type and your space's solution view. You can check your solution view by selecting the Spaces icon.

Select the appropriate tab for your setup, then follow the instructions to create a failed docs rule:

  1. From the main menu, open the Data Set Quality page from Management → Stack Management, or use the global search field.
  2. Find the data set you want to create a rule for in the table, and select Open from the Actions column.
  3. Select Alerts → Create custom threshold rule.
  4. Select Add aggregation/field.
  5. For your new aggregation, set Aggregation type to Count and KQL Filter to _index : ".fs*".
  6. Select Equation, and set the equation to (B / A) * 100.
  7. Set Is above to the desired threshold. For example, 1.5.
  8. Set the Label to Failed docs.
  9. Select Next to go to the Details step.
  10. Set the Rule name to Data set quality and add failed_docs to the Tags.
  11. Select Create rule.
  1. Select Manage rules and connectors.
  2. Select Create rule, then Custom threshold.
  3. Select Data view, then Create a data view.
  4. Find your data stream under All sources.
  5. Name your data view.
  6. Add your index pattern with ::failures appended. For example, logs-synth.2-default::data,logs-synth.2-default::failures.
  7. Select Save data view to Kibana.
  8. Select Add aggregation/field.
  9. For your new aggregation, set Aggregation type to Count and KQL Filter to _index : ".fs*".
  10. Select Equation, and set the equation to (B / A) * 100.
  11. Set Is above to the desired threshold. For example, 1.5.
  12. Set the Label to Failed docs.
  13. Select Next to go to the Details menu.
  14. Set the Rule name to Data set quality and add failed_docs to the Tags.
  15. Select Create rule.

You can extend your rules with actions that interact with third-party systems, write to logs or indices, or send user notifications. You can add an action to a rule at any time. You can create rules without adding actions, and you can also define multiple actions for a single rule.

To add actions to rules, you must first create a connector for that service (for example, an email or external incident management system), which you can then use for different rules, each with their own action frequency.

Connector types

Connectors provide a central place to store connection information for services and integrations with third party systems. The following connectors are available when defining actions for alerting rules:

Note

Some connector types are paid commercial features, while others are free. For a comparison of the Elastic subscription levels, go to the subscription page.

For more information on creating connectors, refer to Connectors.

Action frequency

After you select a connector, you must set the action frequency. You can choose to create a summary of alerts on each check interval or on a custom interval. Alternatively, you can set the action frequency such that you choose how often the action runs (for example, at each check interval, only when the alert status changes, or at a custom action interval). In this case, you must also select the specific threshold condition that affects when actions run: Alert, No Data, or Recovered.

Configure when a rule is triggered

You can also further refine the conditions under which actions run by specifying that actions only run when they match a KQL query or when an alert occurs within a specific time frame:

  • If alert matches query: Enter a KQL query that defines field-value pairs or query conditions that must be met for notifications to send. The query only searches alert documents in the indices specified for the rule.
  • If alert is generated during timeframe: Set timeframe details. Notifications are only sent if alerts are generated within the timeframe you define.
Configure a conditional alert
Action variables

Use the default notification message or customize it. You can add more context to the message by clicking the Add variable icon Add variable and selecting from a list of available variables.

Action variables list

The following variables are specific to this rule type. You can also specify variables common to all rules.

context.alertDetailsUrl
Link to the alert troubleshooting view for further context and details. This will be an empty string if the server.publicBaseUrl is not configured.
context.cloud
The cloud object defined by ECS if available in the source.
context.container
The container object defined by ECS if available in the source.
context.group
The array of objects containing groups that are reporting data.
context.grouping Stack 9.1.0
The object containing groups that are reporting data.
context.host
The host object defined by ECS if available in the source.
context.labels
List of labels associated with the entity where this alert triggered.
context.orchestrator
The orchestrator object defined by ECS if available in the source.
context.reason
A concise description of the reason for the alert.
context.tags
List of tags associated with the entity where this alert triggered.
context.timestamp
A timestamp of when the alert was detected.
context.value
List of the condition values.
context.viewInAppUrl

Link to the alert source.