Alerts

Create alert rules and route firing alerts to Slack and email.

The Alerts page shows alerts and lets you manage the rules that produce them. Alerts have two states: firing and resolved. When an alert is firing you can Resolve it from the list, or click Declare incident to escalate it into Incident Management (Enterprise) for team response and a postmortem.

Alert rules

Open the Rules tab and click + Add rule. A rule has:

  • Name
  • Description
  • Metric, grouped by area:
    • Resource Usage: Container CPU %, Container Memory %, Container Memory MB, Pod CPU %, Pod Memory %, Pod Restart Count, Node Not Ready
    • Fleet Health: Agent Silent (minutes)
    • CI/CD: Pipeline Failed, Pipeline Stuck, Pipeline Flaky, Deploy Regression, VCS Connector Unhealthy
    • Backups: Cluster/Host Backup Failed, Cluster/Host Backup Overdue, Disaster Recovery Backup Failed. Covers both Cluster & Host Backups and Disaster Recovery, which back up different things (see each page for the distinction)
    • Ingress & Gateway API: NGINX Ingress Reload Failed, NGINX Ingress Config Reload Errors, NGINX Gateway Fabric Reload Errors
    • Autoscaling: Cluster Autoscaler Unschedulable Pods, Cluster Autoscaler Not Safe to Scale
  • Operator: greater than, greater-or-equal, less than, or less-or-equal
  • Threshold
  • Severity

You can enable or disable, edit, and delete rules at any time. The alert engine checks each enabled rule every cycle against the latest sampled value: it fires as soon as the condition is true and resolves as soon as it isn't, since there's no sustained-duration window to wait out.

Some metrics are absolute values rather than percentages (for example a raw MB figure instead of a 0-100% one). A threshold that made sense as a percentage almost never makes sense as the same number against an absolute value: a rule like "greater than 70" against a percent-based metric fires around 70% usage as intended, but the identical "greater than 70" against a raw MB metric fires the moment usage passes 70 MB, which most workloads exceed immediately and never drop back below, so the alert never auto-resolves and has to be resolved by hand every time. If an alert never auto-resolves, double check the rule is using the percent-based version of its metric, not the absolute one, before assuming it's a bug.

Notifications

Configure notification channels under Alerts → Notifications (or Settings → Notifications). Supported channels are Slack and Email. When an alert fires, KubeWatch delivers it to the enabled channels. Each channel has Test (sends a real test message so you can confirm delivery actually works, independent of whether any alert has fired recently), Edit (change its name, Slack webhook URL, or email recipients without deleting and recreating it: the channel type itself can't be changed after creation), and Delete.

Alerting evaluates the metrics your agents report, so a rule only fires once the relevant metric is flowing in.