Every alert pages on-call
Without severity-aware routing, warning-level alerts wake people up for non-incidents.
Fix
Map severity to channel: warnings go to chat, criticals go to voice. Severity is the only knob.
Hook your infrastructure monitoring (Prometheus, Zabbix, Uptime Kuma, custom) into WardenPoint. Severity decides channel; labels decide responder; resolve hooks cancel the chain.
What a quiet day looks like
Where infra teams hurt
Three complaints from infrastructure teams whose alerts wake people for the wrong reasons.
Without severity-aware routing, warning-level alerts wake people up for non-incidents.
Fix
Map severity to channel: warnings go to chat, criticals go to voice. Severity is the only knob.
A flaky node fires alert/resolve/alert/resolve until someone silences it manually.
Fix
We dedupe by source-and-key. Identical alerts within a window collapse into one notification with a counter.
When a cluster is in maintenance, alerts from that cluster should not page anyone.
Fix
Maintenance windows mute by label match. Set a TTL and we resume normal routing automatically.
Escalation policy
Three severity tiers, three different paths. Configure once, reuse everywhere.
Severity=critical fires phone + Telegram voice immediately. Cluster down, payments broken, customer-impacting outage.
Severity=warning posts to the ops Telegram chat. No one wakes up; the next-shift handles it.
If five minutes pass without ack, secondary on-call gets the phone call.
Info-level alerts collapse into a daily email digest. Useful for trends, not for action.
Where alerts come from
Whatever you run, WardenPoint accepts the webhook.
What changes
Warning-level alerts move out of the on-call phone and into the chat. Sleep ratings improve.
Identical alerts collapse. Post-mortems no longer count 30 pages for one incident.
Maintenance windows mute by label. No one apologises for paging during planned downtime.
Infra FAQ
Wire one Prometheus or Zabbix source, define severity routing and watch the next warning land in chat — not in the bedroom.