An alert that never reaches anyone is useless. This guide walks through connecting WeCom, DingTalk, and email in OBSERVE, configuring severity routing and escalation, with webhook examples and a pre-launch checklist.
An alert that never reaches anyone is worse than no alert. OBSERVE ships with email, WeCom (WeChat Work) bots, DingTalk bots, and generic webhooks, plus SMS and phone for high-severity alerts. Picking a channel is simple:
The mistake most teams make is routing everything to one noisy channel, then muting it. Split the stream instead: routine stuff to email, actionable alerts to the group bot, and only the truly urgent to phone. If every alert is a phone call, within a week nobody answers.
First add a bot to the group and grab a URL like https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxxx. Then create a channel under "Alert Notifications → Channels":
channel:
name: oncall-wecom
type: wecom
webhook: https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxxx
template: |
[{level}] {title}
Target: {target}
Value: {value} (threshold {threshold})
Time: {time}
Detail: {url}DingTalk is nearly identical — just set type: dingtalk and use a webhook with an access_token. Use placeholders like {level}, {title}, and {value} to compress the essential info into one message so the on-call channel doesn't turn into a wall of spam. A good template answers four questions at a glance: what broke, how bad, on which target, and where's the detail link. Everything else — the full metric history, the related traces — belongs in the detail page, not the chat.
Route alerts by severity and escalate automatically when nobody acknowledges them:
route:
rules:
- severity: P0
channels: [sms, phone, wecom-p0]
repeat: 5m
- severity: P1
channels: [wecom-p0, dingtalk]
repeat: 15m
- severity: P2
channels: [wecom-general]
repeat: 30m
escalation:
after: 15m
to: [phone, manager]The escalation block is the part people skip, and it's the part that actually closes the loop: a P1 that goes unacknowledged for 15 minutes gets bumped to phone plus the on-call manager. repeat controls how often an unresolved alert re-sends, so a single alert doesn't fire once and sink to the bottom of the chat. Set repeat longer than your average time-to-acknowledge, or you'll re-ping someone who's already looking at it.
Finally, keep a short on-call runbook pinned in the group: who acknowledges, who it escalates to, and after how long. That is what turns a stream of notifications into a closed loop.
A couple of operational details worth flagging. First, mind the time zones: if your on-call rotates across regions, make sure the escalation window respects the local hours of whoever is on duty, otherwise a "no ack in 15 minutes" rule can fire at 4 a.m. for the wrong person. Second, test the full path end to end — trigger a real alert against a staging target and watch it land in the group, then acknowledge it and confirm the escalation does not fire. Nothing exposes a broken webhook faster than a real alert on a Friday afternoon. Finally, keep the channel count per alert small: one primary channel plus one escalation target is usually right. More than three and you're generating noise, not coverage — and noise is what makes people mute the whole group.