← Back to blog
Changelog 4 min read 炬鲸团队

OBSERVE v2.4: Alert Deduplication, Trace Drill-Down, and Native DM Support

Release notes for v2.4: a new alert deduplication engine, trace drill-down, native support for DM and KingbaseES, automatic log hot/cold tiering, plus bug fixes, upgrade steps, and compatibility notes.

New features

Alert deduplication engine: multiple alerts with the same root cause are merged into one, grouped by service, host, or time window, preventing alert storms. Deduplicated alerts include a "related alerts" list and a root-cause hint, so the on-call engineer is no longer buried in repeated messages. The engine runs before notification, which means your paging integrations see a single consolidated alert instead of a flood. The grouping keys and time window are configurable per rule, so a noisy service can use a wider window than a quiet one.

Trace drill-down: the application topology now supports drilling down layer by layer. Click a node to view that service's metrics, error rate, and top slow calls, then jump from a slow call straight to the trace detail and correlated logs — troubleshooting shortens to a single click. This closes the loop between "something is slow" and "here is the exact stack trace." The drill-down runs from the service level down to the individual span, and from there into the correlated log lines and the host metrics of the machine that served the request — one continuous path with no context switch.

Native DM / KingbaseES support: the metadata layer now adapts to DM8 and KingbaseES dialects. Just select the database type when creating the schema — no third-party compatibility layer needed — making Xinchuang deployment simpler and removing a whole class of driver-related failures.

Automatic log hot/cold tiering: logs are retained for 30 days by default, with the last 7 days in hot storage and automatic migration to cold storage after that. Migration is transparent to queries; cold-data search latency is slightly higher but cost drops by about 70%.

Improvements and fixes

  • Full-text search: phrase query performance improved roughly 3x, with custom Chinese tokenizer dictionaries
  • Metrics: P99/P95 percentile calculations are more stable over long time spans
  • Fixed: intermittent trace sampling loss under high concurrency
  • Fixed: duplicate reporting from the log collector after restart on KylinOS V10
  • UI: the alert rule editor now supports drag-to-adjust thresholds
  • Alert rules: added a "for" duration (sustained condition) to reduce flapping alerts
  • Collector: multiline Java stack traces are now parsed as a single log event instead of one event per line

Upgrade steps

Back up the metadata database and configuration before upgrading, and do it during a maintenance window. After extracting the offline bundle, run:

cd observe-2.4.0
./install.sh upgrade

The upgrade script performs schema migration and data validation automatically; the whole process takes about 10-20 minutes, during which the console is read-only. After upgrading, run ./check.sh for a health check and confirm all components are healthy before restoring traffic. If you're upgrading from 2.2 or earlier, go to 2.3 first, then 2.4 — you cannot skip versions. Before running the upgrade, record the current version and the list of active alert rules and dashboards, and confirm the backup actually restores — an untested backup is a backup you don't have. Coordinate the window with whoever owns downstream alert routing, since the console is read-only during migration.

Rollback

If the upgrade goes wrong, ./install.sh rollback restores the previous version's binaries and the metadata snapshot taken during the upgrade. Keep the backup directory until you've run 2.4 in production for at least a few days — most upgrade problems only surface under real traffic, not in a staging smoke test. On a multi-node cluster, upgrade the control plane node first and confirm it's healthy before touching the search nodes.

Compatibility and known limitations

The agent side is fully backward compatible; older collectors do not need to be upgraded. Known limitations: alert deduplication currently supports only "service" and "host" grouping dimensions — grouping by business tags will arrive in 2.5. Full-text search in DM mode does not yet support pinyin search. These are already queued for the next iteration.