v2.5.0 ships three major features: generating alertable metrics directly from log streams, alert escalation with on-call rotation, and per-service trace sampling control. Here is how each works and what to watch when upgrading.
Previously, turning numbers in logs into metrics meant wiring up a separate metrics collector and aligning its labels with the logging system—two systems, two retention policies, and a lot of glue. v2.5.0 closes that gap: define a logs-to-metrics rule on top of a search query, and the system periodically aggregates matching logs into time-series metrics.
Usage is simple—specify the query, the aggregation, and the target metric name. For example, to count errors per minute grouped by level:
source: app-logs AND level:ERROR
| rate 1m by level
-> metric: app_error_rateGenerated metrics behave like native ones: build dashboards, set alerts, configure retention. It is ideal for numbers that used to require scripts—login failures per minute, 5xx ratio by region—now becoming history-backed, queryable time series without any extra collector to maintain.
A single alert notification is easy to miss in the middle of the night. v2.5.0 adds escalation chains: when an alert fires, it notifies the on-call person first, then escalates up a level after N minutes unacknowledged, up to three levels. Each level can use a different channel—WeCom at level one, SMS at level two, a phone call at level three.
On-call rotation generates weekly or daily schedules, and alerts route to the current on-call rather than a fixed person. Two fields in the rule config enable it:
escalation:
steps:
- wait: 5m
- wait: 10m
- wait: 15m
oncall: team-ops-weeklyThe result is that a page-out never dies with a single unread notification, and handoffs stop being an ad-hoc spreadsheet nobody updates.
A global sampling rate is a blunt instrument: either you miss slow requests or costs explode. v2.5.0 pushes sampling down to the service level—100% on core paths, 5% on edge services—and adds error-always-keep: any span carrying an error tag is retained regardless of the rate.
Config overrides the global default per service and hot-reloads without restarting collectors. It also spares you the embarrassment of a critical service happening not to be sampled during a slow-call investigation—errors and anomalies stay visible even when the tail is sampled away.
We also fixed a pagination cursor issue in log search over large time ranges, added ARM collector plugins to the Xinchuang image, and shipped 30+ new Grok templates for common formats.
Two upgrade notes: first, export your alert rules and sampling config as a backup, since a few config structures changed in this release; second, logs-to-metrics rules generate new time-series writes, so reserve time-series storage space ahead of time. Rolling upgrade is enough—no downtime required—and the new features are additive, so existing dashboards and alerts keep working unchanged.
For the full list of fixes and API changes, see the release notes in the documentation. We recommend upgrading collectors first, then the control plane, and only then enabling logs-to-metrics rules—this keeps ingestion stable while the new features settle in. Rolling back is a single version step, and config exported before the upgrade restores in minutes. If you are on v2.4.x, this release is fully backward compatible: dashboards, alert rules, and retention policies carry over untouched, so the upgrade is mostly scheduling a short rolling window and watching the ingest rate for a few minutes afterward.
Compatibility details worth knowing: the logs-to-metrics engine reuses the existing query parser, so any saved search can become a metric without rewriting; escalation chains work with every notification channel already configured, including custom webhooks; and per-service sampling accepts a percentage or an explicit always/never. If you have been holding off on enabling sampling because the global knob was too coarse, this is the release to turn it on.