Observe OBSERVE v2.4: SQL Search, Alert Aggregation, and Domestic Stack Enhancements

Observe OBSERVE v2.4 is out. This release graduates SQL-based log search, adds alert aggregation, and extends domestic-stack support.

New features

SQL-based log search (GA)
Log search upgrades from keyword mode to full SQL: SELECT / WHERE / GROUP BY / ORDER BY / LIMIT, field aggregation, multi-condition filtering, and time-window analysis are all writable directly in the search page. Queries can be saved as views and converted into alerts. New aggregate functions PERCENTILE and DATE_FORMAT make long-tail latency analysis and time bucketing easier. Switching from keywords to SQL has a learning curve, so we kept the original keyword entry—both coexist and you can migrate gradually.

Alert aggregation
High-frequency alerts are the fastest way to “flood the channel.” Alert rules now support aggregation by dimension (e.g. host, region, error_code): events sharing a dimension are merged into a single notification with a count and first/last timestamps. A database outage that makes 20 machines time out at once now produces one alert with the list of affected hosts attached. The aggregation window and merge strategy are configurable per rule.

Tail sampling for tracing
Policy-based tail sampling arrives: three policies—by error flag, by latency threshold, or by specific service—keep every trace that’s errored or timed out, while normal requests are sampled at a ratio. Storage costs stay down without missing the slow paths that matter. Tail sampling runs in the Collector with zero changes to your business processes; just enable the policy in the console. Sampling can be overridden per service instead of being global.

Domestic-stack expansion
New support for Loongson 3A5000 / LoongArch, DM8 as a metadata store, and full validation of Kylin V10 (LoongArch) and UnionTech UOS 1060. We also shipped Loongson agent binaries and container images, so domestic users can pull them directly.

Performance and capacity

• Single-node log write throughput up from 40k to 60k lines/sec (measured on Phytium FT-2000+).
• GROUP BY aggregation P95 over a billion rows down from 2.8s to 2.1s.
• With DM8 as the metadata store, config-query latency is on par with MySQL.

Improvements

• Log write throughput up ~30%, thanks to columnar batch-write optimization.
• Search pagination is now cursor-based; paging past 100,000 results no longer stutters.
• The collection agent supports resume-from-breakpoint, so logs aren’t lost during network blips.
• Dashboards support cross-source correlated queries—logs and metrics can be compared on the same chart.
• Better search autocomplete: field names and functions are suggested as you type, lowering the learning curve.

Fixes

• Fixed GROUP BY on time fields mis-bucketing across midnight.
• Fixed alert silence windows occasionally not taking effect.
• Fixed the agent stalling on Kylin V10 due to file-handle limits.
• Fixed type inference on some JSON log fields causing skewed aggregation results.

Known issues

• On clusters with more than 500 services, tail-sampling policy sync can lag by up to a minute; fixed in the next patch.
• On Loongson, PERCENTILE shows slight precision drift at very high volumes; use APPROX_PERCENTILE for now.

Upgrade notes

• Back up metadata and alert configuration first; this upgrade migrates the search index schema. Expect roughly 10 minutes per 100 million log rows.
• Users with custom SQL views: view fields auto-map after upgrade with no manual changes. If a field drops, re-save the view once.
• Domestic environments should use the new ARM64 / LoongArch packages; the x86 package is incompatible with previous versions.
• For multi-node clusters, upgrade in order—data nodes first, then query nodes—to avoid a brief search outage.

See the docs “Version History” for the full changelog. For upgrade issues, open a ticket—we respond by impact level.

What's next

v2.5 is already in progress, focused on metric anomaly detection and a rebuilt waterfall view for traces. If you have feedback on v2.4—especially the new SQL search—open a ticket or post in the community; we prioritize by how often a feature is actually used in production, not by how loudly it's requested.