← Back to blog
Changelog 4 min read 炬鲸团队

OBSERVE v2.5: Dynamic Thresholds, Data Masking, and Faster Collection

v2.5 ships history-based dynamic thresholds, automatic sensitive-data masking, lower agent resource usage, and query engine improvements, with upgrade steps and backward-compatibility notes.

Dynamic thresholds: stop paging at “CPU is 90%”

The problem with fixed thresholds is that they don't know what “normal” looks like for your business. CPU at 90% during the overnight batch is fine; CPU at 90% at noon is a problem. v2.5 adds dynamic thresholds: a baseline is computed automatically from the same time window over the past seven days, and an alert fires only when the metric deviates from that baseline by a configured margin.

The config gains a baseline option:

rules:
  - name: cpu-anomaly
    expr: cpu_usage{service="billing"}
    baseline: weekly      # computed from the same window over the past 7 days
    deviation: 2.0        # fire when 2x off the baseline

Dynamic thresholds work best on metrics with a clear cycle — high during the day, low at night; high on weekdays, low on weekends. During the cold-start period (the first two weeks of a new service) there isn't enough history, so run fixed thresholds first and switch to dynamic once two weeks of data have accumulated. Trying to run dynamic thresholds on a metric with no cycle just moves your false alarms around instead of removing them.

Automatic sensitive-data masking

The most common data-leak sources in logs: phone numbers, national ID numbers, bank card numbers, and tokens. Until now you had to write regexes on the collection side. v2.5 pushes masking down into the ingest pipeline, with built-in recognition for common sensitive types, out of the box:

masking:
  enabled: true
  rules:
    - type: phone       # phone number
    - type: id_card     # national ID
    - type: bank_card   # bank card
    - type: token       # JWT / token

Masking happens before the data is written, so what lands on disk is 138****1234 — the raw value never persists. You can add custom regexes for business-specific sensitive fields such as internal employee IDs. Existing production logs are unaffected: masking only applies to newly configured sources, so you can roll it out service by service without a big-bang migration.

Collection performance

Agent CPU usage drops about 30% on average, from three changes: regex parsing now uses a compiled cache, batch compression cuts system calls, and metadata injection no longer allocates objects repeatedly. At 20k lines/sec on a single node, agent CPU fell from 1.2 cores to 0.8, with memory down as well. That matters most on low-end domestic-stack servers — the entry Phytium and Kunpeng models — where the agent previously competed with the workload it was monitoring.

Query and storage improvements

  • The columnar store now uses ZSTD compression, cutting log storage cost by about 35% on average.
  • Queries spanning days or weeks run about 50% faster.
  • Dashboards can now export to PDF or image, so the on-call report is a direct export instead of a manual screenshot.

Fixes

  • Fixed a race where alert rules spanning midnight could fail to evaluate on the day boundary.
  • Fixed a token-refresh bug that caused collectors to see intermittent 401s.
  • Fixed a rendering hang in the trace view when hydrating very large spans.
  • Fixed SQL search escaping that returned wrong results for special characters in field values.

A note on enabling dynamic thresholds

When you turn dynamic thresholds on for an existing fixed-threshold rule, don't delete the fixed threshold immediately. Run them side by side for a week and diff which one fires first and which one you'd have ignored. The baseline needs that week of history to be trustworthy, and the side-by-side run tells you whether the dynamic threshold is actually catching things your fixed threshold missed, or just re-deriving the same number you already had.

The ZSTD change deserves one operational note: enabling the new compression means a one-time recompression pass over existing data when you upgrade. On large clusters, schedule it for a maintenance window rather than letting it compete with live queries during business hours.

Upgrade notes

This release is backward compatible and supports an in-place upgrade from 2.4 with no data format change. Upgrade order: storage → ingester → querier → gateway → agent. Dynamic thresholds and masking are optional and default to off after the upgrade; enable them as needed. Back up metadata before upgrading, and run ob-compat-check.sh afterward. Images for the Phytium + UOS + Kingbase stack are already synced to 2.5.