An overview of Torchwhale OBSERVE's three core modules — log management, monitoring and alerting, and distributed tracing — plus how trace_id, span_id, and host correlate them for faster incident resolution, with a phased rollout path.
Torchwhale OBSERVE puts log management, monitoring and alerting, and distributed tracing in a single platform with shared data and a unified query layer. This article explains what each module solves and how they fit together, so you can decide where to start.
The logging module answers a simple question: where is the failure happening right now? The collection agent supports Filebeat-style glob paths, multi-line merging for Java stack traces, timestamp parsing, and automatic labels derived from Pod and Namespace. Once ingested, logs land in an index layer where you can filter by field, run full-text search, and extract structured fields with Grok.
The query syntax is close to SQL:
level:ERROR AND service:order-svc AND time:[now-1h TO now]Because the syntax also supports aggregation, you can count and group — for example, top error types per service in the last hour — straight from the search box, without shipping the data anywhere else.
A common first win is turning a pile of unstructured application logs into field-addressable records. Start by extracting level, service, and trace_id; everything downstream, alerting and trace correlation included, builds on those three fields. Field extraction is configured per log source with a Grok pattern or a flag for already-structured JSON. Indexes roll over daily and are isolated per project, and cold data can be archived to object storage, which keeps search fast while controlling cost.
The metrics module integrates with Prometheus. You can push metrics over Remote Write or scrape exporters directly, so your existing Prometheus exporters keep working unchanged. Alert rules are written in PromQL, and notifications go to email, WeCom, DingTalk, or a generic webhook.
A typical rule looks like this:
rate(http_requests_total{status=~"5.."}[5m]) > 0.05We split alerts into four tiers, P0 through P3, and pair them with on-call schedules. P0 pages a human immediately; P3 lands in a queue reviewed during working hours. This separation is the single most effective defense against alert fatigue we have seen, because it keeps harmless noise from drowning out the alerts that actually matter.
The tracing module treats OpenTelemetry as the single standard and stays compatible with the Jaeger and Zipkin protocols. Context propagation follows the W3C Trace Context specification, so a trace that begins in a browser request can be followed across services written in different languages without losing its identity.
Sampling is configurable per service, so you can keep 100% of failed requests and a small percentage of healthy traffic. That keeps trace storage affordable while still capturing the requests you actually need to debug.
Logs, metrics, and traces are not three isolated systems. We join them through three fields: trace_id, span_id, and host. The standard path for resolving a production incident looks like this: an alert fires, you open the linked trace, find the slow span, and drill into that host's logs.
Consider a concrete case: an alert says the checkout service's error rate crossed 5%. You click the alert, open the trace for a failed checkout, see that the slow span is a call to the payment gateway, then jump to the gateway host's logs and find the connection timeout. The whole path takes a minute or two, because you never leave the platform. The same fields also power the reverse direction: while reading a log line that contains a trace_id, one click reopens the full trace, so you never have to re-run a search you already did. Correlation works in the metrics module too: a spike in a latency chart can be pinned to the specific spans and logs that caused it, rather than left as a number with no story behind it.
Our advice is to start with the logging module, get collection and search working end to end, then add metrics and tracing, and finally turn on the correlations. Each step delivers value on its own, so there is no need to do everything at once.