What log search, alerting, and distributed tracing each do, how they fit together, and config tips you can apply today — including sampling strategy and alert noise reduction for ops and developers.
Logs, metrics, and traces are three different datasets, but during an incident they all point at the same question: where is the system broken? Used in isolation, each one only shows part of the picture. This article walks through the three core capabilities of JJHub OBSERVE and gives you configuration you can copy directly.
Incident investigation almost always starts with logs. JJHub OBSERVE's log search supports full-text search, field filtering, and SQL-like aggregation in one place, so you don't have to bounce between a search box and a separate BI tool. The query below groups ERROR logs by host, showing at a glance which machine the problem is concentrated on:
SELECT host, count(*) AS cnt
FROM logs
WHERE level = 'ERROR' AND service = 'order-service'
GROUP BY host ORDER BY cnt DESCThe search syntax is also Lucene-compatible, so level:ERROR AND service:order* works out of the box for anyone coming from Elasticsearch habits. Aggregations can pivot by any field — by host, by error type, by release tag — turning "what is happening" into "where exactly is it happening." For queries spanning more than a day, the UI suggests sampling: look at 1% first, then zoom in. That keeps response times stable instead of letting a full scan drag down the cluster, which is the usual way a large log store tips over under load.
Two details are worth calling out. First, log context: clicking any ERROR line reconstructs the 50 lines before and after it, so you don't hunt for surrounding context by paging. Second, trace_id correlation: when logs carry a trace_id, a single error can jump straight to its full trace, so logs and traces stop being two disconnected silos.
The biggest problem with alerting isn't missed alerts — it's noise. When everything pages you, nothing does. JJHub OBSERVE uses the Prometheus metric model, so PromQL works directly in alert rules and existing Grafana dashboards migrate over without rewriting. Beyond plain thresholds, a few capabilities cut the noise dramatically:
A typical rule: notify on-call only when CPU stays above 90% for three consecutive periods, rather than paging on every flicker. Attach the last 15 minutes of the metric curve to the alert body, so the person taking the call doesn't have to open a laptop first. That single change — putting the chart in the notification — removes a whole round-trip from every incident, and it's usually the first thing teams tell us made on-call bearable again.
The value of tracing isn't "we have traces" — it's quickly answering "where is this request slow?" JJHub OBSERVE fully supports the OpenTelemetry protocol, so a standard agent or SDK is all the instrumentation you need. The trace view sorts by service, endpoint, and latency, and the flame graph pinpoints the specific SQL query or downstream HTTP call, down to the exact milliseconds a span spent waiting on a slow dependency.
For sampling, we suggest 10% on normal traffic and 100% on errors and slow requests. With tail-based sampling, every request that ends badly is retained even though the decision is made after the fact — which is exactly the data you need most during an incident, and precisely what head-based sampling throws away.
Logs, metrics, and traces each have blind spots on their own: metrics say "something broke," logs say "what went wrong," and traces say "which step was slow." JJHub OBSERVE connects them, so a single alert carries the linked metric curve, log snippet, and trace link — no more hopping between three systems during an incident. That correlation is the dividing line between a monitoring stack that shows you dashboards and an observability platform that answers questions.