SQL-Like Log Search: Pin Down Incidents with a Single SELECT

Why log search should "speak SQL"

The biggest obstacle to searching logs is not data volume — it's the query language. Elasticsearch uses Lucene syntax, Splunk uses SPL, and Loki uses LogQL, each with its own rules: keywords need quotes, fields need colons, wildcards need *. Switch platforms and you forget everything. In most teams only one or two people actually know these dialects, so every production incident turns into a queue outside their desks. Those two people become a bottleneck, and their absence on a weekend means an outage drags on far longer than it should. There's a quiet training cost too: every new hire has to learn the dialect before they can contribute to an incident at all.

Observe's log search takes a plainer route: it supports SQL directly. If you can write SELECT ... WHERE ..., you can query logs — and almost every backend engineer can write SQL. What it saves is not the time spent learning a syntax, but the minutes you waste hunting through docs and trying queries mid-incident. SQL is also the one language your whole team already shares, which is exactly what you want at 2 a.m. during an outage, when nobody wants to recall which brackets go where.

What SQL-like search looks like

To find "ERROR logs from the payment service in the last hour, newest first, 100 rows," you write:

SELECT * FROM logs
WHERE service = 'pay-service'
  AND level = 'ERROR'
  AND ts > now() - 1h
ORDER BY ts DESC
LIMIT 100

What's supported today:

• WHERE: field equality, ranges, LIKE pattern matching, IN sets
• Time filters: relative time like ts > now() - 1h, or absolute ranges
• Aggregation: GROUP BY level, COUNT(*) gives an error distribution instantly
• ORDER BY / LIMIT / OFFSET pagination

The key difference from traditional syntax: field names are the actual field names in your logs — there is no separate mapping table to memorize. What you see in the raw log is exactly what you filter on, so the mental model stays flat.

Wired into distributed tracing

Finding an error log is only the start. Once you spot it, you usually need the whole call chain behind it. Click any log row and jump straight to the tracing page by trace_id, where you can see every service the request touched, each segment's latency, and the state of every span. The jump is not a separate search — the log row carries its trace context, so one click takes you there.

In the other direction, when a span fails in tracing, you can drill down into the logs it emitted. Bidirectional jumping between logs and traces is the two biggest time-savers in production debugging, because most incidents need both the "what happened" of logs and the "where in the chain" of traces. Having both in one product means you don't copy a trace id into a second tool and hope the time ranges line up.

Practical techniques

• Aggregate first, then inspect: don't scroll raw rows — start with GROUP BY level or aggregate by service to see where errors cluster before drilling into any single line.
• Narrow with LIKE: for stack traces, message LIKE '%NullPointerException%' is far faster than full-text search, and it returns exact matches rather than ranked guesses.
• Mind the default time window: queries carry a default range; logs older than it won't show up, so check the window first when a query comes back unexpectedly empty.
• Save query templates: store the few queries you run constantly and reuse them, or share them with the team so new hires don't start from zero.

How it stays fast under the hood

SQL-like queries are first parsed into a structured form, then pushed to time-partitioned indexes backed by field-level inverted indexes and columnar aggregation, so a single query returns in seconds even across tens of billions of rows. On query timeout, the engine degrades to a sampling mode to protect availability rather than erroring out and blocking the whole console.

SQL-like search is not a gimmick — it exists so everyone on the team can query logs themselves, instead of always waiting for the one person who remembers the query syntax.