Query Logs Like SQL: Inside JuJing OBSERVE Log Search

Why grep doesn't scale

The most common move when debugging production is grep for a keyword, then tail a few hundred lines of context. That works on small logs, but three problems grow quickly with log volume.

First, grep is a sequential scan with no index. Scanning a 50 GB single-day log file takes minutes; run a few of those scans and debugging becomes waiting. Second, grep only does text matching — it can't do structured filtering. "Find 5xx requests whose duration exceeded 2 seconds" is not expressible in grep. Third, grep results can't be aggregated. If you want per-service error rates or an error-over-time histogram, you have to write a separate script.

JuJing OBSERVE treats logs like a table you can query: define fields first, then let the search engine understand filtering, sorting, and aggregation.

Syntax: filter, pipe, aggregate

A query has three stages — filter, process, aggregate:

status >= 500 AND service = "order"
| parse duration_ms
| stats count() by service, status

• Filter: status >= 500, service = "order", message like "%timeout%", with AND/OR/NOT and parentheses.
• Pipe: parse extracts fields from text, extend derives new fields, where re-filters.
• Aggregate: stats, bucket, top return results directly — no export to a spreadsheet.

Fields get automatic type inference. duration_ms=183 in a log line becomes a number, not a string; status=500 an integer; timestamp a time. That's what makes status >= 500 a numeric comparison instead of a string one.

Indexing: no brute-force scanning

The engine does two things at write time: partition by time and build inverted indexes on field values.

• Time partitioning means "last 15 minutes" reads only two partitions instead of the whole store.
• Inverted indexes turn service = "order" into a direct lookup of matching document IDs, dropping complexity from O(n) to roughly O(matches).
• High-frequency fields are indexed by default; cold fields can be enabled on demand to avoid index bloat slowing down writes.

On a single node with 8 cores and 32 GB RAM, a two-condition filter over one billion log lines holds P99 latency under 800 ms.

Four queries you can use today

1. Per-service 5xx error rate:

status >= 500 | stats count() as err by service

1. Top 10 slow requests:

parse "duration_ms=%{number:ms}" | where ms > 2000 | top ms limit 10

1. Errors bucketed by 5 minutes, to pinpoint when an incident started:

status >= 500 | bucket span=5m | stats count() by _time

1. Correlate two services: grab the gateway's trace_id, then filter downstream logs with it to reconstruct one request's full path.

Save common queries as templates and attach them to alerts or dashboards. When something breaks, you click once instead of hand-writing syntax under pressure.