Tracing in 5 Minutes: Wiring OpenTelemetry into OBSERVE

Why OpenTelemetry

The most common tracing mistake is binding an application directly to one vendor's SDK, then rewriting instrumentation when you switch platforms. OpenTelemetry (OTel) decouples collection from storage: applications use a single API, and a Collector fans the data out to multiple backends. For OBSERVE this means you only change the Collector's exporter config—no application code changes at all. It also future-proofs the investment: once your services speak OTLP, moving between backends is a config change, not a code change.

One thing to get right up front: the Collector should be a dedicated deployment (one per environment, ideally one per region), not something co-located with each app. Apps send to the Collector over the local network; the Collector batches and ships to OBSERVE over a single authenticated connection. This keeps app config simple and gives you one place to add processors (sampling, redaction, batching) later.

Step 1: Deploy the Collector

The Collector receives trace data from your apps, batches it, and forwards it. A minimal Docker instance needs an OTLP receiver and an exporter pointed at OBSERVE:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
processors:
  batch:
    timeout: 5s
    send_batch_size: 1024
exporters:
  otlphttp:
    endpoint: https://ob.jjhub.cn/otlp/v1/traces
    headers:
      Authorization: "Bearer <your access token>"
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp]

Create the access token in the OBSERVE console under "Access Management → Access Tokens"—one token per project keeps data isolated and makes revoking a compromised token cheap. Expose ports 4317 (gRPC) and 4318 (HTTP) only to your app network, never publicly; the Collector's outbound connection to OBSERVE is the only one that should leave the boundary. After starting, run docker logs otel-collector; the line Everything is ready means the config loaded cleanly.

Step 2: Instrument the application

For Java, use the OpenTelemetry Java Agent—no code changes, just attach it at startup:

java -javaagent:opentelemetry-javaagent.jar   -Dotel.exporter.otlp.endpoint=http://localhost:4317   -Dotel.service.name=order-service   -Dotel.traces.exporter=otlp   -jar order-service.jar

For Go, use go.opentelemetry.io/otel and instrument manually. The two places that matter most are the HTTP entry point and database calls—without them your traces have no context to hang onto:

import (
    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

handler := otelhttp.NewHandler(mux, "order-service")

Don't chase full automation on day one—covering the entry point, outbound HTTP, and SQL layers already answers 80% of troubleshooting questions. Add the rest (message queues, cache clients) only as you actually need them; every extra layer is maintenance you sign up for. Where manual instrumentation earns its keep is in span attributes: tagging a span with orderId, userId, or channel turns a generic latency spike into a searchable question—"is it one customer or everyone?"—with no extra logging. Set these on the root span so every child inherits them.

Step 3: Verify traces

Fire a few real requests, then search by service name in the OBSERVE "Tracing" page. Confirm three things: spans appear, parent/child relationships are correct, and per-span latencies look sane. Also confirm the service.name resource attribute is set on every span; the console groups by it, and a missing or inconsistent value is why traces sometimes show up under "unknown_service". A useful trick is to add a request header that forces a sampled trace (most SDKs support one), so you can point at one specific request instead of waiting for random sampling to catch it.

If nothing shows up within 30 seconds, check in order: is the token correct, can the app reach the Collector (telnet <collector-ip> 4317), and is the app actually generating traffic? Sampling is a classic silent killer—if your SDK defaults to 1% sampling, you'll wait a long time for a trace in a low-traffic dev environment.

The three most common pitfalls

First, instrumenting only the entry point but not outbound calls leaves you with a single-node trace that can't show where the downstream slowness is. Second, failing to propagate trace context splits cross-service calls into two fragments—check for missing HTTP header propagation (the traceparent header). Third, inconsistent timestamps scramble span ordering; use UTC everywhere and keep NTP synced. A fourth, less obvious one: a single Collector shared by prod and staging will mix their spans—use separate tokens or a resource attribute to keep them apart. Working through these resolves nine out of ten "no data" cases.