Five-Minute Onboarding: Shipping Go Traces to OBSERVE with OpenTelemetry

Prerequisites

Assume you have a Go service running in Kubernetes that already emits structured logs. The goal is to send all three signals—traces, metrics, and logs—into OBSERVE, and have them link to log search and alerting.

First, get the ingest endpoint and token: Console → Onboarding → OpenTelemetry generates an OTLP endpoint with a token, shaped like https://ob.jjhub.cn/otlp, with the token carried in the Authorization HTTP header.

Initializing the SDK

Use the official SDK and initialize the trace and metric providers in main:

res := resource.NewWithAttributes(
    semconv.SchemaURL,
    semconv.ServiceName("order-service"),
    semconv.ServiceVersion("v1.2.3"),
    attribute.String("deployment.environment", "prod"),
)

ctx := context.Background()
exp, _ := otlptracehttp.New(ctx,
    otlptracehttp.WithEndpoint("ob.jjhub.cn"),
    otlptracehttp.WithHeaders(map[string]string{"Authorization": token}),
)
tp := sdktrace.NewTracerProvider(
    sdktrace.WithBatcher(exp),
    sdktrace.WithResource(res),
    sdktrace.WithSampler(sdktrace.ParentBased(sdktrace.TraceIDRatioBased(0.1))),
)
otel.SetTracerProvider(tp)

Three things matter here. Set ServiceName—it is the key OBSERVE uses to aggregate by service. Use deployment.environment for the environment, not a custom key, or the trace view's environment filter won't line up. Start sampling at 0.1 in production; full sampling will eat storage and network budget.

Instrumentation: HTTP and databases

For RPC/HTTP, wrap your handlers with the otelhttp middleware and you get automatic spans. For databases, use otelsql or the driver's own instrumentation. Where you need a manual span in business logic, use tracer.Start:

tr := otel.Tracer("order-service")
ctx, span := tr.Start(ctx, "create_order")
defer span.End()
span.SetAttributes(attribute.Int("order.id", id))

The key move is attaching business IDs (order number, user ID) with SetAttributes, so you can later search traces by those attributes and pin them to a concrete business object.

Correlating logs

To jump from a log line to its trace, inject traceId / spanId into the log records. With zap or logrus, add an OTel hook that writes the current span's traceId into every line. Then a log line in OBSERVE shows which trace it belongs to, and the trace view links back into the logs.

Exporting metrics

Traces alone won't tell you the error rate. Set up the metric exporter alongside the trace exporter:

mexp, _ := otlpmetrichttp.New(ctx,
    otlpmetrichttp.WithEndpoint("ob.jjhub.cn"),
    otlpmetrichttp.WithHeaders(map[string]string{"Authorization": token}),
)
mp := sdkmetric.NewMeterProvider(
    sdkmetric.WithReader(sdkmetric.NewPeriodicReader(mexp)),
    sdkmetric.WithResource(res),
)
otel.SetMeterProvider(mp)

The periodic reader flushes every 60 seconds by default. For HTTP servers, the otelhttp middleware emits request count, duration, and in-flight requests automatically; for your own counters, create a meter and call Add on the instruments. Give every instrument a unit so OBSERVE can render axes and rate conversions correctly.

Verify the data landed

After starting the service, generate a few requests and check the console: Services should list order-service, the Trace view should show spans carrying your attributes, and Metrics should show the HTTP request series. If traces are missing but the process started, read the exporter logs—the usual causes are an unreachable endpoint or an expired token. Keep the exporter's debug logging on for the first few minutes, then turn it off.

Common traps

• Forgetting the resource ServiceName, so everything lands under a node named unknown_service.
• Using an HTTP exporter while the cluster can't resolve the OTLP endpoint—verify network reachability with curl first.
• Sampling too low, leaving no traces to inspect during an incident; keep an always-on branch for error requests.
• Histogram metrics with no configured buckets, where the default boundaries distort latency distributions. Set separate bucket boundaries for QPS and for latency.
• Metric instruments without a unit render incorrectly on dashboards—set a unit on every counter and histogram.

Reading exporter errors

When telemetry doesn't show up, the exporter's own logs are the fastest clue. The common failures, in order of frequency: connection refused (wrong endpoint or port, or an egress network policy), 401 (token expired or missing from headers), and deadline exceeded (batches too large or the network too slow—raise the batch timeout or lower the batch size). OBSERVE also exposes a self-check endpoint you can hit from inside the cluster to confirm reachability before blaming your code.