Migrating an Observability Stack to Xinchuang: A Field Checklist for Kunpeng and Phytium

Why Observability Is the Hard Part of Xinchuang

Replacing business apps on domestic CPUs usually only touches the application layer. An observability platform, by contrast, reaches down into the kernel and container runtime: collectors run on every node, agents must match glibc and kernel versions, and storage engines lean on CPU instruction sets. It has many components, dense dependencies, and when it goes down, the whole company's alerting and triage goes down with it.

We migrated OBSERVE end-to-end onto three domestic chips: Kunpeng 920 (ARM), Phytium S2500 (ARM), and Hygon C86 (x86-compatible). Below is a reusable checklist.

Step 1: Multi-Arch Image Builds

Collectors and service components all move to multi-arch images. buildx produces three variants with one command:

docker buildx build   --platform linux/amd64,linux/arm64   -t registry.local/observe/agent:3.2.1   --push .

Key points:

• Use a multi-arch base image like debian:bookworm-slim instead of hand-assembling arch tags.
• Components with CGO (e.g. some eBPF collectors) must be compiled per architecture. When cross-compiling BPF targets on ARM, mind the endianness difference between -target bpfel and bpfeb.
• Push to Harbor as a manifest list; K8s nodes pull the right variant for their architecture automatically.

Step 2: Dependency and Runtime Compatibility

Three classes of issues came up most:

1. glibc / kernel version. Some Phytium machines ship an older kernel with incomplete eBPF support. Run bpftool feature probe on the target first; if kprobe and tracepoint are unavailable, fall back to /proc-based collection instead of eBPF.
2. Compression and crypto instructions. Kunpeng has NEON, Hygon has AVX2 — but don't assume hardware acceleration is on everywhere. Use portable software implementations (e.g. zstd portable) for compression and trade a little throughput for portability.
3. JVM flags. For storage and compute services on the JVM, G1GC behaves differently on ARM. Re-tune heap and GC thread counts after benchmarking instead of copying x86 flags.

Step 3: Benchmarks and Capacity Recalculation

Migration isn't done when it "runs." You must redo capacity planning. Our collector benchmark (single node, 10k logs/s):

| Metric | x86 baseline | Kunpeng 920 | Phytium S2500 |
| --- | --- | --- | --- |
| Ingest throughput | 100% | 92% | 87% |
| CPU usage | 100% | 108% | 115% |
| Memory usage | 100% | 96% | 99% |

ARM cores are weaker individually but more numerous. After benchmarking, raise collector replica counts by 10–20% based on real throughput. Keep storage shard count unchanged, but reset the rebalance threshold used when migrating shards.

Step 4: Rollback and Canary

Don't do the swap in one shot. Our canary order:

1. Migrate collectors first (stateless, easy to roll back) and watch data continuity.
2. Migrate query/alerting services, running alongside old x86 instances in dual-write mode and comparing results.
3. Migrate the storage engine last, moving shards via snapshots while keeping the old cluster read-only.

Every step keeps an "x86 switch-back" lever: change weights at the DNS or LB layer to roll back instantly, with no data-layer restore required.

Checklist Summary

• Multi-arch images with manifest lists; nodes auto-match.
• Probe the target with bpftool feature probe and degrade collection by capability.
• Use portable compression/crypto; don't bet on a single architecture's instruction set.
• Re-tune JVM/GC on ARM and recalculate capacity after load tests.
• Canary in three stages (collector → services → storage), keeping a rollback lever at each.

Run this checklist cleanly and the Xinchuang swap for observability is controlled. The real work is benchmarking and parameter tuning, not compilation.