JJHub OBSERVE on Xinchuang: Kylin/UOS with Phytium, Kunpeng, and Hygon

This post records what it actually took to move every OBSERVE component onto a domestic (xinchuang) stack, including the pitfalls. It's written for infrastructure leads responsible for xinchuang rollout.

1. Scope and architecture

A typical xinchuang environment combines Kylin V10 or UOS for the OS, Phytium (ARM64) or Kunpeng (ARM64) or Hygon (x86_64) for the CPU, and DM or KingbaseES for the database. Our adaptation strategy is to change the runtime base, not the business logic: the collector, storage layer, and query layer ship dual-architecture ARM64 and x86_64 images, and the metadata layer has dialect support for DM and KingbaseES.

Both single-node and clustered deployments are supported. Under ~500GB of daily logs, a single node running three processes (collector, storage, query) is enough; above that, the storage layer scales out to distributed mode. The key decision behind this was separating metadata from time-series data early — that split is what made the database port a matter of days rather than months, and it keeps query performance off the relational engine entirely.

2. CPU: two things that bite on ARM64

The most common migration problems on Phytium/Kunpeng aren't compilation — they're these two:

1. Byte order and alignment: a few C extensions carried over from the x86 era trip alignment faults on ARM, so suspect any module that manipulates raw memory directly.
2. JDK selection: on ARM64 you must use a vendor-provided or validated JDK (Bisheng JDK, or an aarch64 OpenJDK build). Running the default x86 JDK through a translation layer roughly halves performance.

For benchmarking we ran identical write and search workloads — a write-heavy stream of roughly 500k log lines per second plus a concurrent mixed query load — on equal-core-count nodes. Kunpeng 920 delivered about 80%–95% of the throughput of a same-generation x86 box, with the gap driven mostly by single-core speed. Horizontal scaling closes it, but plan capacity accordingly rather than assuming 1:1 parity when migrating an existing fleet. Hygon, being x86_64 itself, needed essentially no CPU-side work; the effort there landed on the OS and database layers instead.

3. OS: three notes for Kylin/UOS

• systemd differences: Kylin and UOS ship slightly different systemd versions. Use the system's own service template rather than pasting CentOS init scripts.
• OpenSSL version: domestic systems tend to run older OpenSSL. Set TLS between collector and storage nodes to 1.2 rather than defaulting to 1.3.
• glibc compatibility: build binaries against Kylin V10's glibc and stay forward-compatible with UOS; compiling on a newer glibc and running on an older system produces missing-symbol errors at startup.

4. Database: the DM/KingbaseES dialect checklist

Metadata (users, alert rules, dashboard configs) lands in DM or KingbaseES, and the changes concentrate on:

1. Replacing AUTO_INCREMENT with sequences;
2. Adapting pagination syntax to each engine's LIMIT/OFFSET dialect;
3. Aligning string-type lengths and default collations, since indexed fields need consistent collation.

Logs and metrics — the bulk time-series data — never touch a traditional relational store. They go to our columnar storage engine, which is pure Go/C and highly CPU-instruction-set-sensitive, so we ship a separately optimized ARM64 build. Because only a thin metadata layer depends on the relational database, database domestication has almost no effect on query performance. That's the direct payoff of separating metadata from time-series data.

5. A reusable acceptance checklist

Before going live, verify the following:

1. Run a full write + search load test on Phytium, Kunpeng, and Hygon, confirming throughput is in an acceptable range.
2. Do a clean install and reboot self-check on both Kylin V10 and UOS.
3. Run a primary/standby switchover drill on DM/KingbaseES, confirming no metadata loss and automatic recovery.
4. Remove all x86 dependencies across components and confirm ldd reports no missing libraries.
5. Run a 72-hour stability test, watching ARM64 nodes specifically for occasional alignment crashes.

One final note from the field: the certification paperwork often takes longer than the engineering. If you're under a deadline, start the adaptation and the domestic-hardware certification application in parallel, not sequentially.