Deploying OBSERVE on Kunpeng, Kylin, and Dameng: A Field Guide

Environment and versions

The stack is a Kunpeng 920 (64-core ARM64), Kylin V10 SP3, Dameng DM8, with TongWeb replacing the middleware. Moving from x86 to ARM means every image and binary has to be re-validated—you can't reuse the old x86 deployment packages. Before starting, confirm three things: every dependency has an aarch64 build, the database driver is compatible, and the monitoring agent supports ARM.

Dependency replacement

• Use BiSheng JDK (Huawei's ARM-optimized OpenJDK build); in load testing, GC pauses measured about 20% lower than stock OpenJDK.
• For the container runtime, use Kylin's iSulad or a containerd with Docker compatibility; confirm the kernel supports the overlay2 storage driver first.
• Standardize on NTP for time sync. Clock drift on ARM servers visibly distorts trace timelines, so calibrate before going live.

Handle images early: many third-party components only publish x86 images, so you'll have to rebuild them on ARM yourself. Build multi-arch images with buildx and push them to an internal Harbor, so production never has to pull from a slow public registry at deploy time.

Dameng database adaptation

OBSERVE's metadata store defaults to MySQL. Moving to Dameng means swapping the JDBC driver and adjusting SQL dialect. Dameng's MySQL compatibility mode (COMPATIBLE_MODE=4) works well—enable it and turn off case sensitivity:

CREATE DATABASE observe CASE_SENSITIVE=0 COMPATIBLE_MODE=4;

Migration happens in four steps: use Dameng's DTS tool to migrate the MySQL schema and data, verify the column-type mappings one by one, swap the connection string and driver in the app, then run a full regression. Focus on three things:

1. Pagination: Dameng accepts LIMIT, but LIMIT inside complex subqueries must be rewritten to ROWNUM or TOP.
2. Column types: replace TEXT / JSON with CLOB / VARCHAR2, sized to your actual log-metadata needs.
3. Indexing: Dameng's Chinese full-text search uses its own CONTEXT index—build it before loading data, or the later index rebuild is painfully slow.

Performance tuning

Load testing on ARM surfaced two bottlenecks. First, Kylin's default swap is too aggressive—lower swappiness to 10. Second, Dameng's read-ahead and page size need adjustment: arm64 defaults to 4K pages, and enabling 64K hugepages improved write throughput by about 30%.

echo "vm.swappiness=10" >> /etc/sysctl.conf && sysctl -p

Configure hugepages together with the kernel boot parameter and Dameng's HUGE_PAGE setting, then reboot and verify—don't change only half of it. Dameng also benefits from enabling asynchronous IO (AIO) and enlarging the redo log buffer, both of which help a lot in a write-heavy observability workload.

Pitfall checklist

1. Pull images into an internal Harbor first and distribute from there—domestic registries are slow.
2. Match the Dameng JDBC driver version to the DM8 minor version, or you'll hit sporadic connection leaks.
3. Kylin ships SELinux enabled by default—open the collector's 4317 / 4318 ports.
4. Run a full end-to-end load test before sign-off, focusing on trace sampling rate and log flush latency, not just functionality.

Verification and handover

Before sign-off, run a few checks beyond functional tests. Confirm Dameng metadata writes survive a restart, verify log retention actually lands on the object-storage tier after the configured window, and check that traces stay intact across a service restart. Re-run the load test after enabling hugepages and AIO, because those tuning changes can interact with the JVM and the container runtime in non-obvious ways.

If you swapped the middleware for TongWeb, keep its thread-pool and timeout settings in sync with the old middleware—otherwise the collector's HTTP receivers can hit connection resets under load. Document the final architecture, including OS, JDK, DB, and middleware versions, so the next environment rebuild doesn't start from scratch.

Finally, if your internal Harbor or NTP server uses a certificate issued by a domestic CA, import the CA bundle into the JVM truststore and the container runtimes before first boot. A missing root CA is a classic first-boot failure in domestic environments and can eat hours of debugging that have nothing to do with your application.

Plan the rollout in phases: stand up a single-node pilot on the domestic stack first, run it for a week with real traffic, then scale out. The first ARM deployment always surfaces gaps you never saw on x86—a missing aarch64 library, a timezone file, a container runtime flag—and it's far cheaper to find them on one node than across a cluster.