Migrating from ELK to a Xinchuang-Native Observability Platform

Take stock before you move

Before migrating off ELK, inventory what you have: how many indices, daily ingest volume, retention days, alert rules and dashboards, and how many apps ship logs via Filebeat/Logstash. Our baseline: roughly 2 TB/day, 30-day retention, about 200 alert rules, and 60 applications. Write this down - it determines both the migration order and your rollback plan.

The lesson we learned: migrate alerts and dashboards first, because that's what the team stares at daily. Then move the data plane (the Filebeat/Logstash destinations), and only after that tackle historical data. Reversing the order means you either lose visibility during the cutover or spend weeks rehydrating old data before anyone sees value.

Deploying on a Xinchuang stack

The target environment was Kylin V10 SP3 on Kunpeng 920, running JJHub OBSERVE. Three things mattered most during deployment:

1. Architecture: Kunpeng is ARM64, so every image and binary must be aarch64. Running x86 packages directly is the single most common first-day failure - verify before you start.
2. Storage: with heavy log volume, use NVMe SSDs on a separate mount for data and keep them off the system disk. Log ingestion is write-heavy, and a shared disk will throttle both the OS and the indexer.
3. Kernel params: raise vm.max_map_count and nofile, or indexing will OOM repeatedly under load. Set these in the provisioning image so every node gets them uniformly.

One install command brings up control-plane and data nodes, and the cluster shows up in the console right after registration. Before the real cutover, run a smoke test - a small load generator writing representative logs - to confirm the cluster indexes, queries, and alerts correctly on ARM64. A practical expectation to set: domestic CPUs have lower single-core throughput than same-generation x86, so indexing rate will be lower per node. Plan for that by adding data nodes to spread the write load rather than chasing a single oversized box. Record the exact versions of Kylin, Kunpeng firmware, and the platform you validated - compliance reviews will ask, and having it written down saves you from re-deriving it later.

If you operate under classified-protection (等保) or industry compliance, treat it as part of deployment, not an afterthought. Verify that the platform's audit logging, access control, and retention satisfy the relevant requirements before you migrate anything onto it.

Migrating indices and alerts

Create indices by log type - application, audit, access - and keep field mappings aligned with the old ELK setup so downstream query scripts don't break. A field that changes type or name silently breaks dashboards and saved searches, so diff the mappings rather than eyeballing them.

Alert rules are the heavy lift. Convert ELK watchers and alerts into OBSERVE alert policies one by one, and double-check three things:

• Thresholds and "for" durations
• Aggregation windows (ELK buckets and OBSERVE windows have subtly different semantics)
• Notification channels and on-call schedules

Export an alert inventory and verify line by line. Batch-importing and discovering later that notifications never reached anyone is the classic failure mode of this migration.

Dual-write, cutover, and pitfalls

Run dual-write through the cutover so you lose nothing and can always roll back:

1. Weeks 1-2: write new data to both ELK and OBSERVE, compare volume differences
2. Week 3: switch alerts and dashboards to OBSERVE, spot-check for a few days to confirm no missed alerts
3. Week 4: once stable, stop writing new data to ELK and migrate or archive history as needed

Validate consistency during dual-write with volume diffs and sampled comparisons - don't trust "it looks the same" without numbers. Keep ELK read-only for a month after cutover as a rollback backstop, then decommission.

Pitfalls we actually hit: ARM64 image incompatibility (run a smoke test before full rollout); insufficient file handles freezing indexing (tune kernel params uniformly at deploy time); alert window semantics causing missed alerts (verify one by one, never assume); and historical migration taking far longer than expected (plan hot/cold tiers up front - migrate only the last 30 days and archive the rest). None of these are fatal if you know they're coming; all of them are painful if you don't.