Migrating from ELK to a Xinchuang Observability Stack: Inventory, Dual-Write, Cutover, and Rollback

Take inventory before touching anything

Moving from ELK to a Xinchuang observability platform starts with inventory, not action. Skipping this step is how migrations end up half-finished, with one team still querying the old cluster six months later. Write down four things:

• Data volume: daily log volume, peak QPS, and retention window determine target cluster nodes and disk. If you don't know peak, instrument the ELK ingest for a week and take the 95th percentile, not the average;
• Query habits: log every Kibana query, alert rule, and dashboard the team actually uses, so they map over cleanly. You'll find a long tail of one-off queries nobody remembers — that's the point;
• Collection surface: which hosts and apps, and whether they use Filebeat, Logstash, or something else. The new Agent must cover all of it, including the cron jobs and batch nodes people forget exist;
• Downstream consumers: who reads the logs — audit, BI, risk — and their reads must not break during the migration. If a compliance team runs daily extracts, they need to be told, or they'll notice on their own, loudly.

This inventory is the basis for every decision that follows. Treat it as a living document and update it as the migration surfaces things you missed.

Data migration and dual-write

For historical data, use "bulk import + incremental dual-write":

1. Bulk: export history from Elasticsearch with a batch tool and import it into OBSERVE sharded by time. Cold data — anything beyond the hot retention window — goes straight to object-storage archive, which is cheaper than keeping it queryable;
2. Incremental: the collection Agent writes to both ELK and OBSERVE at once (the Agent supports multiple outputs) for two to four weeks until both sides line up. This is the safety valve that makes the whole migration boring instead of scary.

During dual-write, both systems stay usable, so the team can practice on OBSERVE, tune queries, and build dashboards without disrupting day-to-day troubleshooting. Resist the urge to skip dual-write and do a big-bang cutover: it works until the first query that behaves differently from Kibana, and then you have no fallback.

Query syntax differences

Kibana's Lucene/KQL syntax differs from OBSERVE's SQL style, and this is where most migration friction lives. Common mappings:

• message: "error" AND status: 500 → WHERE message LIKE '%error%' AND status = 500;
• level: ERROR | count by host → SELECT host, count(*) FROM log WHERE level='ERROR' GROUP BY host;
• time ranges, field-existence queries, and fuzzy matching all have equivalents, but the exact spelling differs.

Build an "old query → new query" table, pre-translate the few dozen high-frequency queries, and pin it to the team wiki. Migrate alert rules the same way, one by one, verifying each one actually fires against recent data. The alert rules are the highest-risk item: a query that silently returns no results just looks empty, but an alert rule that silently stops firing means an outage goes unnoticed.

Cutover, rollback, and acceptance

Pick a low-traffic window for cutover and follow a fixed sequence:

1. Freeze alert changes a day ahead and confirm the new platform's rules are configured and tested;
2. An hour before cutover, stop incremental writes to ELK but keep it read-only as the rollback safety net;
3. Point writes solely at OBSERVE and watch for 30 minutes, confirming collection, search, and alerting all work;
4. Keep ELK read-only for seven days, then decommission once nothing was missed.

Acceptance must be quantified — search P95 latency, alert accuracy, and historical data retrievability all have to pass, not "feel right." Rollback is just the reverse: since ELK stayed read-only and online, reverting means pointing writes back at ELK. The existence of a tested rollback is what lets you schedule the cutover with confidence instead of dread.

Extra wins on the Xinchuang stack

The migration also clears up a few old problems that were quietly costing you: native support for domestic databases (Dameng, KingbaseES, OceanBase) with no extra middleware or connector hacks, native binaries on domestic OSes (Kylin, UOS) with no on-site compilation, and built-in audit retention and graded-protection compliance, which removes the need for a separate compliance tool. For teams under localization mandates, this last point often tips the business case on its own.