Confluent Cloud Migration Checklist

First written September 2025, last updated September 2026.

Migrating to Confluent Cloud is a cutover. Clients, schemas, offsets, connectors, and stream topologies all move, and each one has its own failure mode. This checklist is the working document we use to structure those conversations with development and operations teams.

Work through it section by section and adjust for your topology.

Three things that bite people most often, all covered under "Gotchas" at the end: Java and librdkafka clients hash keys differently by default, so partition placement can change after replication; a replay from seven days back can drop records in intermediate topics with five-day retention; and schema IDs don't match across registries.

Confluent Cloud Provisioning

Before migration, you need a Confluent Cloud instance to migrate into. This should be done with careful planning, ensuring access and networking are correct.

• Cluster Type
• Networking
• Provisioning
• Connectors
• Managed - Which will be managed by Confluent Cloud?
• Self Hosted - Which need to be self-hosted?
• Where will self-hosted connectors be deployed?
• How will they be secured?
• Schema Registry
• How many schemas do you anticipate having?
• Understanding logical and physical deletes
• Developer Access
• Confluent Cloud Control plane and data plane are separate.
• Will the network of developer machines have access?
• Environments
• Number of environments
• Naming standards for shared Confluent Cloud environments can make migration harder.
• Sharing (replicating) data between environments
• Do you replicate production data to staging for testing?
• PCI/PII concerns?
• Network Validation
• Can you produce to Confluent Cloud from the network of your source cluster?
• Can you consume from the source cluster from the network you will be using for your Confluent Cloud applications?
• Monitoring
• How will you be monitoring your cluster?
• Metric Understanding
• Unlike typical Apache Kafka metrics collection, many metrics are "since the previous" and are per minute, not per second.
• From documentation: "Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds."
• Metrics should be reviewed and understood.

Client Migration

Prior to migration of data, a plan for client migration is also required. Apache Kafka provides a rich set of security options, but not all Kafka-API SaaS services support all of them. There is likely a need to change how your client applications connect between instances. Confirm each application instance can connect to either cluster with appropriate credentials. This flexibility makes migration smoother.

• Will there need to be a change to the security.protocol and/or sasl.mechanism to the clients?
• Migration from mTLS
• Migration from IAM (MSK)
• SASL mechanism change
• Security configuration provisioning with clients
• Serdes Migration
• Migration from AWS Glue Schema Registry requires migration from schema as well as message changes.
• Schema Registry Migration
• Consider the use of schema contexts
• Consider the use of Schema Linking
• Consumer group.id reuse or rename?
• Kafka Streams application.id reuse or rename?
• Smoke Test
• How to deploy a consumer in both environments without impacting downstream systems?

Replication

Replication Toolage

• Cluster Linking
• If you are migrating from Confluent Platform or another Confluent Cloud instance, use cluster linking.
• Moving from a Confluent instance to another Confluent instance also means there isn't a serdes concern, making things easier.
• Schema Registry migration still has complexities, and needs to be addressed.
• For the sake of this migration checklist, it is assumed that cluster linking isn't an available option.
• Replicator
• Schema migration
• How to handle offset cutover without loss of messages
• Mirror Maker 2
• Schema migration
• How to handle offset cutover without loss of messages
• Change Data Capture (CDC)
• In place of replicating data from one cluster to another, go to the source system and replicate it from there, such as with CDC sources.
• Schema migration handled in the configuration of the CDC tool
• Custom Applications
• Consume from one cluster and write to another with full control of any serde changes or other data conversions necessary.
• Schema migration handled in the application code

Replication Client Security

The system replicating the topics needs authentication and authorization to both clusters.

• Source cluster credentials
• Source cluster authorization
• Destination cluster credentials
• Destination cluster authorization

Replication Infrastructure

How is the replication code going to be deployed?

• Existing Connect cluster
• New Connect cluster
• Standalone deployment
• Custom code

Inventory

Inventory of Kafka resources to be migrated, grouped in units that can be migrated together. The goal is to minimize the complexity of each migration and avoid big-bang migrations, which are challenging to roll back.

• For each Business Context
• Topics
• Schemas
• Will schemas be created by the producer's serde, or will they be created through CI/CD (REST API)?
• Consumer Groups
• ACLs
• Producers
• Consumers
• Connectors
• Managed
• Self Hosted
• Stream Processing
• Kafka Streams
• Flink
• ksqlDB
• Other

Data Migration

Data migration is typically done by a specific business context or domain. Orchestration depends on the complexity of your streaming platform.

Key considerations

• Idempotency
• If end systems aren't idempotent, how to handle deduplication
• Compacted topics (KTables)
• Migration of data vs application recreating them
• Each application uses compacted topics differently; understand the business domain of each compacted topic.

Steps

These steps are a starting point. Once a set of topics, applications, and consumer groups is identified as a unit to be migrated, review and adjust accordingly.

• Replicate Source Topics
• Migrate Consumer Groups
• Evaluate offset migration
• Start at earliest, latest, setting offset by timestamp, etc.
• Migrate Consumers
• Migrate Sink Connectors
• Migrate Streams
• While streams are consumers and producers, they tend to consume first and then produce, so they are considered a consumer application for migration, but additional analysis is required.
• Migrate Source Connectors
• Disable Original Producers
• Disable Original Source Connectors
• Disable Replication
• Migrate Producers
• Can rollback be achieved by producers being reset?
• If not, consider dual producing to both clusters (hence replication is disabled first).

Migration "Gotchas"

Here are some additional considerations and trouble scenarios that can happen with a migration.

• Remove Application Cluster Connection Assumptions, if possible.
• Kafka clients can be configured dynamically, allowing for authentication to be changed through configuration; confirm your development team hasn't made configuration assumptions
• For example, application assumes 'mTLS' authentication and fails if keystore isn't provided, but you now want to connect to a cluster with SASL.
• Timestamp Preservation
• If a topic in the source system uses log append time and the destination topic does too, timestamps will change when replicated. Consider using producer timestamp until replication is completed, and then migrate to LogAppendTime once producers are migrated to the new cluster.
• Schema Preservation
• Until Schema Registry 7.0, all schemas would have a unique id. This makes migration of data between clusters quite challenging, due to the destination cluster not having the same IDs, as IDs were created as schemas were added.
• Use Contexts and Import functionality.
• Byte-for-byte copy considerations when it comes to schema migration.
• Compacted Topics can have old data (and schema ids)
• Kafka Streams Migration
• State stores (changelog topics)
• How to hydrate changelog topics
• Will windowed state stores need to be migrated?
• CDC Migration
• Cutover
• retention.ms and old data.
• Example:
• Source topic has 7-day retention
• Intermediate stream topics have 5-day retention
• Replication of data starting from 7 days ago could lead to records dropped during stream processing.
• Increasing partitions at time of migration
• Replication method can't be configured to preserve partition number for messages
• Could lead to out of order events
• Kafka Stream processing with time semantics and grace periods can help with this.
• Verifying hashing algorithm (see below)
• Key hashing
• kafka-clients default hashing algorithm, murmur2.
• librdkafka default hashing algorithm, crc32.
• When using migration tools on topics where partitioning is being preserved, force partition matching.
• Otherwise, verify hashing algorithms to avoid a replication tool using a different algorithm than your applications.