Kafka MirrorMaker replication between multiple regions -

note: posted users@kafka.apache.org.

i need solution mirror topic remote location avoids cycles , topic per region issue.

this solution require 2 topics regardless of how many regions (datacenters) replicates to.

i initial producer (p1) write local (l1) , remote topic (r1). mirrormaker (m1) read remote topic (r1) , write other regions local topic (l2). other region in kind write using (p2) local (l2) , remote (r2) , separate mirrormaker (m2) replicate remote (r2) other regions local (l1).

if add third region, need add mirrormaker existing regions , replicate third existing.

-------------------------------  region total: 2 -------------------------------  p1 -> l1, r1  p2 -> l2, r2  m1 (r1 -> l2)  m2 (r2 -> l1)  -------------------------------  region total: 3 -------------------------------  p1 -> l1, r1  p2 -> l2, r2  p3 -> l3, r3  m1.1 (r1 -> l2)  m1.2 (r1 -> l3)  m2.1 (r2 -> l1)  m2.2 (r2 -> l3)  m3.1 (r3 -> l1)  m3.2 (r3-> l2) 

here solution.

each region writes own topic. mirrormaker replicates each region's specific topic every other region. consumer in regions consumes 3 topics.

this solution requires 1 write per producer instead of two. n topics (1 per region) , n(n-1) mirror makers , n consumers consuming n topics each.