Master-Slave Architecture Patterns

Master-slave architecture (increasingly referred to as primary-replica or leader-follower in modern terminology) is one of the most enduring patterns in distributed systems. It underpins everything from relational database replication to distributed task processing frameworks. This post explores how the pattern works, when to use it, and how to avoid its common pitfalls.

A note on terminology: Many projects and organizations have moved away from "master-slave" in favor of "primary-replica," "leader-follower," or "coordinator-worker." This article uses "master-slave" for historical clarity but the concepts apply identically regardless of naming.

What Is the Master-Slave Pattern?

At its core, the pattern designates one node as the master (authoritative source of truth) and one or more nodes as slaves (replicas or workers). The master coordinates operations, while slaves either replicate data or execute delegated work.

There are two dominant variations:

1. Data replication – The master handles writes; slaves serve reads.
2. Task delegation – The master distributes work; slaves execute it.

Variation 1: Data Replication

In replication scenarios, all write operations flow through the master. Changes are then propagated to slaves, which serve read traffic. This is the model used by MySQL, PostgreSQL, Redis, and MongoDB.

          ┌──────────┐
  Writes  │  MASTER  │
 ───────► │          │
          └────┬─────┘
               │ replication stream
      ┌────────┼────────┐
      ▼        ▼        ▼
  ┌───────┐┌───────┐┌───────┐
  │SLAVE 1││SLAVE 2││SLAVE 3│ ◄── Reads
  └───────┘└───────┘└───────┘

Replication Modes

Synchronous replication blocks the master's write acknowledgment until at least one slave confirms receipt. This guarantees durability at the cost of latency.

Asynchronous replication acknowledges writes immediately and propagates changes in the background. It's fast but risks losing recent writes if the master fails before replication completes.

Variation 2: Task Delegation

In compute-oriented systems, the master acts as a coordinator that partitions and distributes work.

import queue
import threading

class Master:
    def __init__(self, num_slaves):
        self.task_queue = queue.Queue()
        self.results = []
        self.lock = threading.Lock()
        self.slaves = [Slave(i, self) for i in range(num_slaves)]

    def submit(self, task):
        self.task_queue.put(task)

    def collect(self, result):
        with self.lock:
            self.results.append(result)

    def run(self):
        threads = [threading.Thread(target=s.work) for s in self.slaves]
        for t in threads:
            t.start()
        self.task_queue.join()

class Slave:
    def __init__(self, slave_id, master):
        self.slave_id = slave_id
        self.master = master

    def work(self):
        while True:
            try:
                task = self.master.task_queue.get(timeout=1)
            except queue.Empty:
                return
            result = self.process(task)
            self.master.collect(result)
            self.master.task_queue.task_done()

    def process(self, task):
        return f"slave-{self.slave_id} processed {task}"

Handling Failover

The single most important design consideration is what happens when the master fails. Without a plan, the master becomes a single point of failure.

Automatic Promotion

When the master dies, a slave must be promoted. This requires:

1. Failure detection – Typically heartbeat-based monitoring.
2. Leader election – Choosing which slave becomes the new master.
3. Reconfiguration – Redirecting clients and re-pointing remaining slaves.

Tools like Redis Sentinel, Patroni (PostgreSQL), and ZooKeeper-based coordinators automate this process.

# Example: Redis Sentinel configuration
sentinel monitor mymaster 10.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1

The 2 in the monitor line is the quorum—the number of Sentinels that must agree the master is unreachable before failover triggers.

Common Pitfalls

1. Split-Brain

If a network partition isolates the master but it remains alive, a slave may be promoted while the old master still accepts writes. Now you have two masters with divergent data.

Mitigation: Use quorum-based decisions and fencing (STONITH — "Shoot The Other Node In The Head") to forcibly demote the old master.

2. Replication Lag

Asynchronous slaves fall behind under heavy write load, causing clients to read stale data.

Mitigation: Route read-after-write operations to the master, or use "read-your-writes" consistency by tracking replication position.

3. Thundering Herd on Failover

When the master fails, all clients reconnect simultaneously, overwhelming the new master.

Mitigation: Implement exponential backoff with jitter in client reconnection logic.

When to Use This Pattern

Good fit:

• Read-heavy workloads where reads vastly outnum