Replication
- Lesson 12 of 40
- 12 min
- Beginner
- 11 steps
A single database is both a bottleneck and a single point of failure. With several copies you spread the reads and survive losing a machine.
- Leader CPU
- 30%
- Machines reading
- 2
- Replica lag
- 20 ms
- Failed writes
- 0%
- request
- response
- batch
- replication
Step 1 of 11
Problem
One database for everything
Your app keeps everything on a single database server. Every read and every write goes through it, and it is already 90 % busy. Of those 90 points, 60 are reads (product pages, profiles, lists) and 30 are writes.
On top of that, if that machine goes down, the whole app stops working. The figures in this lesson are an example scenario.
Step 2 of 11
Solution
One leader and two replicas
Add two machines holding a copy of the data: the replicas (also called followers). Writes still go to a single machine, the leader.
Every change the leader applies is recorded in a replication log. The leader sends that log to the replicas and each one applies the same changes in the same order, so they end up with the same data.
What changed
- Two replicas
- The leader sends every change to the replicas
- The app still asks the leader for everything
Step 3 of 11
Solution
Reads go to the replicas
Now the app sends writes to the leader and reads to the replicas. The 60 points of reads are split across two machines and the leader drops from 90 % to 30 %: what the writes cost.
Each replica sits at 40 %: 30 for reads and about 10 for applying the changes that arrive from the leader. Need more reads? Add another replica.
What changed
- Reads → replicas
- Writes → leader only
- Leader CPU: 90 % → 30 %
Step 4 of 11
Problem
Replicas fall behind
Replication is usually asynchronous: the leader confirms the write as soon as it has stored it and sends it to the replicas right after.
Ana changes her name and reloads her profile. That read lands on a replica that has not received the change yet: she sees her previous name and thinks it was not saved.
What changed
- Ana saves a new name
- The replica still has the old one
- Lag: 20 ms → 2 s
Step 5 of 11
Technique
Read what you just wrote
A practical fix: for a few seconds after someone writes, their reads go to the leader. Everyone else keeps reading from the replicas. That gives the read-your-own-writes guarantee.
The other route is synchronous replication: the leader waits for a replica to confirm the change before answering "saved". That replica is never behind, but every write takes longer and, if the replica does not answer, writes are left waiting.
What changed
- Ana's reads → leader for 5 s
- Leader CPU: 30 % → 31 %
- Ana sees her new name
Step 6 of 11
Problem
The leader goes down
The leader's machine shuts down. Reads keep working, because the replicas have the data, but every write fails: nobody can sign up, buy or save changes.
The replicas drop to 30 %: they still serve reads, but no more changes arrive for them to apply.
What changed
- Leader down
- 100 % of writes fail
Step 7 of 11
Solution
Failover: a replica becomes the leader
Failover promotes the most up-to-date replica to leader, the app sends its writes there and the other replica starts copying from the new leader. A new machine replaces the dead one: it copies the data and, while it catches up, the only ready replica serves every read and climbs to 70 %.
Writes come back within seconds or a few minutes. Anything the old leader confirmed but never sent may be lost: another reason to keep one replica synchronous.
The hard part is being sure the leader is dead. If it was only slow or cut off from the network, you can end up with two leaders accepting writes: split brain. Avoiding it takes a majority agreeing; you will see how in the consensus lesson.
What changed
- The dead leader leaves
- A replica becomes the leader
- New replica copying data
- Remaining replica: 30 % → 70 %
Step 8 of 11
Problem
Case: the 20-minute report
Every month, finance runs a report that scans years of orders and takes 20 minutes. If it runs on a replica that also serves the app, that replica saturates, falls minutes behind and user reads turn slow.
The way out: an analytics replica the app never reads from. Heavy queries can run it at 100 % and take as long as they need without bothering anyone.
What changed
- Analytics replica, outside the app's reads
- The new replica now serves reads
- Reports no longer slow the app
Step 9 of 11
Problem
Case: the migration without WHERE
A deploy runs a migration that is missing its WHERE: UPDATE orders SET status = 'cancelled'. The leader applies it to 3 million rows and replication copies the mistake to every replica in under a second.
Replicas protect you from a machine breaking, not from a wrong write. For that you need backups: a daily snapshot plus the archive of the change log. Together they can rebuild the database as it was at any moment: point-in-time recovery.
What changed
- The migration overwrites 3M orders
- The replicas copy the damage
- Daily snapshot + log archive
- You restore to 14:02:59, just before the mistake
Step 10 of 11
Trade-off
Three ways to replicate
Everything so far is single-leader replication: one machine writes and several read. It is the simplest, and the default in most relational databases.
With multiple leaders, each region has its own and writes stay close to the user, but if two regions change the same row at the same time there is a conflict to resolve.
With no leader, the client writes to W of the N copies and reads from R. If W + R > N, at least one of the copies read always has the latest write. You will see more in the CAP lesson.
Step 11 of 11
Summary
Summary
The leader takes the writes and sends them to the replicas through the replication log.
Replicas multiply reads and survive a machine going down.
Asynchronous replicas lag: right after writing, read from the leader.
If the leader dies, promote a replica and avoid ending up with two leaders.
Heavy queries go on their own replica.
A replica is not a backup: keep snapshots and the log, and rehearse restoring.
← → step · R replay · Space pause
You will learn
- Understand the leader, the replicas and the replication log (covered)
- Spread reads across replicas (covered)
- Handle lag and read what you just wrote (covered)
- Promote a replica when the leader goes down (covered)
- Keep analytics and backups apart, and pick a replication style (covered)