System Scalability — Database failover strategy

Adela Chao
2 min readMay 10, 2022

We have talked about the scaling for web servers.

In this article, we will look at the scaling of database.

Cold standby — periodically backup

We have a primary database running, and a standby database idling sideway.

Periodically backup the primary database and save the backup file somewhere, for example, AWS S3.

If the primary database goes down, we can restore the backup file to the standby server and redirect the traffic to new host.

It’s a simple and cheap approach, but comes with a cost:

  • To provision a new server and restore database from backup file takes time, especially if there is a lot of data.
  • The data that happened after the last backup will be lost.
  • So we need to prepare to tolerate a certain amount of downtime and data loss.

Warm standby — replication

Again, we have a primary database, and a backup database.

Instead of doing periodically backup, we use replication to constantly copy data from primary to backup database.

So the backup database is always warm and ready to take over in case the primary goes down, without going through the trouble of manually restore.

Although this is a better approach than cold standby, it’s still in the sense of vertical scaling : a giant primary database and a giant backup database, so we are still limited by how much traffic a single server can provide.

Hot standby — simultaneously write/Read

Instead of doing replication, we are actually writing data simultaneously to every database instance, so both primary and backup database hold the same data at the same time.

Then we can also distribute read operation simultaneously to each database. This gives us the benefit of horizontal scaling.