The solution of Redis master slave replication problem and capacity enlargement problem

  • 2020-05-10 23:07:28
  • OfStack

1. Address master-slave replication

When using Redis as storage engines, and use separate Redis, speaking, reading and writing, as read from the machine, from machine downtime or disconnected and host need to reopen the connection between a host and reconnect the host will trigger the full amount of master-slave replication, and then generates a memory snapshot, host can still provide service, but as read from the machine, will not be able to provide foreign services, if the data quantity is big, recovery time will be quite long. To solve the Redis master-slave Copy problem, there are two solutions:

Active replication
Active replication is when the business layer double-writes multiple Redis, avoiding the master-slave replication that comes with Redis. However, in order to ensure master-slave 1, a series 1 verification mechanism needs to be added. And doing so can degrade system performance.
Modify the source code to support incremental synchronization
Redis writes AOF files and turns off Redis rewrite AOF files. In order to avoid oversize files, you can split files by yourself.
During low business times, generate a memory snapshot and record the point where AOF was at the time of the snapshot.
When reconnection from machine, from machine send synchronous command to host, the host after receiving the command, sends the latest snapshot files from machine, from machine from a snapshot file recovery, and obtained the snapshot corresponding AOF point, from the machine send AOF bit to host, the host will AOF file after that point all of the data synchronization to from the machine operation, to achieve the effect of incremental synchronization.

2. Capacity expansion

Redis the author's idea is: Redis Presharding (http: / / oldblog. antirez. com post/redis - presharding. html)

The budget sets the number of Redis instances, assuming the number of instances n, n = number of machines * number of instances of redis per machine
The late extension only needs to migrate some instances of redis from the old machine to the new machine to achieve smooth expansion.
The migration steps are as follows:

Instances are created on the new machine, and each instance is set to the slave of the migrated instance.
After master slave replication is complete, the setup program makes the new instance master.
Stop the old instance
After the above steps, the memory of the old machine is increased to a maximum of 1 Redis instance per machine.

As mentioned in the author's article, starting multiple instances on one machine does not actually consume too much resource, because Redis is light enough, and rewriting the AOF file one by one or generating a memory snapshot for multiple instances can reduce the footprint of memory without affecting external services.


Related articles: