How Redis implements persistence solution (used by RDB and AOF)

1. The role of persistence

1. What is persistence

All data saved by redis In memory, updates to data will be asynchronously saved to the hard disk

2. How to implement persistence

Snapshot: a completion of data at a certain time Backup - mysql's Dump - redis's RDB log writing: any operation is recorded in the log. To restore the data, just go through the log again - mysql's Binlog - Hhase's HLog - Redis's AOF

2. RDB

1. What is RDB

2. Trigger Mechanism-main three methods

The first one: save (synchronization)

1 The client enters the save command----》redis server----》Synchronized creation RDB binary file

2 will cause redis blocking (when the amount of data is very large)

3 File strategy: If the old RDB exists, it will replace the old one

4 Complexity o(n)

Second type: bgsave (asynchronous, Backgroud saving started)

1 The client enters the save command----"redis server----"asynchronous Create an RDB binary file (the fork function generates a child process (fork will block reids), execute createRDB, the execution is successful, and a reids message is returned)

2 When accessing redis at this time, the client will respond normally

3 File strategy: Same as save, if the old RDB exists, it will replace the old one

4 Complexity o(n)

The third method: (common method) (** ****) Automatically (through configuration file)
Configuration seconds changes
save 900 1save 300 10save 60 10000If 1w pieces of data are changed in 60s, automatically generate rdb
If 10 pieces of data are changed in 300s , automatically generate rdb
If 1 piece of data is changed in 900s, automatically generate rdb

If any of the above three conditions is met, the rdb will be automatically generated, and bgsave is used internally

#Configuration:

save 900 1 #Configure one

save 300 10 #Configure one

save 60 10000 #Configure one

dbfilename dump.rdb #The name of the rdb file, the default is dump.rdb

dir ./ #The rdb file exists in the current directory

stop-writes-on-bgsave-error yes #If an error occurs in bgsave, whether to stop Write, the default is yes

rdbcompression yes #Use compression format

rdbchecksum yes #Whether to checksum the rdb file

#Best configuration

save 900 1

save 300 10

save 60 10000 dbfilename dump-${port}.rdb

#With port As the file name, there may be many reids on one machine, so it will not be messy

dir /bigdiskpath #Put the save path to a large hard disk location directory

stop-writes-on-bgsave-error yes

#Error stop

rdbcompression yes #Compression

rdbchecksum yes #Verification

RDB trigger mechanism generally uses the first There are three ways, but this method also has shortcomings. If the number of modified items is not within the setting range, it will not be triggered, which will lead to a lot of data not being persisted. So we generally use the following method: AOF.

If you want to save unimportant data, you can use RDB (such as cache data). If you want to save very important data, you should use AOF, but both methods can also be used at the same time.

3. AOF

1. RDB problem

is time consuming and performance consuming. Uncontrollable, data may be lost.

2. AOF introduction

Every time the client writes a command, a log is recorded and placed in the log file. If there is a downtime, the data can be completely restored

3. Three strategies of AOF

The log is not written directly to the hard disk, but is first placed in the buffer. The buffer is written to the hard disk according to some strategies

#The first type: always: redis--》Write command refresh buffer---》Fsync each command to the hard disk---》AOF file

#The second type: everysec (default value ): redis——》Buffer refreshed by writing command---》fsync buffer to hard disk every second--》AOF file

#The third type: no:redis——》Buffer refreshed by writing command Buffer---》The operating system determines, the buffer fsyncs to the hard disk--》AOF file

Command	always	everysec	no
Advantages	No data loss	Once fsync per second, lose 1 second of data	Don’t worry
Disadvantages	The IO overhead is large, and the average sata disk only has a few hundred TPS	lost 1 second data	Uncontrollable

4.AOF rewriting

As the commands are gradually written, the amount of concurrency increases , the AOF file will become larger and larger, solve this problem through AOF rewriting

##set hello world
##The essence is to optimize expired, useless, repeated, and optimizable commands, which can reduce disk usage and accelerate recovery speedImplementation method

Native AOF	##AOF Rewrite
set hello java set hello hehe incr counter ncr counter rpush mylist a rpush mylist b rpush mylist c Expired data ##set hello hehe	set counter 2 rpush mylist a b c

bgrewriteaof: The client sends the bgrewriteaof command to the server, and the server will start a fork process to complete the AOF rewrite

AOF rewrite configuration:

Rewrite process

AOF configuration file (******)

appendonly yes #Set this option to yes and open appendfilename "appendonly-${port}.aof " #The name of the file saved appendfsync everysec #Adopt the second strategy dir /bigdiskpath #The storage path no-appendfsync-on-rewrite yes #When rewriting aof, whether to do the append operation of aof, because aof rewriting consumes Performance, disk consumption, normal AOF writing to disk has certain conflicts, data during this period is allowed to be lost

4. Selection of RDB and AOF

1. Comparison of rdb and aof

CommandLow High (hang and restart, aof data will be loaded) Small Large FastLost data Decided according to strategy Heavy Light

rdb	aof	Startup priority
		Size
		Recovery speed
Slow		Data security
		Light and heavy
		2.rdb best strategy Rdb is turned off, master-slave operation Centralized management: backup data by day, by hour Master-slave configuration, slave node is turned on 3.aof best strategy Open: cache and storage, open in most cases, aof rewrite centralized management everysec: strategy refreshed every second 4. Best strategy Small sharding: the maximum memory of each redis is 4g Cache or storage: use different strategies according to the characteristics Monitor the hard disk at all times, Memory, load network, etc. Have enough memory The above is the entire content of Redis (4)-persistence solution (used by RDB and AOF). Related references:PHP Chinese website

The above is the detailed content of How Redis implements persistence solution (used by RDB and AOF). For more information, please follow other related articles on the PHP Chinese website!