Learn about RDB and AOP persistence in redis in one article

This article will take you through the two persistence mechanisms (RDB and AOP) in redis. I hope it will be helpful to everyone!

#Redis is an in-memory database, and the data is stored in the memory. However, we all know that the data in the memory changes quickly and is prone to loss. Fortunately, Redis also provides us with persistence mechanisms, namely RDB (Redis DataBase) and AOF (Append Only File). [Related recommendations: Redis Video Tutorial]

It is assumed here that you already understand the basic syntax of redis. A certain alphabet website has a good tutorial, you can check it out. I won’t write articles about basic usage, they are all commonly used commands.

The following is an introduction to these two methods. From shallow to deep.

1. Persistence Process

Since redis data can be saved on disk, what is this process like?

The following five processes are required:

(1) The client sends a write operation to the server (the data is in the client's memory).

(2) The database server receives the data of the write request (the data is in the memory of the server).

(3) The server calls the write system call to write the data to the disk (the data is in the buffer of the system memory).

(4) The operating system transfers the data in the buffer to the disk controller (the data is in the disk cache).

(5) The disk controller writes the data to the physical media of the disk (the data actually falls on the disk).

These 5 processes are a normal saving process under ideal conditions, but in most cases, our machines, etc. will have various failures. Here are two situations:

(1) If the Redis database fails, as long as the third step above is completed, it can be persisted and saved, and the operating system will complete the remaining two steps for us.

(2) If the operating system fails, all the above 5 steps must be completed.

Only possible failures during the saving process are considered here. In fact, the saved data may also be damaged, requiring a certain recovery mechanism, but this will not be extended here. The main consideration now is how redis implements the above five steps of saving disks. It provides two policy mechanisms, namely RDB and AOF.

2. RDB mechanism

RDB actually saves data on the disk in the form of snapshots. What is a snapshot? You can understand it as taking a photo of the data at the current moment and saving it.

RDB persistence refers to writing a snapshot of the data set in memory to disk within a specified time interval. It is also the default persistence method. This method writes the data in the memory to a binary file in the form of a snapshot. The default file name is dump.rdb.

After we installed redis, all configurations are in the redis.conf file, which stores various configurations of the two persistence mechanisms, RDB and AOF.

Since the RDB mechanism saves all data at a certain moment by generating a snapshot, there should be a trigger mechanism to implement this process. For RDB, three mechanisms are provided: save, bgsave, and automation. Let’s take a look at them separately

1. Save triggering method

This command will block the current Redis server. During the execution of the save command, Redis cannot process it. other commands until the RDB process is completed. The specific process is as follows:

#If there is an old RDB file when the execution is completed, replace the old one with the new one. Our clients may be tens of thousands or hundreds of thousands, so this approach is obviously not advisable.

2. bgsave triggering method

#When executing this command, Redis will perform snapshot operations asynchronously in the background, and the snapshot can also respond to the client. ask. The specific process is as follows:

The specific operation is that the Redis process performs a fork operation to create a child process. The RDB persistence process is responsible for the child process and ends automatically after completion. Blocking only occurs in the fork phase and is generally very short-lived. Basically all RDB operations inside Redis use the bgsave command.

3. Automatic triggering

Automatic triggering is done by our configuration file. In the redis.conf configuration file, there are the following configurations, which we can set:

①save: This is used to configure the RDB persistence conditions that trigger Redis, that is, when to save the data in the memory to harddisk. For example "save m n". Indicates that bgsave is automatically triggered when the data set is modified n times within m seconds.

The default configuration is as follows:

# means that if the value of at least 1 key changes within 900 seconds, save 900 1# means that if the value of at least 10 keys changes within 300 seconds, save 300 10# means that if at least 10 keys change within 60 seconds, save 900 If the value of 10000 keys changes, then save 60 10000

does not require persistence, then you can comment out all save lines to disable the save function.

②stop-writes-on-bgsave-error: The default value is yes. When RDB is enabled and the last background save of data fails, whether Redis stops receiving data. This would make the user aware that the data was not persisted to disk correctly, otherwise no one would notice that a disaster had occurred. If Redis is restarted, it can start receiving data again

③rdbcompression; the default value is yes. For snapshots stored on disk, you can set whether to compress them for storage.

④rdbchecksum: The default value is yes. After storing the snapshot, we can also let redis use the CRC64 algorithm for data verification, but this will increase performance consumption by about 10%. If you want to get the maximum performance improvement, you can turn off this function.

⑤dbfilename: Set the file name of the snapshot, the default is dump.rdb

⑥dir: Set the storage path of the snapshot file. This configuration item must be a directory, not a file name.

We can modify these configurations to achieve the effects we want. Because the third method is configured, we compare the first two:

4. Advantages and Disadvantages of RDB

①. Advantages

(1) RDB files are compact and fully backed up, making them very suitable for backup and disaster recovery.

(2) When generating an RDB file, the redis main process will fork() a sub-process to handle all saving work. The main process does not need to perform any disk IO operations.

(3) RDB is faster than AOF when restoring large data sets.

② Disadvantages

RDB snapshot is a full backup, which stores the binary serialized form of memory data and is very compact in storage. When snapshot persistence is performed, a child process will be started specifically responsible for snapshot persistence. The child process will own the memory data of the parent process. Modifications to the memory by the parent process will not be reflected by the child process, so the data modified during snapshot persistence will not be reflected. is saved, data may be lost.

3. AOF mechanism

Full backup is always time-consuming. Sometimes we provide a more efficient method AOF. The working mechanism is very simple. Redis will Each received write command is appended to the file via the write function. The popular understanding is logging.

1. Persistence principle

Look at the picture below for his principle:

Whenever a write command comes, it is saved directly in our AOF file.

2. Principle of file rewriting

The AOF method also brings another problem. Persistence files will become larger and larger. In order to compress the persistent file of aof. Redis provides the bgrewriteaof command. Save the data in the memory to a temporary file in the form of a command, and at the same time fork a new process to rewrite the file.

The operation of rewriting the aof file does not read the old aof file, but rewrites the entire database content in the memory into a new one using the command. aof file, which is somewhat similar to a snapshot.

3. AOF also has three triggering mechanisms

(1) Synchronization always after every modification: synchronization persistence will be triggered every time a data change occurs. Immediately record to the disk, the performance is poor but the data integrity is better

(2) Synchronize everysec every second: Asynchronous operation, record every second. If the machine goes down within one second, there will be data loss

( 3) Different no: never synchronized

4. Advantages

(1) AOF can better protect data from loss, generally AOF An fsync operation will be performed through a background thread every 1 second, and up to 1 second of data will be lost. (2) AOF log files do not have any disk addressing overhead, the writing performance is very high, and the files are not easily damaged.

(3) Even if the AOF log file is too large, background rewrite operations will not affect the client's reading and writing.

(4) The commands of the AOF log file are recorded in a very readable way. This feature is very suitable for emergency recovery of catastrophic accidental deletion. For example, someone accidentally clears all the data using the flushall command. As long as the background rewrite has not occurred at this time, the AOF file can be copied immediately, the last flushall command is deleted, and then the AOF file is put back. Automatically restore all data through the recovery mechanism

5. Disadvantages

(1) For the same data, AOF log files are usually larger than RDB data snapshot files

(2) After AOF is turned on, the write QPS supported will be lower than the write QPS supported by RDB. Because AOF is generally configured to fsync the log file once per second. Of course, fsync once per second, and the performance is still very high

(3) There were bugs in AOF before, and the data was collected through the logs recorded by AOF. During recovery, the exact same data was not recovered.

4. How to choose between RDB and AOF

If you choose, the two together will be better. Because you understand the two persistence mechanisms, the rest depends on your own needs. You may not necessarily choose one based on different needs, but they are usually used in combination. There is a picture to summarize:

# After comparing these features, the rest is up to you.

For more programming related knowledge, please visit: Programming Video! !

The above is the detailed content of Learn about RDB and AOP persistence in redis in one article. For more information, please follow other related articles on the PHP Chinese website!