Background

In many IO scenarios, we often need to ensure that the data has been safely written to the disk so that it can be read after the system crashes and restarts. data. But we all know that the IO path of the Linux system is still very complicated and is divided into many layers. Each layer may have buffers to accelerate IO reading and writing. At the same time, user-mode applications and library functions may also have their own buffers, which adds some complexity to the IO path. It can be seen that if you want to ensure that data is safely written to the disk, it cannot be done by simply adjusting write/fwrite.

So what to do? Many people will think of many ways, such as: fflush(), fsync(), fdatasync(), sync(), open() using the O_DIRECT or O_SYNC flag, etc. Well, these means (or some combination) can indeed ensure the safe persistence of data, so what is the difference between them? What is the difference between fflush() and fsync()? What does O_DIRECT mean? Can it ensure safe data persistence? What is the difference between O_DIRECT and O_SYNC? What about O_SYNC and fsync()? Can fsync complete the functions of msync? This article will try to understand and explain the functions and differences of these concepts.

Linux IO

It is said that a picture is worth a thousand words. In order to analyze the difference between these concepts clearly, I specially drew a picture and look at it carefully. , it should be possible to clearly see their functions and differences.

## Here we focus on O_DIRECT and O_SYNC. First of all, it must be clear that O_DIRECT only means that the data will not go through the page cache (generally used in user mode to manage the buffer) but Submit directly to the block device layer, but will not wait synchronously for the data to be safely written to the disk before returning (for example, the data may still be queued at the block layer or in the disk's own cache). As for the O_SYNC flag, although the data will still be written to the page cache, the write through strategy will be adopted at this time, and the data will be returned synchronously until the data is safely written to the disk. Therefore, if O_DIRECT and O_SYNC are used at the same time, it means that the data will not go through the page cache and will wait for the data to be safely written to the disk before returning. Of course, the IO performance will be very low.

Since O_DIRECT will bypass the page cache, if another process uses the normal method to read the file, data inconsistency may occur. This also needs to be paid attention to.

In order to provide some auxiliary explanation, I will post here some information that I have read during the discussion. The first is to quote the open system call:

http://man7.org/linux/man-pages/man2/open.2.html Description of related parameters:

And innodb related documents:

https://lwn.net/Articles/457667/

The difference between fsync and fdatasync:

http://man7.org/linux/man-pages/man2/fsync.2.html

msync:

http://man7.org/linux/man-pages/man2/msync.2.html

DAX

In fact, there is another IO mode, which is DAX (Direct Access). Does it look similar to O_DIRECT? This mode requires support from both the filesystem and block driver. It is generally used on non-volatile memory. In essence, it bypasses the page cache and directly operates the device. This article will not discuss DAX in depth. Later, I will write a ramdisk block device driver that supports DAX mode, then format it into an ext4 file system and mount it in -o dax mode, and then study the IO path of DAX in detail.

For more Linux articles, please visit the Linux Tutorial column to learn!

The above is the detailed content of How to ensure safe data transfer in Linux. For more information, please follow other related articles on the PHP Chinese website!