Linux memory mechanism and manual release of swap, buffer and cache

This article introduces the principles and practical operations of Linux memory mechanism, virtual memory swap, buffer/cache release, etc.

1. What is the memory mechanism of Linux?

2. When did Linux start using virtual memory (swap)?

3. How to release memory? 4. How to release swap?

1. What is the memory mechanism of Linux?

We know that reading and writing data directly from physical memory is much faster than reading and writing data from the hard disk. Therefore, we hope that all data reading and writing will be faster. It is completed in memory, and memory is limited, which leads to the concepts of physical memory and virtual memory.

Physical memory is the memory size provided by the system hardware. It is real memory. Compared with physical memory, there is a concept of virtual memory under Linux. Virtual memory is a strategy proposed to meet the shortage of physical memory. , it is a piece of logical memory that is virtualized using disk space. The disk space used as virtual memory is called swap space.

As an extension of physical memory, Linux will use the virtual memory of the swap partition when the physical memory is insufficient. In more detail, the kernel will write the temporarily unused memory block information to the swap space. In this way, The physical memory is released, and this memory can be used for other purposes. When the original content is needed, the information will be read from the swap space into physical memory again.

Linux's memory management adopts a paging access mechanism. In order to ensure that physical memory can be fully utilized, the kernel will automatically swap infrequently used data blocks in physical memory into virtual memory at appropriate times. , while retaining frequently used information to physical memory.

To have a deep understanding of the Linux memory operating mechanism, you need to know the following aspects:

Linux system will perform page swap operations from time to time to maintain as much free physical memory as possible, even if there is no When something requires memory, Linux will also swap out temporarily unused memory pages. This avoids the time required to wait for the exchange.

Linux page swapping is conditional. Not all pages are swapped to virtual memory when not in use. The Linux kernel only swaps some infrequently used page files to virtual memory based on the "most recently used" algorithm. Memory, sometimes we will see this phenomenon: Linux still has a lot of physical memory, but a lot of swap space is also used. In fact, this is not surprising. For example, when a process that takes up a lot of memory needs to consume a lot of memory resources when running, some uncommon page files will be swapped into virtual memory, but later this process that takes up a lot of memory resources will be swapped into virtual memory. When the process ends and a lot of memory is released, the page file that was just swapped out will not be automatically swapped into the physical memory. Unless this is necessary, the system's physical memory will be much free at this moment, and the swap space is also being used. The phenomenon just mentioned occurred. There's nothing to worry about at this point, as long as you know what's going on.

The pages in the swap space will first be swapped to physical memory when they are used. If there is not enough physical memory to accommodate these pages at this time, they will be swapped out immediately. In this way, there may be no virtual memory. Enough space to store these swap pages will eventually lead to problems such as false crashes and service abnormalities in Linux. Although Linux can recover by itself within a period of time, the recovered system is basically unusable.

Therefore, it is very important to properly plan and design the use of Linux memory.

In the Linux operating system, when an application needs to read data in a file, the operating system first allocates Some memory, read data from the disk into these memories, and then distribute the data to the application; when it is necessary to write data to the file, the operating system first allocates memory to receive the user data, and then writes the data from the memory to the disk. superior. However, if there is a large amount of data that needs to be read from the disk to the memory or written from the memory to the disk, the read and write performance of the system becomes very low, because whether it is reading data from the disk or writing data to the disk, it is a very long process. A process that consumes time and resources. In this case, Linux introduced the buffers and cached mechanisms.

Buffers and cached are both memory operations, used to save files and file attribute information that have been opened by the system. In this way, when the operating system needs to read certain files, it will first search in the buffers and cached memory areas. If found, Directly read out and send it to the application. If the required data is not found, it will be read from the disk. This is the caching mechanism of the operating system. Through caching, the performance of the operating system is greatly improved. But the contents of buffers and cached buffers are different.

Buffers is used to buffer block devices. It only records the metadata of the file system and tracking in-flight pages, while cached is used to buffer files. To put it more simply: buffers are mainly used to store the contents of the directory, file attributes and permissions, etc. And cached is directly used to remember the files and programs we have opened.

In order to verify whether our conclusion is correct, you can open a very large file through vi to see the changes in cache, and then vi the file again to feel the similarities and differences in the speed of the two openings. Is it the first time? Is the second opening speed significantly faster than the first time? Then execute the following command:

find / -name .conf to see if the value of buffers changes, and then execute the find command repeatedly to see the difference in display speed between the two times.

2. When did Linux start using virtual memory (swap)?

[root@wenwen ~]# cat /proc/sys/vm/swappiness 60

The 60 above means that swap will only be used when 40% of the physical memory is used (refer to network information: when the remaining physical memory is less than 40% (40 =100-60), start using the swap space) When swappiness=0, it means using the physical memory to the maximum extent, and then the swap space. When swappiness=100, it means actively using the swap partition and transferring the data on the memory in a timely manner. Move it to the swap space.

The larger the value, the more likely it is to use swap. It can be set to 0, which does not prohibit the use of swap, but only minimizes the possibility of using swap.

通常情况下：swap分区设置建议是内存的两倍 （内存小于等于4G时），如果内存大于4G，swap只要比内存大就行。另外尽量的将swappiness调低，这样系统的性能会更好。

B.修改swappiness参数

临时性修改： [root@wenwen ~]# sysctl vm.swappiness=10 vm.swappiness = 10 [root@wenwen ~]# cat /proc/sys/vm/swappiness 10

永久性修改：

[root@wenwen ~]# vim /etc/sysctl.conf 加入参数： vm.swappiness = 35 然后在直接： [root@wenwen ~]# sysctl -p /etc/sysctl.conf 查看是否生效： cat /proc/sys/vm/swappiness 35

立即生效，重启也可以生效。

三、怎么释放内存？

一般系统是不会自动释放内存的关键的配置文件/proc/sys/vm/drop_caches。这个文件中记录了缓存释放的参数，默认值为0，也就是不释放缓存。他的值可以为0~3之间的任意数字，代表着不同的含义：

0 – 不释放1 – 释放页缓存2 – 释放dentries和inodes3 – 释放所有缓存

实操：

很明显多出来很多空闲的内存了吧

4. How to release swap?

Premise: First of all, make sure that the remaining memory is greater than or equal to the swap usage, otherwise it will crash! According to the memory mechanism, once the swap partition is released, all files stored in the swap partition will be transferred to physical memory. Releasing swap is usually accomplished by remounting the swap partition.

a. Check where the current swap partition is mounted? b. Shut down this partition c. Check the status: d. Check whether the swap partition is shut down. The bottom line shows all e. Mount swap to /dev/sda5 f. Check whether the mounting is successful