The system obviously has a lot of memory, but it cannot allocate a large piece of memory?

Today’s question: The system obviously has a lot of memory, but it cannot allocate a large piece of memory?

Why is this?

This question involves an aspect of memory management - memory fragmentation

What is memory fragmentation?

Memory fragmentation has appeared very early in Linux. Understanding the history of early memory fragmentation will help us understand it.

Assume that there is a piece of 32MB memory. At first, the operating system uses the smallest piece of memory - 4MB, and the remaining memory is reserved for 4 processes, as shown in Figure (a).

Process A uses 10MB of memory above the operating system, process B uses 6MB of memory above process A, and process C uses 8MB of memory above process B. , as shown in Figure (b),:

Process D requires 5MB of memory, so the remaining memory is not enough to load process D, and the last bit of this memory forms The first hole (memory fragment). Assume that at a certain moment, the operating system needs to run process D because there is not enough memory in the system, so it needs to select a process to swap out to make enough space for process D. Assume that the operating system selects process B to swap out, so that process D is loaded into the address space of the original process B, thus creating a second hole, as shown in Figure (c):

Assume that the operating system needs to run process B at a certain moment, and also needs to select a process to swap out. Assuming that process A is swapped out, then a third process will occur in the operating system holes , as shown in Figure (d):

As time goes by, there will be more and more memory holes, and the memory utilization will also increase. Decreasing, these memory holes are what we often call Memory fragmentation.

After seeing this, you already know what memory fragmentation is, and you also understand a memory management mechanism-dynamic partitioning method. The above example is actually dynamic partitioning method. In the early days of the operating system, dynamic partitioning method was used to manage memory.

How to solve the memory fragmentation problem?

The idea is actually very simple: Put multiple small blocks of memory into one large block of memory.

In early operating systems that used dynamic partitioning, in order to solve the problem of fragmentation, they dynamically moved the process so that the space occupied by the process was continuous, and all free space was also continuous, In this way, multiple small memory blocks are put together. But the shortcomings are also very obvious. Process migration takes a lot of time.

Inner fragmentation and outer fragmentation

There are two types of memory fragmentation: Inner fragmentation and External fragmentation

Inner fragmentation: The part of the memory allocated to the program but not used

External fragmentation: System Small memory blocks that cannot be used (such as the fragments generated by the above-mentioned dynamic partitioning method)

Nowadays, operating systems use paging or segmentation mechanisms to manage memory, but some memory fragments will inevitably be generated.

In order to solve the problem of internal fragmentation and external fragmentation, Linux introduced two things: Partner system and slab.

The partner system is used to solve the problem of external fragmentation, and the slab is used to solve the problem of internal fragmentation.

The partner system and slab are also core contents in memory management. If you are interested, you can study it.

Summary

So, when the system has a lot of memory but cannot allocate a large block of memory, that is Because a lot of memory fragmentation is generated, there are many discontinuous small pieces of memory in the system. On the surface, it seems that the system has a lot of free memory, but in fact it is just scattered memory.