Practical combat | RISC-V Linux entry address 2M reserved memory optimization

The previous article analyzed the page table creation for RISC-V Linux startup. It was mentioned that the RISC-V Linux entry address must be 2M aligned. Today I will talk about how to solve the 2M alignment problem, or how to optimize the part. Memory.

Note: This article is based on the linux5.10.111 kernel

Memory usage analysis

When each chip leaves the factory, its bootrom has been solidified inside the chip. Assume that the address of the bootrom is 0x0, that is, after power-on, the program will start running from the 0x0 address.

Before starting RISC-V Linux, you need to run opensbi first, so opensbi should be placed at the address 0x0, so that after the chip is powered on, it will start from 0x0Execute opensbi at the address. After opensbi runs, it will jump to the location where the opensbi running address is offset by 2M to execute the next-level boot (the next-level boot here is the kernel), that is, jump to the 0x200000 address to run the kernel. Therefore, the kernel should be placed at 0x200000 in the memory.

The memory distribution diagram is as follows:

For the kernel, the page table mapping will be established starting from its own kernel loading address (i.e. 0x200000) at startup. Only the physical memory is established. Page table mapping, these memories can be accessed later. The 2M memory in front of the kernel loading address (i.e. 0x0 - 0x200000) will be ignored by the kernel, and a page table will not be established for this 2M memory, that is, the kernel cannot access this 2M memory.

Startup information of RISC-V Linux on QEMU:

But opensbi does not actually need to use a range as large as 2M, the default is 512KB , opensbi's pmp will protect this 512KB memory and prevent other programs from accessing it.

Therefore, there will be a memory gap of 1.5M between Kernel and opensbi, and this part of the memory gap is not used by the program, which will cause memory It's a waste, so how do you let the kernel use the previous part of the memory?

Optimization plan

There are two options for optimizing this 2M memory:

Option 1: Put opensbi at the end of the memory, and the kernel entry address remains 2M aligned.

Option 2: Opensbi is still placed at the starting position of the memory. By modifying the kernel source code and lifting the 2M alignment restriction, the kernel address can be moved forward.

Option 1

We put opensbi at the end of the memory, and the kernel entry address still maintains 2M alignment.

That is, the kernel is placed at the front of the memory, and opensbi is placed at the back:

For example, the kernel is placed at the address 0x0 of the memory. , opensbi is placed at the 0x10000000 address in the memory. In this way, there will be no reserved memory in front of the kernel, but needs to modify the bootrom address from 0x0 to 0x0x10000000. This solution is only suitable before the chip leaves the factory, because the user cannot modify the bootrom address. After the chip leaves the factory, the bootrom address is fixed. Assuming that the bootrom address is 0x0, then the bootrom address on the chip After powering up, the program will start running from 0x0, so opensbi must be placed at the address 0x0, so the kernel can only be offset by 2M.

Option 2

We can also modify the kernel source code of RISC-V Linux to lift the 2M alignment restriction. We only need to change the original second-level page table to the third-level page table in the setup_vm() function, so that the kernel entry address only needs to be aligned at 4K, so the kernel can be Move forward to use the memory at the front.

Modify the code

Path: arch/riscv/mm/init.c

Comment original 2M alignment check:

The mapping of the first 2M page table of the kernel is changed from the second-level page table to the third-level page table:

//新增一个PTE
pte_t trampoline_pte[PTRS_PER_PTE] __page_aligned_bss;

create_pgd_mapping(trampoline_pg_dir,PAGE_OFFSET,
                   (uintptr_t)trampoline_pmd,PGDIR_SIZE,PAGE_TABLE);
create_pmd_mapping(trampoline_pmd,PAGE_OFFSET,
                   (uintptr_t)trampoline_pte,PMD_SIZE,PAGE_TABLE);

end_va = PAGE_OFFSET + PMD_SIZE;
for (va = PAGE_OFFSET; va < end_va; va += PAGE_SIZE)
{
    create_pte_mapping(trampoline_pte,PAGE_OFFSET,
                   load_pa + (va - PAGE_OFFSET),
                       PAGE_SIZE,PAGE_KERNEL_EXEC);
}

The page table mapping of the entire kernel is changed from the second-level page table to the third-level page table:

Assume that the kernel size is 4M

//定义三个PTE
pte_t load_sz_pte[PTRS_PER_PTE] __page_aligned_bss;
pte_t load_sz_pte1[PTRS_PER_PTE] __page_aligned_bss;
pte_t load_sz_pte2[PTRS_PER_PTE] __page_aligned_bss;

//=======0-2M======
create_pgd_mapping(early_pg_dir,PAGE_OFFSET,
                   (uintptr_t)early_pmd,PGDIR_SIZE,PAGE_TABLE);
create_pmd_mapping(early_pmd,PAGE_OFFSET,
                   (uintptr_t)load_sz_pte,PMD_SIZE,PAGE_TABLE);

end_va = PAGE_OFFSET + PMD_SIZE;
for (va = PAGE_OFFSET; va < end_va; va += PAGE_SIZE)
{
    create_pte_mapping(load_sz_pte,PAGE_OFFSET,
                   load_pa + (va - PAGE_OFFSET),
                       PAGE_SIZE,PAGE_KERNEL_EXEC);
}

//=======2-4M==========
create_pgd_mapping(early_pg_dir,PAGE_OFFSET + PMD_SIZE,
                   (uintptr_t)early_pmd,PGDIR_SIZE,PAGE_TABLE);
create_pmd_mapping(early_pmd,PAGE_OFFSET,
                   (uintptr_t)load_sz_pte1,PMD_SIZE,PAGE_TABLE);

end_va = PAGE_OFFSET + (PMD_SIZE * 2);
for (va = PAGE_OFFSET + PMD_SIZE; va < end_va; va += PAGE_SIZE)
{
    create_pte_mapping(load_sz_pte1,va,
                   load_pa + (va - PAGE_OFFSET),
                       PAGE_SIZE,PAGE_KERNEL_EXEC);
}

//=======4-6M==========
create_pgd_mapping(early_pg_dir,PAGE_OFFSET + (PMD_SIZE*2),
                   (uintptr_t)early_pmd,PGDIR_SIZE,PAGE_TABLE);
create_pmd_mapping(early_pmd,PAGE_OFFSET,
                   (uintptr_t)load_sz_pte2,PMD_SIZE,PAGE_TABLE);

end_va = PAGE_OFFSET + (PMD_SIZE * 3);
for (va = PAGE_OFFSET + (PMD_SIZE*2); va < end_va; va += PAGE_SIZE)
{
    create_pte_mapping(load_sz_pte2,va,
                   load_pa + (va - PAGE_OFFSET),
                       PAGE_SIZE,PAGE_KERNEL_EXEC);
}

Through the above code modification, the Kernel entry address can be moved forward by 1.5M, and only 512KB is reserved for opensbi. In this way, after RISC-V Linux starts, Available physical memory will increase.

Summary

RISC-V Linux entry address 2M alignment operation is not yet available I saw someone explaining it, but it should be to reserve 2M for opensbi, so the kernel only established a secondary page table, so that the entry address must be aligned with 2M. No one has yet given an optimization solution for this part of memory. I hope the optimization solution in this article can help some people and give you some inspiration.