Writing to mmap allocated slice panics

php editor Strawberry will introduce to you the problem of "panic occurs when writing slices allocated by mmap". When using mmap to allocate memory, if write operations are performed at the same time, a memory allocation panic may occur. In this case, the program may encounter unexpected errors or even cause the program to crash. Therefore, when using mmap to allocate memory, you need to pay attention to avoid simultaneous write operations, or use other memory allocation methods to avoid this problem. Next, we will analyze this problem in detail and provide corresponding solutions.

Question content

The scenario is to cache real-time streaming data into a file-backed memory mapped area. The goal is to make the state fault tolerant using mmap.

Using mmap in your application will avoid relying on the Redis key-value store.

The following is the program:

package main

import (
    "fmt"
    "os"

    "github.com/edsrzf/mmap-go"
)

func main() {
    f, _ := os.OpenFile("./file", os.O_RDWR, 0644)
    defer f.Close()

    mmap, _ := mmap.Map(f, mmap.RDWR, 0)
    defer mmap.Unmap()
    fmt.Println(string(mmap))

    mmap[0] = 'X'
    mmap.Flush()
}

$ go build -gcflags -m=2 -o main cmd/layer/main.go
$ ./main

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
main.main()
        /..//cmd/layer/main.go:21 +0x1d7

Why mmap[0] = 'X' cannot write data to the file?

Solution

Your file is empty, therefore your slices are empty too. You have to allocate some bytes to the file first and then try to change the slice you get from mmap.Map() or mmap.MapRegion():

package main

import (
    "os"

    "github.com/edsrzf/mmap-go"
)

var testPath = "./file"

func initFile() {
    size := int64(10 * 1024 * 1024)
    fd, err := os.Create(testPath)
    if err != nil {
        panic("Failed to create output")
    }
    _, err = fd.Seek(size-1, 0)
    if err != nil {
        panic("Failed to seek")
    }
    _, err = fd.Write([]byte{0})
    if err != nil {
        panic("Write failed")
    }
    err = fd.Close()
    if err != nil {
        panic("Failed to close file")
    }
}

func main() {
    f, _ := os.OpenFile("./file", os.O_RDWR, 0644)
    defer f.Close()

    mem, _ := mmap.Map(f, mmap.RDWR, 0)
    defer mem.Unmap()

    mem[0] = 'X'
    mem.Flush()
}

Please note that the file created in initFile() is not human readable, you can make a fixed size string and write it to the file so that your mem The content of becomes human-readable

The above is the detailed content of Writing to mmap allocated slice panics. For more information, please follow other related articles on the PHP Chinese website!