Home > Backend Development > Golang > How Can I Efficiently Read the Last Two Lines of a Large File in Go Every 10 Seconds?

How Can I Efficiently Read the Last Two Lines of a Large File in Go Every 10 Seconds?

Patricia Arquette
Release: 2024-11-08 08:50:02
Original
785 people have browsed it

How Can I Efficiently Read the Last Two Lines of a Large File in Go Every 10 Seconds?

Reading Last Lines of a Large File Incrementally in Go

In this scenario, we aim to read the last two lines of a large log file without loading it into memory and repeating this process every 10 seconds.

Within the provided Go code snippet:

package main

import (
    "fmt"
    "time"
    "os"
)

const MYFILE = "logfile.log"

func main() {
    c := time.Tick(10 * time.Second)
    for now := range c {
        readFile(MYFILE)
    }
}

func readFile(fname string){
    file, err:=os.Open(fname)
    if err!=nil{
        panic(err)
    }
Copy after login

We can enhance its functionality to achieve our goal by leveraging the file.Stat method to determine the file's size and the file.ReadAt method to read data from a specific byte offset within the file.

import (
    "fmt"
    "os"
    "time"
)

const MYFILE = "logfile.log"

func main() {
    c := time.Tick(10 * time.Second)
    for _ = range c {
        readFile(MYFILE)
    }
}

func readFile(fname string) {
    file, err := os.Open(fname)
    if err != nil {
        panic(err)
    }
    defer file.Close()

    // Determine the size of the file
    stat, statErr := file.Stat()
    if statErr != nil {
        panic(statErr)
    }
    fileSize := stat.Size()

    // Assuming you know the size of each line in bytes (e.g., 62)
    start := fileSize - (62 * 2)

    // Read the last two lines from the file
    buf := make([]byte, 62 * 2)
    _, err = file.ReadAt(buf, start)
    if err == nil {
        fmt.Printf("%s\n", buf)
    }
}
Copy after login

By utilizing file size information and direct byte offset reading, we can efficiently read the last two lines of the file without loading it entirely into memory and repeat this process every 10 seconds.

The above is the detailed content of How Can I Efficiently Read the Last Two Lines of a Large File in Go Every 10 Seconds?. For more information, please follow other related articles on the PHP Chinese website!

source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Latest Articles by Author
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template