Reading Last Lines of a Large File Incrementally in Go
In this scenario, we aim to read the last two lines of a large log file without loading it into memory and repeating this process every 10 seconds.
Within the provided Go code snippet:
package main import ( "fmt" "time" "os" ) const MYFILE = "logfile.log" func main() { c := time.Tick(10 * time.Second) for now := range c { readFile(MYFILE) } } func readFile(fname string){ file, err:=os.Open(fname) if err!=nil{ panic(err) }
We can enhance its functionality to achieve our goal by leveraging the file.Stat method to determine the file's size and the file.ReadAt method to read data from a specific byte offset within the file.
import ( "fmt" "os" "time" ) const MYFILE = "logfile.log" func main() { c := time.Tick(10 * time.Second) for _ = range c { readFile(MYFILE) } } func readFile(fname string) { file, err := os.Open(fname) if err != nil { panic(err) } defer file.Close() // Determine the size of the file stat, statErr := file.Stat() if statErr != nil { panic(statErr) } fileSize := stat.Size() // Assuming you know the size of each line in bytes (e.g., 62) start := fileSize - (62 * 2) // Read the last two lines from the file buf := make([]byte, 62 * 2) _, err = file.ReadAt(buf, start) if err == nil { fmt.Printf("%s\n", buf) } }
By utilizing file size information and direct byte offset reading, we can efficiently read the last two lines of the file without loading it entirely into memory and repeat this process every 10 seconds.
The above is the detailed content of How Can I Efficiently Read the Last Two Lines of a Large File in Go Every 10 Seconds?. For more information, please follow other related articles on the PHP Chinese website!