golang log rotation

Golang Log Rotation

With the continuous development of applications, it is an inevitable problem that log files become larger and larger. For long-running applications, log files may reach several gigabytes, which creates disk space issues and causes log files to be opened and read slowly. Therefore, for a good application, reasonable management and rotation of log files is very necessary.

In Golang, some powerful logging libraries are provided, such as log package, zap, logrus, etc. What these log libraries have in common is that they all generate log files. If the log files become too large, we need to consider the rotation of the log files.

Log levels in Golang log module

Golang log module defines different log levels, such as Debug, Info, Warning, Error, Fatal and Panic, starting from the lowest level Debug. The following are their definitions:

const (
    Ldate         = 1 << iota     // 日期
    Ltime                         // 时间
    Lmicroseconds                 // 微秒时间戳
    Llongfile                     // 完整文件路径名和行号：XXXX/XXXX/line
    Lshortfile                    // 文件名和行号：line
    LUTC                          // 如果设置了 Ldate 或 Ltime，则使用 UTC 时间，否则使用本地时间
    LstdFlags     = Ldate | Ltime // 指定标准日志记录器应具有的默认标志
)
const (
    TraceLevel int = iota
    DebugLevel
    InfoLevel
    WarnLevel
    ErrorLevel
    PanicLevel
    FatalLevel
    NoLevel
)

In the log, we only need to record logs with a higher level than the current log level, because higher level logs are more urgent. In addition to helping us diagnose errors, log levels can also control what is recorded in the log. For example, we can log more information in the Debug level logger and only log information about errors or warnings in Release builds.

Rotation of log files

In Golang, rotating logs is controlled by the programmer. Common techniques for implementing log rotation are replication and compression. When a log file reaches a certain size or interval, it needs to be renamed and a new log file created.

In order to achieve log rotation, we can choose the following two methods:

1. Rotate by time: This method takes time as the axis, saves one file every day, and only records the daily log file Today's log. When the program starts the next day, a new log file is created and the old log file is compressed or deleted. The disadvantage is that there is no guarantee that the file size limit will not be exceeded within the same day.

2. Rotate by file size: This method takes the file size as the axis. Whenever the log file reaches a certain size limit, it is renamed and a new log file is created. The disadvantage is that log files cannot be distinguished based on time.

Below we will step by step introduce how to implement log rotation based on file size through Golang code.

Use Lumberjack library for log rotation

Lumberjack is a reliable log library used to rotate log files according to file size. It supports writing multiple log files in parallel and takes advantage of Go's features to avoid race conditions. Lumberjack can also automatically compress old log files, saving disk space.

Let’s look at a simple example first:

package main

import (
    "github.com/natefinch/lumberjack"
    "log"
)

func main() {
    logger := &lumberjack.Logger{
        Filename:   "./log/test.log",
        MaxSize:    5, // megabytes
        MaxBackups: 3,
        MaxAge:     28, // days
    }

    log.SetOutput(logger)

    // 测试日志轮转
    for i := 0; i < 12000; i++ {
        log.Println(i)
    }
}

In the above code, we use the Lumberjack log library for log rotation. We set the log file ./log/test.log as the output file, set the file size limit to 5 MB, the maximum number of backups to 3, and the maximum number of storage days to 28 days. When the log file size exceeds 5 MB, Lumberjack writes the data to a new file and saves the old file as a .1, .2, or .3 backup. When the number of backups exceeds 3, old backup files will be deleted. When an old log is older than 28 days, it will be automatically deleted.

After 12,000 cycles, we can see the generated log file in the ./log directory. You can see that Lumberjack has automatically divided the log files into different parts and automatically deleted some outdated log files within the specified time.

Common log rotation

The common configuration of rotating logs by time is as follows:

logger := &lumberjack.Logger{
    Filename:   logPath,
    MaxSize:    0,
    MaxBackups: 0,
    MaxAge:     7,  // 保留最近 7 天的日志
    LocalTime:  true,
    Compress:   true, // 压缩日志文件
}

The common configuration of rotating logs by file size is as follows:

logger := &lumberjack.Logger{
    Filename:   logPath,
    MaxSize:    100, // megabytes
    MaxBackups: 5,
    MaxAge:     30, // 保留最近 30 天的日志
    LocalTime:  true,
    Compress:   true, // 压缩日志文件
}

Summary

In Golang, we can use the Lumberjack log library to implement automated log rotation to solve the problem of excessively large log files. Through flexible configuration, we can select the time or file size for log rotation according to needs, and automatically delete outdated log files. This not only helps solve the problem of log files meeting minimum storage requirements, but also helps ensure that applications run smoothly and are not affected by large log files.

The above is the detailed content of golang log rotation. For more information, please follow other related articles on the PHP Chinese website!