Golang concurrent programming: using Go WaitGroup to implement task queue

Golang concurrent programming: using Go WaitGroup to implement task queue

With the continuous improvement of computer performance, using multi-core processors for concurrent programming has become a must Skill. As a programming language that emphasizes concurrency, Golang provides a simple and powerful concurrency model, allowing developers to more easily utilize multi-core processors.

In Golang, WaitGroup is a mechanism used to coordinate synchronization between goroutines. It sets the counter to a non-zero value before starting the goroutine, and then decrements the counter after each goroutine completes its work. When the counter value is 0, it means that all goroutines have completed their tasks and can continue to perform subsequent operations.

In this article, we will use WaitGroup to implement a simple task queue to show how to use this mechanism to implement concurrent programming. Specifically, we will implement a simple file handling program that can handle multiple files at the same time.

First, we need to define a structure to represent the file task:

type FileTask struct {
    filename string
}

func (task *FileTask) Process() {
    // 文件处理逻辑
}

In the above code, the FileTask structure represents a file task, which contains a file name field. The Process method is used for the actual logic of processing the file.

Next, we define a task queue:

type TaskQueue struct {
    tasks []FileTask
    wg    sync.WaitGroup
}

func (queue *TaskQueue) Add(task FileTask) {
    queue.tasks = append(queue.tasks, task)
}

func (queue *TaskQueue) StartWorkers(numWorkers int) {
    for i := 0; i < numWorkers; i++ {
        go queue.worker()
    }
}

func (queue *TaskQueue) worker() {
    defer queue.wg.Done()

    for {
        task, ok := queue.getNextTask()

        if !ok {
            return
        }

        task.Process()
    }
}

func (queue *TaskQueue) getNextTask() (FileTask, bool) {
    if len(queue.tasks) == 0 {
        return FileTask{}, false
    }

    task := queue.tasks[0]
    queue.tasks = queue.tasks[1:]

    return task, true
}

func (queue *TaskQueue) Wait() {
    queue.wg.Wait()
}

The above code defines a TaskQueue structure, which contains a task queue and a WaitGroup. The Add method is used to add tasks to the queue, and the StartWorkers method starts a specified number of worker goroutines. Each worker will continue to obtain tasks from the queue and execute them until the queue is empty. The getNextTask method is used to get the next task from the queue, and returns false if the queue is empty. Finally, the Wait method is used to wait for all worker goroutines to complete their tasks.

Finally, we can use the above task queue to process files:

func main() {
    queue := TaskQueue{}

    for _, filename := range filenames {
        task := FileTask{filename: filename}
        queue.Add(task)
    }

    queue.StartWorkers(numWorkers)
    queue.Wait()
}

In the above code, we first create an empty queue, then iterate through all the file names and add each file The name is constructed into a task object and added to the queue. Next, we start a specified number of worker goroutines, which will get tasks from the queue and process them. Finally, we call the Wait method to wait for all task processing to be completed.

Through the above code examples, we can see that using Go WaitGroup to implement task queues is very simple and intuitive. We only need to define the structures of tasks and queues, and write corresponding methods to realize the distribution and execution of tasks, and to wait after all tasks are completed.

Summary:
Using Go WaitGroup to implement task queues is a simple and powerful concurrent programming technology. By rationally utilizing WaitGroup and goroutine, we can make full use of the performance of multi-core processors and implement efficient concurrent programs. Therefore, when doing Golang concurrent programming, reasonable use of WaitGroup is very helpful for implementing task queues.

The above is the detailed content of Golang concurrent programming: using Go WaitGroup to implement task queue. For more information, please follow other related articles on the PHP Chinese website!