How to use the multiprocessing module for multi-process management in Python 3.x

How to use the multiprocessing module for multi-process management in Python 3.x

Introduction:
In Python, the popularity of multi-core CPUs has made multi-process programming an important skill. The multiprocessing module is a standard library in Python for handling multi-processes. This article will introduce how to use the multiprocessing module for multi-process management, and illustrate it with code examples.

1. Introduction to the multiprocessing module
Python's multiprocessing module provides a wrapper that can map Python programs to run on multiple processes. The multiprocessing module is thread-safe and provides more functionality than the threading module.

2. Common functions and classes of the multiprocessing module

1. multiprocessing.Process class
   The multiprocessing.Process class is used to create and manage processes. It has 4 main methods:
2. start(): Start the process and call the run() method.
3. run(): The entry function of the process needs to be defined by the user.
4. join([timeout]): Wait for the process to end. If timeout is not specified, it will wait until the process ends.
5. is_alive(): Determine whether the process is still running.
6. multiprocessing.current_process() function
   multiprocessing.current_process() function returns the Process object of the current process.
7. multiprocessing.active_children() function
   The multiprocessing.active_children() function returns a list of running child processes, including only child processes created by the current process.
8. multiprocessing.Pool class
   The multiprocessing.Pool class is used to create a process pool, which can easily manage the execution of multiple processes. The most commonly used methods are:
9. apply(func, args): execute the function synchronously and return the result.
10. apply_async(func, args): Execute the function asynchronously and return the AsyncResult object.
11. close(): Close the process pool and no longer accept new tasks.
12. join(): The main process waits for all child processes to end.

3. Sample code using the multiprocessing module
The following is a simple example showing how to use the multiprocessing module for multi-process management:

import multiprocessing

def worker(name):
    print('Worker %s' % name)
    return name

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=4)
    results = []
    
    for i in range(4):
        result = pool.apply_async(worker, args=(i,))
        results.append(result)

    pool.close()
    pool.join()
    
    for result in results:
        print(result.get())

In the above code, we First, a worker function is defined, which accepts a name parameter and prints out the name. Then, use multiprocessing.Pool in the main program to create a process pool containing 4 processes. Next, we use the apply_async method to asynchronously execute the worker function, passing in a parameter i, and adding it to the results list. Finally, wait for all processes to complete execution through the pool.close() and pool.join() methods. Finally, we obtain the execution result through the result.get() method and print it out.

Executing the above code will output the following results:

Worker 0
Worker 1
Worker 2
Worker 3
0
1
2
3

Summary:
By using the multiprocessing module, we can easily perform multi-process programming. This article introduces the common functions and classes of the multiprocessing module and demonstrates how to use them through sample code. Using the multiprocessing module can better utilize multi-core CPUs and improve program execution efficiency.

Reference materials:
[1] Python official documentation - multiprocessing module. https://docs.python.org/3/library/multiprocessing.html

The above is the detailed content of How to use the multiprocessing module for multi-process management in Python 3.x. For more information, please follow other related articles on the PHP Chinese website!