How to use the multiprocessing module for inter-process communication in Python 3.x

How to use the multiprocessing module for inter-process communication in Python 3.x

With the development of computer technology, we often need to perform multiple tasks at the same time in programming. To take better advantage of multi-core processors, Python's multiprocessing module provides a simple yet powerful set of tools for creating concurrent programs. The multiprocessing module allows us to use multiple processes in Python, which can execute simultaneously and communicate when needed. This article will introduce how to use the multiprocessing module for inter-process communication and provide corresponding code examples.

1. Using Queue for inter-process communication
   The multiprocessing module provides a Queue class for passing data between processes. The following is an example of using Queue for inter-process communication:

from multiprocessing import Process, Queue

def worker(q):
    # 从队列中获取数据并处理
    while True:
        data = q.get()
        if data is None:
            break
        # 处理数据
        print("Worker got:", data)

if __name__ == '__main__':
    # 创建一个队列
    q = Queue()

    # 创建多个进程
    processes = []
    for i in range(3):
        p = Process(target=worker, args=(q,))
        processes.append(p)
        p.start()

    # 往队列中放入数据
    for i in range(10):
        q.put(i)

    # 添加结束标记到队列中
    for i in range(3):
        q.put(None)

    # 等待进程结束
    for p in processes:
        p.join()

In the above code, we create a worker function that obtains data from the queue through Queue and processes it. Then we created three processes, each process will execute the worker function. In the main process, we put some data into the queue and add an end marker. Finally, we wait for all processes to end.

2. Using Pipe for inter-process communication
   In addition to using Queue, the multiprocessing module also provides the Pipe class for creating bidirectional pipes between processes. The following is an example of using Pipe for inter-process communication:

from multiprocessing import Process, Pipe

def worker(conn):
    # 接收数据并打印
    data = conn.recv()
    print("Worker got:", data)

    # 发送数据回主进程
    conn.send("Hello from worker")

    # 关闭连接
    conn.close()

if __name__ == '__main__':
    # 创建一个管道
    parent_conn, child_conn = Pipe()

    # 创建子进程
    p = Process(target=worker, args=(child_conn,))
    p.start()

    # 发送数据到子进程
    parent_conn.send("Hello from main process")

    # 接收子进程的返回数据
    data = parent_conn.recv()
    print("Main process got:", data)

    # 等待子进程结束
    p.join()

In the above code, we create a worker function that receives the data sent by the main process through Pipe and prints it. Then it sends a message back to the main process. In the main process, we create a pipe and pass one of the ports to the child process. Then we send a message to the child process and receive the return data from the child process. Finally, we wait for the child process to finish.

Summary:
Using the multiprocessing module for inter-process communication is very simple. It provides two classes, Queue and Pipe, to implement data transmission between processes. The Queue class is used for one-way communication, passing data between processes through the put and get methods. The Pipe class is used for two-way communication between processes through the send and recv methods. Whether using Queue or Pipe, we can easily transfer data between different processes to achieve concurrent execution of tasks and inter-process communication.

The above is the detailed content of How to use the multiprocessing module for inter-process communication in Python 3.x. For more information, please follow other related articles on the PHP Chinese website!