The multiprocessing module in Python allows you to create and manage processes, enabling you to take full advantage of multiple processors on a machine. It helps you achieve parallel execution by using separate memory spaces for each process, unlike threading where threads share the same memory space. Here's a list of commonly used classes and methods in the multiprocessing module with brief examples.
The Process class is the core of the multiprocessing module, allowing you to create and run new processes.
from multiprocessing import Process def print_numbers(): for i in range(5): print(i) p = Process(target=print_numbers) p.start() # Starts a new process p.join() # Waits for the process to finish
Starts the process’s activity.
p = Process(target=print_numbers) p.start() # Runs the target function in a separate process
Blocks the calling process until the process whose join() method is called terminates. Optionally, you can specify a timeout.
p = Process(target=print_numbers) p.start() p.join(2) # Waits up to 2 seconds for the process to finish
Returns True if the process is still running.
p = Process(target=print_numbers) p.start() print(p.is_alive()) # True if the process is still running
Returns the current Process object representing the calling process.
from multiprocessing import current_process def print_current_process(): print(current_process()) p = Process(target=print_current_process) p.start() # Prints the current process info
Returns a list of all Process objects currently alive.
p1 = Process(target=print_numbers) p2 = Process(target=print_numbers) p1.start() p2.start() print(Process.active_children()) # Lists all active child processes
Returns the number of CPUs available on the machine.
from multiprocessing import cpu_count print(cpu_count()) # Returns the number of CPUs on the machine
A Pool object provides a convenient way to parallelize execution of a function across multiple input values. It manages a pool of worker processes.
from multiprocessing import Pool def square(n): return n * n with Pool(4) as pool: # Pool with 4 worker processes result = pool.map(square, [1, 2, 3, 4, 5]) print(result) # [1, 4, 9, 16, 25]
A Queue is a shared data structure that allows multiple processes to communicate by passing data between them.
from multiprocessing import Process, Queue def put_data(q): q.put([1, 2, 3]) def get_data(q): data = q.get() print(data) q = Queue() p1 = Process(target=put_data, args=(q,)) p2 = Process(target=get_data, args=(q,)) p1.start() p2.start() p1.join() p2.join()
A Lock ensures that only one process can access a shared resource at a time.
from multiprocessing import Process, Lock lock = Lock() def print_numbers(): with lock: for i in range(5): print(i) p1 = Process(target=print_numbers) p2 = Process(target=print_numbers) p1.start() p2.start() p1.join() p2.join()
The Value and Array objects allow sharing simple data types and arrays between processes.
from multiprocessing import Process, Value def increment(val): with val.get_lock(): val.value += 1 shared_val = Value('i', 0) processes = [Process(target=increment, args=(shared_val,)) for _ in range(10)] for p in processes: p.start() for p in processes: p.join() print(shared_val.value) # Output will be 10
A Pipe provides a two-way communication channel between two processes.
from multiprocessing import Process, Pipe def send_message(conn): conn.send("Hello from child") conn.close() parent_conn, child_conn = Pipe() p = Process(target=send_message, args=(child_conn,)) p.start() print(parent_conn.recv()) # Receives data from the child process p.join()
A Manager allows you to create shared objects, such as lists and dictionaries, that multiple processes can modify concurrently.
from multiprocessing import Process, Manager def modify_list(shared_list): shared_list.append("New item") with Manager() as manager: shared_list = manager.list([1, 2, 3]) p = Process(target=modify_list, args=(shared_list,)) p.start() p.join() print(shared_list) # [1, 2, 3, "New item"]
A Semaphore allows you to control access to a resource, permitting only a certain number of processes to access it at a time.
from multiprocessing import Process, Semaphore import time sem = Semaphore(2) # Only 2 processes can access the resource def limited_access(): with sem: print("Accessing resource") time.sleep(2) processes = [Process(target=limited_access) for _ in range(5)] for p in processes: p.start() for p in processes: p.join()
The multiprocessing module in Python is designed to take full advantage of multiple processors on a machine. From creating and managing processes using Process, to controlling shared resources with Lock and Semaphore, and facilitating communication through Queue and Pipe, the multiprocessing module is crucial for parallelizing tasks in Python applications.
The above is the detailed content of A Quick Guide to the Python multiprocessing Module with Examples. For more information, please follow other related articles on the PHP Chinese website!